GCE11 Workshop Paper Submission Deadline Extension

Papers for the GCE11 workshop at Supercomputing 2011 can be submitted through October 3rd, 2011. See http://www.collab-ogce.org/gce11 for more details. 

OGCE Workflow Tools Now in Apache Airavata

The OGCE's tools for managing scientific applications and workflows on grids and clouds are now the Apache Airavata incubator project.  Community contributions are welcome, and you can become part of the project.  We are now following the Apache Software Foundation model for building open source communities.  

GCE11 Workshop

The Gateway Computing Environments 2011 workshop will be at Supercomputing 2011.  Submissions are open and welcome.  See http://www.collab-ogce.org/gce11/index.php/Main_Page  for more information.

Positions available

The OGCE team at Indiana University has positions available.  See https://jobs.iu.edu/joblisting/index.cfm?jlnum=4193&search=2 for more information.  If you are interested, please apply through Jobs@IU.  If you would like more information, contact Marlon: marpierc [at] indiana edu.

OGCE Gadget Container GridShib Support

The OGCE gadget container now includes GridShib support as part of the default build.  We use GridShib to decorate TeraGrid community security credentials with logging attributes: the authentication instant, the user's email address, the user's browser's IP address, and the user's gadget container user name.  All installation and configuration steps are handled by the OGCE build process: "mvn clean install" handles the jar deployments and resource configuration files.  Decorated credentials can be used or exported to other services.

GridShib is optional, and gadget container deployments not using it will work as normal.

OGCE Gadget Container Part of Apache Rave Incubator

The OGCE Gadget Container is being contributed to the Apache Rave incubator project. The ultimate goal is to produce high quality software for running production Open Social containers.  See http://wiki.apache.org/incubator/RaveProposal for more information.  We will continue to develop the OGCE gadget container in the interim until Rave is ready to release.  We will also extend the Rave core to accommodate the requirements of Science Gateways.

TG10 OGCE Paper

The official version of our TeraGrid 10 paper, "Open grid computing environments: advanced gateway support activities", is now available through the ACM:

http://portal.acm.org/citation.cfm?id=1838590&CFID=6079067&CFTOKEN=84307850

OGCE SciDAC2010 Tutorial

SciDAC 2010 tutorial slides.

OGCE SciDAC2010 Tutorial

View more presentations from marpierc.

OGCE TeraGrid 2010 ASTA Support

Presentation on advanced support activities by the IU Science Gateway Group at TeraGrid 10 (accepted presentation).

OGCE TeraGrid 2010 ASTA Support

View more presentations from marpierc.

OGCE TeraGrid 2010 Science Gateway Tutorial Intro

Tutorial intro slides at TG10 in Pittsburgh, August 2-5, 2010

OGCE TeraGrid 2010 Science Gateway Tutorial Intro

View more presentations from marpierc.

Building Science Gateways with Gadgets and OpenSocial

Check out this SlideShare Presentation:

Building Science Gateways with Gadgets and OpenSocial

View more presentations from marpierc.

Twitter, SlideShare

OGCE's Twitter feed is http://twitter.com/ogceproject.  We have many slide presentations at SlideShare: http://www.slideshare.net/marpierc/presentations.

OGCE's XBaya Running a GridChem Workflow

The movie shows the XBaya workflow composer creating and executing a computational chemistry workflow in the GridChem Science Gateway.

OGCE Demos and Presentations at SC10 in New Orleans

Please stop by and see us at SC10 November 15-18 in New Orleans at the following times and locations:

• We have a mini-workshop Tuesday from 10:00 am - 12:00 noon at the IU booth.
• Monday-Thursday at the IU Booth, we have walk-up demos on demand at the interactive touch screen kiosk.
• Monday-Thursday at the Purdue Booth, we also have on demand demos.
• Wednesday from 1:00-2:00 pm, we have on demand presentations and demos at the NCSA booth.

GCE10 Program Now Available

The GCE10 workshop program is now available from http://www.collab-ogce.org/gce10/index.php/Program.  GCE10 is being held Sunday, November 14th at SC10 in New Orleans.  Pre-prints and slides will be made available from the GCE10 web site.

GCE10 Deadline Extension

The new submission deadline for GCE10 papers is September 13th.  The call for papers is available from www.collab-ogce.org.

TG10 Tutorial Agenda

Here is the agenda for the OGCE tutorial at TG10 on Monday, August 2nd.  Tutorial material will be available for download.

1:00-2:15  

• Introduction
• computational chemistry workflow demo
• build demo

2:15-2:30: Break
2:30-5:00 

• Hands on demo of OREChem workflow (using XBaya composer)
• Hands data mining tutorial (using XRegistry and Experiment Builder gadgets)
• Gadget container overview

Agenda is subject to improvisation based on audience interest and participation.

Sample Build Instructions Movie for OGCE Portal and Services

This link shows how to get the OGCE code base and build the portal and services: http://ogceportal.iu.teragrid.org/tutorials/CodeBuild.htm

GCE10 Workshop Accepted for Supercomputing 2010

The Gateway Computing Environments workshop (GCE10) has been accepted at SC10. The workshop will be held in room 387 in the New Orleans Convention center on Sunday, Nov 14^th from 9-5:30.  More details coming soon. For previous proceedings of this series, please see http://www.collab-ogce.org/gce10/index.php/Main_Page.

Condor jDRMAA Example

Notes on using the DRMAA java implementation to submit Condor Jobs.

The Condor jDRMAA implementation can be downloaded from their sourceforge page using svn

svn co https://condor-jdrmaa.svn.sourceforge.net/svnroot/condor-jdrmaa

and use ant to build the jar or alternatively download the jar from

http://sourceforge.net/projects/condor-jdrmaa/files/condor-jdrmaa/0.2/condor-drmaa-0.2.jar/download

Version 0.2 uses the condor command line clients to submit and manage jobs.

Hence you must make sure that the condor binaries are on path and CONDOR_CONFIG is set if your Condor config file is not in /etc/condor.

The notes here pertain to Condor 7.4.2 on Mac OS X but should work on any platform that supports Condor and Java.

140-182-140-134:0.3 archit$ condor_version

$CondorVersion: 7.4.2 Mar 29 2010 BuildID: 227044 $

$CondorPlatform: I386-OSX_10_4 $

Here is the Java Class I used to perform the Job Submission.

Note that it is in the package org.ogce.jobsub.api

To replicate my setup do

mkdir -p org/ogce/jobsub/api

Create org/ogce/jobsub/api/DRMAAJobSubmission.java with the following using your favorite editor.

package org.ogce.jobsub.api;

import java.util.ArrayList;

import java.util.LinkedList;

import java.util.List;

import java.util.Set;

import org.ggf.drmaa.*;

import net.sf.igs.*;

public class DRMAAJobSubmission {

public static void main(String[] args) {

Session session = null;

String jobPath = "/bin/echo";

session = SessionFactory.getFactory().getSession();

String jobId = null;

JobTemplate jt = null;

JobInfo status = null;

ArrayList argslist = new ArrayList();

argslist.add("Hello");

try {

System.out.println(session.getDrmSystem());

System.out.println(session.getContact());

System.out.println(session.getDrmaaImplementation());

session.init("localhost");

jt = session.createJobTemplate();

jt.setRemoteCommand(jobPath);

jt.setWorkingDirectory("/tmp");

jt.setJoinFiles(true);

jt.setOutputPath("/tmp/hello.out" );

jt.setArgs(argslist);

jt.setTransferFiles(new FileTransferMode(false, false, false));

} catch (DrmaaException e) {

e.printStackTrace();

}

try {

jobId = session.runJob(jt);

status = session.wait(jobId, Session.DONE);

if (status.wasAborted()) {

System.out.println("job " + jobId + " never ran");

} else if (status.hasExited ())

{

System.out.println("job " + jobId +

"finished successfully with Exit Status "

+ status.getExitStatus());

} else if (status.hasSignaled ())

{

System.out.println("job " + jobId + " finished due to signal" +

status.getTerminatingSignal ());

} else

{

System.out.println("job " + jobId +

" finished with unclear conditions");

}

} catch (DrmaaException e) {

System.out.println("Internal Exception: " + e.getMessage());

e.printStackTrace();

}

catch (org.ggf.drmaa.InternalException ie)

{

System.out.println("Internal Exception: " + ie.getMessage());

ie.printStackTrace();

}

}

}

To Compile the file use

javac -cp ./condor-drmaa-0.2.jar org/ogce/jobsub/api/DRMAAJobSubmission.java

and to submit the echo job do

java -cp ./condor-drmaa-0.2.jar:. org.ogce.jobsub.api.DRMAAJobSubmission

This assumes you have the condor-drmaa jar in the current folder.

You should see something like

Condor

Condor

job 25.0 finished successfully with Exit Status 0

Session.init("localhost") call is used to initiate the Condor DRM adaptor and tell it to connect to the scheduler running locally.

The JobTemplate class is used to create the job description which gets translated into a condor submit file which is used as an argument to condor_submit.

The Session class has a number of static members that are used to indicate Job Status. The JobInfo class returned by Session.wait is used to retrieve the current status of the job after it has been submitted using the session.runJob method. Session.wait takes as its second argument the time until which the call must block. In this case the call blacks until the job Completes as indicated by Session.DONE.

In Order to specify job attributes not supported by the Getters and Setters we can use the setNativeSpecification method of the JobTemplate class.

So In order to submit a Condor-G job all we need to do is add the line

jt.setNativeSpecification("Universe=Grid \n globusscheduler=qb1.loni.org/jobmanager");

before submitting the job.

Please note that the attributes must be separated by a new line (\n) character so that the submit file is generated correctly. Also it must be noted that jDRMAA sets the Universe to Vanilla by default, but since the Native Specification is written in the submit file after the Universe=Vanilla line, it can be used to override the Universe setting.