GCE10 Workshop Accepted for Supercomputing 2010

The Gateway Computing Environments workshop (GCE10) has been accepted at SC10. The workshop will be held in room 387 in the New Orleans Convention center on Sunday, Nov 14^th from 9-5:30.  More details coming soon. For previous proceedings of this series, please see http://www.collab-ogce.org/gce10/index.php/Main_Page.

Condor jDRMAA Example

Notes on using the DRMAA java implementation to submit Condor Jobs.

The Condor jDRMAA implementation can be downloaded from their sourceforge page using svn

svn co https://condor-jdrmaa.svn.sourceforge.net/svnroot/condor-jdrmaa

and use ant to build the jar or alternatively download the jar from

http://sourceforge.net/projects/condor-jdrmaa/files/condor-jdrmaa/0.2/condor-drmaa-0.2.jar/download

Version 0.2 uses the condor command line clients to submit and manage jobs.

Hence you must make sure that the condor binaries are on path and CONDOR_CONFIG is set if your Condor config file is not in /etc/condor.

The notes here pertain to Condor 7.4.2 on Mac OS X but should work on any platform that supports Condor and Java.

140-182-140-134:0.3 archit$ condor_version

$CondorVersion: 7.4.2 Mar 29 2010 BuildID: 227044 $

$CondorPlatform: I386-OSX_10_4 $

Here is the Java Class I used to perform the Job Submission.

Note that it is in the package org.ogce.jobsub.api

To replicate my setup do

mkdir -p org/ogce/jobsub/api

Create org/ogce/jobsub/api/DRMAAJobSubmission.java with the following using your favorite editor.

package org.ogce.jobsub.api;

import java.util.ArrayList;

import java.util.LinkedList;

import java.util.List;

import java.util.Set;

import org.ggf.drmaa.*;

import net.sf.igs.*;

public class DRMAAJobSubmission {

public static void main(String[] args) {

Session session = null;

String jobPath = "/bin/echo";

session = SessionFactory.getFactory().getSession();

String jobId = null;

JobTemplate jt = null;

JobInfo status = null;

ArrayList argslist = new ArrayList();

argslist.add("Hello");

try {

System.out.println(session.getDrmSystem());

System.out.println(session.getContact());

System.out.println(session.getDrmaaImplementation());

session.init("localhost");

jt = session.createJobTemplate();

jt.setRemoteCommand(jobPath);

jt.setWorkingDirectory("/tmp");

jt.setJoinFiles(true);

jt.setOutputPath("/tmp/hello.out" );

jt.setArgs(argslist);

jt.setTransferFiles(new FileTransferMode(false, false, false));

} catch (DrmaaException e) {

e.printStackTrace();

}

try {

jobId = session.runJob(jt);

status = session.wait(jobId, Session.DONE);

if (status.wasAborted()) {

System.out.println("job " + jobId + " never ran");

} else if (status.hasExited ())

{

System.out.println("job " + jobId +

"finished successfully with Exit Status "

+ status.getExitStatus());

} else if (status.hasSignaled ())

{

System.out.println("job " + jobId + " finished due to signal" +

status.getTerminatingSignal ());

} else

{

System.out.println("job " + jobId +

" finished with unclear conditions");

}

} catch (DrmaaException e) {

System.out.println("Internal Exception: " + e.getMessage());

e.printStackTrace();

}

catch (org.ggf.drmaa.InternalException ie)

{

System.out.println("Internal Exception: " + ie.getMessage());

ie.printStackTrace();

}

}

}

To Compile the file use

javac -cp ./condor-drmaa-0.2.jar org/ogce/jobsub/api/DRMAAJobSubmission.java

and to submit the echo job do

java -cp ./condor-drmaa-0.2.jar:. org.ogce.jobsub.api.DRMAAJobSubmission

This assumes you have the condor-drmaa jar in the current folder.

You should see something like

Condor

Condor

job 25.0 finished successfully with Exit Status 0

Session.init("localhost") call is used to initiate the Condor DRM adaptor and tell it to connect to the scheduler running locally.

The JobTemplate class is used to create the job description which gets translated into a condor submit file which is used as an argument to condor_submit.

The Session class has a number of static members that are used to indicate Job Status. The JobInfo class returned by Session.wait is used to retrieve the current status of the job after it has been submitted using the session.runJob method. Session.wait takes as its second argument the time until which the call must block. In this case the call blacks until the job Completes as indicated by Session.DONE.

In Order to specify job attributes not supported by the Getters and Setters we can use the setNativeSpecification method of the JobTemplate class.

So In order to submit a Condor-G job all we need to do is add the line

jt.setNativeSpecification("Universe=Grid \n globusscheduler=qb1.loni.org/jobmanager");

before submitting the job.

Please note that the attributes must be separated by a new line (\n) character so that the submit file is generated correctly. Also it must be noted that jDRMAA sets the Universe to Vanilla by default, but since the Native Specification is written in the submit file after the Universe=Vanilla line, it can be used to override the Universe setting.