Using GFac to wrap an application as a service

Launching applications on Super Computers and managing the application during its lifetime and its output after it ends are challenging tasks. GFac aims at being able to provide simple GUI tools that can help launch and manage the application on remote supercomputers.

This post aims at serving as a quickstart to configuring a XSul based GFac persistent service.

The first step is to install GFac and XRegistry components. Good documentation to do this can be found on the OGCE web pages.


After the installation is complete set the GFAC_HOME environment variable to point to the GFac's target/dist-bin folder:

export GFAC_HOME=/Users/archit/ogce/ogce-xsul-services-1.0-RC1/sgfac/target/dist-bin

Now create a folder to house the configuration files that will be used by our service.

cd ogce-xsul-services-1.0-RC1/sgfac
mkdir helloworld
cd helloworld

Now use your favorite editor to create the three XML configuration files needed by GFac to start the persistent service namely the App description document, the host description document and service description document.

Application Description File helloworldapp.xml:

<ApplicationDescription xmlns="http://www.extreme.indiana.edu/namespaces/2004/01/gFac" xmlns:lead="http://www.extreme.indiana.edu/lead">
<applicationName
targetNamespace="http://www.extreme.indiana.edu/lead">HelloWorldApp</applicationName>
<jobType>single</jobType>
<project> <projectName>TG-XXXXXXXXXX</projectName>
<projectDescription>
TG staff
</projectDescription>
</project>
<queue>
<queueName>SERIAL</queueName>
<queueDescription>
BigRed queue
</queueDescription>
</queue>
<maxWallTime>30</maxWallTime>
<count>1</count>
<rslparm name="project">TG-XXXXXXXXXX</rslparm>
<rslparm name="queue">SERIAL</rslparm>
<deploymentDescription>
<hostName>
bigred.iu.teragrid.org
</hostName>
<executable>/N/u/tg-archi/BigRed/helloworld</executable>
<workdir>
/N/u/tg-archi/BigRed/gfactmp
</workdir>
<!-- Tmp directory for this application.
if not specified, the tmp directory of the host will be used-->
<tmpDir>
/N/u/tg-archi/BigRed/gfactmp
</tmpDir>
<applicationEnv>
<name>FIRST_NAME</name>
<value>Archit
</value>
</applicationEnv>
</deploymentDescription>
</ApplicationDescription>

ServiceDescription file: helloworldsvc.xml

<ServiceMap xmlns="http://www.extreme.indiana.edu/namespaces/2004/01/gFac"
xmlns:lead="http://www.extreme.indiana.edu/lead">
<service>
<serviceName targetNamespace="http://www.extreme.indiana.edu/lead">HelloWorldService</serviceName>
</service>
<lifeTime>
<notAfterInactiveMinutes>-1</notAfterInactiveMinutes>
</lifeTime>
<portType>
<method stageOutputDataFiles="true" forceFileStagingToWorkDir="true">
<methodName>Run</methodName>
<application paramValuesOnly="true" useLEADNameListFile="true">
<applicationName targetNamespace="http://www.extreme.indiana.edu/lead">HelloWorldApp</applicationName>
</application>
<inputParameter>
<parameterName>lastname</parameterName>
<parameterType>String</parameterType>
</inputParameter>
<inputParameter>
<parameterName>inputfile</parameterName>
<parameterType>URI</parameterType>
</inputParameter>
<outputParameter>
<parameterName>friendCount</parameterName>
<parameterType>String</parameterType>
</outputParameter>
<outputParameter>
<parameterName>friendListFile</parameterName>
<parameterType>URI</parameterType>
</outputParameter>
</method>
</portType>
</ServiceMap>

Host Description File: bigred.xml

<HostDescription xmlns="http://www.extreme.indiana.edu/namespaces/2004/01/gFac" xmlns:lead="http://www.extreme.indiana.edu/lead">
<documentInfo>
<documentName targetNamespace="http://www.extreme.indiana.edu/lead">BigRed</documentName>
</documentInfo>
<hostName>bigred.iu.teragrid.org</hostName>
<hostConfiguration>
<tmpDir>/N/u/tg-archi/BigRed/gfactmp</tmpDir>
<globusGatekeeper wsGram="true">
<endPointReference>
https://gatekeeper.bigred.iu.teragrid.org:8443/wsrf/services/ManagedJobFactoryService
</endPointReference>
<jobmanagertype>Loadleveler</jobmanagertype>
</globusGatekeeper>

<gridFTP>
<endPointReference>gridftp.bigred.iu.teragrid.org:2811</endPointReference>
</gridFTP>
</hostConfiguration>
</HostDescription>

Create the service properties file:
Finally, create the GFac service property file: helloworld.properties. This will tell GFac where to get the service, application, and host description files you just made.
serviceMapFile = helloworld/helloworldsvc.xml
appDescFile = helloworld/helloworldapp.xml
hostDescFile = helloworld/bigred.xml
port = 12346
#Comment this to make Factory and Services starts up without https
#transportSecurity=true
#private key used by the GFac to connect to other services, and start https socket, if it is enabled
ssl.hostcertsKeyFile=/home/ogce/ogce-xsul-services-1.0-RC1/sgfac/target/dist-bin/conf/ogce_services_key.pem
ssl.trustedCertsFile=/home/ogce/ogce-xsul-services-1.0-RC1/sgfac/target/dist-bin/conf/trusted_cas.pem
registryURL=https://<yourip>:6666/xregistry?wsdl
#Ask Gfac to go to local mode, so it can work without credentails. Comment if you need a remote installation
installation=local
myproxyServer=myproxy.teragrid.org
myproxyUserName=XXXXX
myproxyLifetime=86400
myproxyPasswd=XXXXXX
#gfac.prefferedProvider=Local|Gram|WSGram|SSH|ResourceBroker
wsgramPrefered=true

Now with your JAVA_HOME and GFAC_HOME set start the persisitent HelloWorldService using the run.sh provided with GFac as foolows:

./run.sh helloworld/helloworld.properties

This will start the service on port 12345 and you are ready to submit HelloWorld jobs.