Gadget Container, Registry, and Experiment Builder Movie Links

The links below show movie tutorials for the OGCE Gadget Container and two major new gadgets in the "ogce-all" release described in the previous post. 

• OGCE Gadget Container: portal software for running secure Open Social gadgets:  http://ogceportal.iu.teragrid.org/tutorials/Gadget_Container.htm
• XRegistry Gadget: a gadget for registering host computers and scientific application services into the OGCE XRegistry service: http://ogceportal.iu.teragrid.org/tutorials/XregistryInterface.htm
• Experiment Builder Gadget and XBaya Workflow Composer: the experiment builder gadget allows you to build, execute, and monitor experiments from services and workflow stored in XRegistry: http://ogceportal.iu.teragrid.org/tutorials/ExperimentBuilder.htm

OGCE "All" Release Candidate

An initial release candidate of the next generation of the OGCE software stack is now available.  The download is available by anonymous SVN checkout using the command.

svn co  https://ogce.svn.sourceforge.net/svnroot/ogce/ogce-all-rc1

This is a preview release that will need to undergo additional acceptance testing, but it is stable for early users.  Future updates can be obtained using the usual "svn update" command, and the release can be modularly rebuilt.

This download integrates several software components:

1. The OGCE Gadget Container: an iGoogle-style portal container for science gateways.
2. Experiment Builder and XRegistry gadgets for interfacing with the OGCE GFAC and XRegistry services and the XBaya workflow composer
3. XRegistery and GFAC services for registering, launching, monitoring, and sharing application services and workflows.
4. A collection of third party gadgets.

Everything can be built with a single command: mvn clean install.  Maven profiles also allow you to build subcomponents: 

1. mvn clean install -Dservices-only builds only GFAC and XBaya.  These services can run on separate servers from the gadget container, and you can run more than one instance of these services.
2. mvn clean install -Dgadget-container builds just the gadget container.
3. mvn clean install -Dgadgets-and-clients builds the gadget container, registry gadget, and experiment builder gadget.

Updated TeraGrid CA Files

(Reposting from general TeraGrid mailing lists)

Changes have been made to the list of Teragrid recognized Certificate Authorities.

The new CA tarball, v1.10, can be found on repo. This tarball adds the new PSC MyProxy CA, with hash value 4b2783ac. You can obtain a copy of the tarball and signatures file from the software WG site at:
http://software.teragrid.org/security/teragrid-certs.tar.gz
http://software.teragrid.org/security/teragrid-certs.tar.gz.asc

Please incorporate the new list as specified (RPs, Science Gateways and users within 24 hours)

New OGCE Discussion List, Google Group

We have created a new, open group for OGCE discussion questions and announcements.  The group URL is http://groups.google.com/group/ogce-discuss, and the group email address is ogce-discuss@googlegroups.com. This email address replaces the older discuss@ogce.org mailing list.

You are welcome to join, and we also allow anonymous viewing and moderated posts by group non-members.  Only project managers can see the member list.

The project RSS feed is http://groups.google.com/group/ogce-discuss/feed/rss_v2_0_msgs.xml.

Build and Deploy to Maven2 Repo

OGCE components use Maven2 for build and Archiva for repository management. For legacy reasons some dependencies are exist in extreme lab maven repository. To upgrade these components and release a version and deploy a jar file to extreme lab maven 2 repository, add the following distributionManagement tag to the pom.xml.

<distributionmanagement>
<repository>
<uniqueversion>false</uniqueversion>
<id>xRepo</id>
<name>Extreme Lab Repository</name>
<url>scp://mneme.extreme.indiana.edu/l/extreme/java/repository</url>
<layout>default</layout>
</repository>
</distributionmanagement>

mvn deploy will deploy the generated jar files to the repository but will prompt to enter the password and will also assume local account name. To avoid password prompt and to specify a different user name the following settings.xml can be placed in the $HOME/.m2 directory. For security reasons, never include the settings.xml in any of the packages, this file should be protected by file system security and is strongly advised to set the file permissions to 600.

<settings xmlns="http://maven.apache.org/SETTINGS/1.0.0" xsi="http://www.w3.org/2001/XMLSchema-instance" schemalocation="http://maven.apache.org/SETTINGS/1.0.0 http://maven.apache.org/xsd/settings-1.0.0.xsd">
<servers>
<server>
<id>xRepo</id>
<username>smarru</username>
<privatekey>${user.home}/.ssh/id_dsa</privatekey> <!-- if ssh keys are not available the password can be set as <password>my_password</password> -->
<passphrase>some_passphrase</passphrase>
<filepermissions>664</filepermissions>
<directorypermissions>775</directorypermissions>
<configuration></configuration>
</server>
</servers>
</settings>

Gadget Container Theme Customization

The OGCE gadget container has been updated to simplify theme customization. See http://www.collab-ogce.org/ogce/index.php/OGCE_Gadget_Container_Theme_Customization for more information.

OGCE Messaging Service Released

We are pleased to announce the release of our Axis 2 based Messaging Service. This is a standalone publish/subscribe service that is also being integrated into the OGCE workflow suite.

For more information and download instructions, see http://www.collab-ogce.org/ogce/index.php/Messaging.

OGCE Simple Monitoring Script

A simple perl based service monitoring script is available to check out from sourceforge svn at Service Monitoring Dashboard. This is a cleaned up version of the LEAD SOA Status Page authored by Suresh Marru and Marcus Christie.

The perl script periodically pings the registered services and if a service is done, sends an emails to the registered contacts. The scripts also writes our a html page with the following color convention:

• Green - Service is up
• Red - Service is down
• Yellow - Request timed out (i.e., took longer than the default 20 seconds)

The OGCE Team has deployed and will maintain the script. The status of the services can be viewed at Monitoring Dash Board. All services are pinged every 10 mins and the page is refreshed. Additionally, XSUL based services are instrumented with finer details like uptime and total requests served. In the future Axis2 based services will also be explored for more instrumented data. OGCE hosted gateway (demo/tutorial) services as well as gateway services will be monitored. To add a service to the monitoring list, send an email to {smarru or mpierce} at cs.indiana.edu (Note that only project supported gateways will be monitored).

Gateways can checkout the script and deploy and maintain on their own servers. To add a service to the script, use the following template:

push (@services, {
$NAME => "OGCE Gateway Portal",
$ID => "OGCEPortal",
$URL => "https://ogceportal.iu.teragrid.org:8443/gridsphere/gridsphere",
$CONTACTS => [admin@gateway.org],
$GATEWAYS_R_US => $OGCE
});

The GATEWAYS_R_US is a variable to categorize the services. For instance, to generate a separate listing of OGCE Hosted Services and a list of Gateway Services. The perl script is commented to explain these terminologies. The documentation is very naive but will be enhanced more.

Questions or comments, email smarru at cs.indiana.edu

Overview of OGCE Science Gateway Support

The slideshow below was presented at the Indiana University Pervasive Technology Institute and focuses on IU OGCE activities.  For fuller overviews, see for example http://www.collab-ogce.org/ogce/index.php/Tutorials  
OGCE's Advanced Science Gateway Support

View more presentations from marpierc.

Analytics Reports for OGCE Web Site

Images below are from our Google Analytics traffic tracking from March 1, 2009-March 1, 2010. The first shows overall site traffic. The second shows global traffic sources. Click the image to see a larger version.

Download Java Server Faces Components for Amazon Cloud Services

We have Amazon EC2 user interface components built with Java Server Faces available for download or SVN checkout.  

The tar.gz version is available from http://ogce.svn.sourceforge.net/viewvc/ogce/ogce-amazon-gadgets/awsGadget.tar.gz?view=tar.

To check the code out from SVN, use the command

svn checkout  https://ogce.svn.sourceforge.net/svnroot/ogce/ogce-amazon-gadgets/awsGadget

This will create a directory $HOME/awsGadget.  To build the gadgets, use the command 

mvn clean install -Dcatalina.home=/path/to/your/tomcat

For example, if you download (separately) Apache Tomcat 6.0.18 and put it in your awsGadget directory, you would run the command

mvn clean install -Dcatalina.home=./apache-tomcat-6.0.18/  

Start your tomcat server and point your browser to http://localhost:8080/AWSGadget/index.jsp.

Note the login page requires your Amazon secret key, so you will need to run this over HTTPS.  This code is provided as is under our usual SourceForge license.  

OGCE Jira URL

We are opening up our Jira project and bug tracking system for anonymous viewing.  The URL is http://grids.ucs.indiana.edu:8080/jira.

Using GFac to wrap an application as a service

Launching applications on Super Computers and managing the application during its lifetime and its output after it ends are challenging tasks. GFac aims at being able to provide simple GUI tools that can help launch and manage the application on remote supercomputers.

This post aims at serving as a quickstart to configuring a XSul based GFac persistent service.

The first step is to install GFac and XRegistry components. Good documentation to do this can be found on the OGCE web pages.


After the installation is complete set the GFAC_HOME environment variable to point to the GFac's target/dist-bin folder:

export GFAC_HOME=/Users/archit/ogce/ogce-xsul-services-1.0-RC1/sgfac/target/dist-bin

Now create a folder to house the configuration files that will be used by our service.

cd ogce-xsul-services-1.0-RC1/sgfac
mkdir helloworld
cd helloworld

Now use your favorite editor to create the three XML configuration files needed by GFac to start the persistent service namely the App description document, the host description document and service description document.

Application Description File helloworldapp.xml:

<ApplicationDescription xmlns="http://www.extreme.indiana.edu/namespaces/2004/01/gFac" xmlns:lead="http://www.extreme.indiana.edu/lead">
<applicationName
targetNamespace="http://www.extreme.indiana.edu/lead">HelloWorldApp</applicationName>
<jobType>single</jobType>
<project> <projectName>TG-XXXXXXXXXX</projectName>
<projectDescription>
TG staff
</projectDescription>
</project>
<queue>
<queueName>SERIAL</queueName>
<queueDescription>
BigRed queue
</queueDescription>
</queue>
<maxWallTime>30</maxWallTime>
<count>1</count>
<rslparm name="project">TG-XXXXXXXXXX</rslparm>
<rslparm name="queue">SERIAL</rslparm>
<deploymentDescription>
<hostName>
bigred.iu.teragrid.org
</hostName>
<executable>/N/u/tg-archi/BigRed/helloworld</executable>
<workdir>
/N/u/tg-archi/BigRed/gfactmp
</workdir>
<!-- Tmp directory for this application.
if not specified, the tmp directory of the host will be used-->
<tmpDir>
/N/u/tg-archi/BigRed/gfactmp
</tmpDir>
<applicationEnv>
<name>FIRST_NAME</name>
<value>Archit
</value>
</applicationEnv>
</deploymentDescription>
</ApplicationDescription>

ServiceDescription file: helloworldsvc.xml

<ServiceMap xmlns="http://www.extreme.indiana.edu/namespaces/2004/01/gFac"
xmlns:lead="http://www.extreme.indiana.edu/lead">
<service>
<serviceName targetNamespace="http://www.extreme.indiana.edu/lead">HelloWorldService</serviceName>
</service>
<lifeTime>
<notAfterInactiveMinutes>-1</notAfterInactiveMinutes>
</lifeTime>
<portType>
<method stageOutputDataFiles="true" forceFileStagingToWorkDir="true">
<methodName>Run</methodName>
<application paramValuesOnly="true" useLEADNameListFile="true">
<applicationName targetNamespace="http://www.extreme.indiana.edu/lead">HelloWorldApp</applicationName>
</application>
<inputParameter>
<parameterName>lastname</parameterName>
<parameterType>String</parameterType>
</inputParameter>
<inputParameter>
<parameterName>inputfile</parameterName>
<parameterType>URI</parameterType>
</inputParameter>
<outputParameter>
<parameterName>friendCount</parameterName>
<parameterType>String</parameterType>
</outputParameter>
<outputParameter>
<parameterName>friendListFile</parameterName>
<parameterType>URI</parameterType>
</outputParameter>
</method>
</portType>
</ServiceMap>

Host Description File: bigred.xml

<HostDescription xmlns="http://www.extreme.indiana.edu/namespaces/2004/01/gFac" xmlns:lead="http://www.extreme.indiana.edu/lead">
<documentInfo>
<documentName targetNamespace="http://www.extreme.indiana.edu/lead">BigRed</documentName>
</documentInfo>
<hostName>bigred.iu.teragrid.org</hostName>
<hostConfiguration>
<tmpDir>/N/u/tg-archi/BigRed/gfactmp</tmpDir>
<globusGatekeeper wsGram="true">
<endPointReference>
https://gatekeeper.bigred.iu.teragrid.org:8443/wsrf/services/ManagedJobFactoryService
</endPointReference>
<jobmanagertype>Loadleveler</jobmanagertype>
</globusGatekeeper>

<gridFTP>
<endPointReference>gridftp.bigred.iu.teragrid.org:2811</endPointReference>
</gridFTP>
</hostConfiguration>
</HostDescription>

Create the service properties file:
Finally, create the GFac service property file: helloworld.properties. This will tell GFac where to get the service, application, and host description files you just made.
serviceMapFile = helloworld/helloworldsvc.xml
appDescFile = helloworld/helloworldapp.xml
hostDescFile = helloworld/bigred.xml
port = 12346
#Comment this to make Factory and Services starts up without https
#transportSecurity=true
#private key used by the GFac to connect to other services, and start https socket, if it is enabled
ssl.hostcertsKeyFile=/home/ogce/ogce-xsul-services-1.0-RC1/sgfac/target/dist-bin/conf/ogce_services_key.pem
ssl.trustedCertsFile=/home/ogce/ogce-xsul-services-1.0-RC1/sgfac/target/dist-bin/conf/trusted_cas.pem
registryURL=https://<yourip>:6666/xregistry?wsdl
#Ask Gfac to go to local mode, so it can work without credentails. Comment if you need a remote installation
installation=local
myproxyServer=myproxy.teragrid.org
myproxyUserName=XXXXX
myproxyLifetime=86400
myproxyPasswd=XXXXXX
#gfac.prefferedProvider=Local|Gram|WSGram|SSH|ResourceBroker
wsgramPrefered=true

Now with your JAVA_HOME and GFAC_HOME set start the persisitent HelloWorldService using the run.sh provided with GFac as foolows:

./run.sh helloworld/helloworld.properties

This will start the service on port 12345 and you are ready to submit HelloWorld jobs.

Web Server DNS Updates Complete

The DNS entry for the host server of the OGCE and GCE Workshop websites has been updated.  The usual URLs (http://www.collab-ogce.org, etc) should work correctly.

OGCE Web Site Problems

We are changing the IP address of the Web server that virtually hosts the www.collab-ogce.org web site.  You can access the OGCE site with this URL:  

http://gf18.ucs.indiana.edu/ogce/index.php/Main_Page

The GCE08 and 09 workshops can be reached at

http://gf18.ucs.indiana.edu/gce08/index.php/Main_Page

and

http://gf18.ucs.indiana.edu/gce09/index.php/Main_Page

OGCE Cyberaide JavaScript Release

The SC09 stable release of Cyberaide JavaScript is available from http://www.collab-ogce.org/ogce/index.php/JavaScript_COG.  You can either check the code out from SVN or download the tar:

SVN Checkout Command: svn checkout https://cyberaide.googlecode.com/svn/tags/cyberaide-javascript-SC09

Tar download link: http://www.collab-ogce.org/ogce/downloads/cyberaide-javascript-SC09.tar.gz

OGCE Gadget Container Added to Nightly Build and Test

The OGCE Gadget Container has been added to the NMI nightly build and test system.  To see a dashboard of all tests, go to http://nmi-s003.cs.wisc.edu/nmi/index.php?page=results%2Foverview&opt_project=OGCE.

OGCE Gadget Container REST API

Notes by Gerald Guo, the developer of the gadget container. We are developing a REST API for the gadget container that allows you to work with it programmatically.  The following example shows how to add a gadget to a layout using curl.  If you prefer, a tool like the Firefox plugin Poster can be used.

Curl example:
1) Authentication
    curl -H Connection:keep-alive -d "screenname=testuser&password=testuser"
-v -k -o output.txt -c cookies
https://gf13.ucs.indiana.edu:7443/ishindig-webapp/signin

2) Add a new gadget (Calender gadget)

First, create a file named "gadget.json" with following content:
{ "gadgetname":"Embedded Calendar",
"gadgetspecsrc":"https://gf13.ucs.indiana.edu:7443/gadgets-repo/EmbeddedCale
ndar.xml"}

Then execute this command:
curl -d @gadget.json -H Content-Type:application/json -H
Connection:keep-alive -v -k -c cookies -b cookies
'https://gf13.ucs.indiana.edu:7443/ishindig-webapp/rest/users/@self/layouts/
index/0/tabs/index/0/columns/index/2/gadgets/index/-1'

In the long POST URL, position (consisting of tuple
gadget index>) of the new gadget is specified.
    layouts/index/0  : this specifies the layout the gadget will be added to
    tabs/index/0     : this specifies the tab the gadget will be added to
    columns/index/2  : this specifies the column the gadget will be added to
    gadgets/index/-1 : this specifies position of the gadget within
containing column. -1 means appending.

Then you can log in OGCE gadget container via web interface
(https://gf13.ucs.indiana.edu:7443/ishindig-webapp/www/sign.jsp) to check
whether it has been added. 

This feature is currently only available in the SVN trunk.  It is not part of the SC09 tagged release.

XBaya Workflow Composer as Mashup Builder

WS02 is using and extending XBaya to work as a mashup composer: http://kkpradeeban.blogspot.com/2009/11/this-screen-cast-shows-development-of.htm.

 

OGCE Gadget Container Release

The SC09 tagged release of the OGCE Gadget Container is now available.  See http://www.collab-ogce.org/ogce/index.php/OGCE_Gadget_Container for download links and more information.  Features include

• Several gadgets pre-configured in the build, including RSS readers, GMail, Google Talk, TeraGrid status monitoring gadgets, and more.
• Support for Social Networking using Google Friend Connect.
  
• Embedded Apache Shindig container.  
• Support for OAuth-secured gadgets.
• Support for OpenID login.
• Administration interfaces.
• Alternate layouts and skins.
• Drag-and-drop gadget layouts.