This project is the backend for the tool integration tool (TITO). It allows the front-end to interact with both the OSM and the Discovery Environment through a single service.
This code is being released purely to be more open. A complete rewrite in Clojure is currently in development. This code is primarily intended as a reference.
Zoidberg is packaged as a an RPM. This RPM depends on node-launch, which is another project that allows node.js apps to run as background processes. node-launch is written in Python 2.6, which means you should have the EPEL repository enabled when installing zoidberg.
Here are the steps needed to deploy Zoidberg:
-
Enable the EPEL repository for CentOS 5.X. It's available here - http://fedoraproject.org/wiki/EPEL.
-
Make sure the iPlant yum repo appropriate for your environment is enabled.
-
Install Zoidberg via yum.
sudo yum install iplant-zoidberg
-
Edit the configuration file located at /etc/iplant-zoidberg.conf (described below).
-
(Optional, but recommended) Enable the service with chkconfig.
-
Start the service
sudo /sbin/service iplant-zoidberg start
-
Check to make sure it's running
sudo /sbin/service iplant-zoidberg status
And:
ps aux | grep node
Zoidberg talks to both the OSM and the DE, so you'll need hostnames and ports for both of those. Additionally, you'll need to know the hostname and port of the box where Zoidberg itself will be running.
By default, the configuration file will look something like this:
{ "de_base" : "", "osm_base" : "", "osm_bucket" : "zoidberg", "port" : "10000", "logfile" : "zoidberg.log",
"de_services" : {
"/components" : "/get-workflow-elements/components",
"/formats" : "/get-workflow-elements/formats",
"/info-types" : "/get-workflow-elements/info-types",
"/meta-elements" : "/get-workflow-elements/all",
"/property-types" : "/get-workflow-elements/property-types",
"/rule-types" : "/get-workflow-elements/rule-types",
"/value-types" : "/get-workflow-elements/value-types",
"/preview" : "/preview-template",
"/only-analysis-groups/{email}" : "/get-only-analysis-groups/{email|0}"
},
"analysis_publisher" : "/update-template"
}
The osm_base setting contains the base URL used to connect to the OSM. The port option at the top-level of the JSON document is the option that sets the port that Zoidberg itself listens to. The de_base setting contains the base URL used to connect to the DE to access services.
The de_services section contains endpoint definitions that are forwarded to the discovery environment. The keys in this associative array are URL patterns that are used to access Zoidberg endpoints. The values are URL patterns that are used to access corresponding services in the DE. The simplest case is a one-to-one mapping for plain URLs. For example, "/foo": "/bar". A more complex case is where components in the key are mapped to the value. For example, "/foo/{id}": "/bar/{id}". In this example, the second path component from the source URL will be mapped to the second path component in the target URL. There can be any number of path components and the path components are associated by name. For example, if the source path is "/foo/{id}/{num}", the target path is "/bar/{num}/{id}" and someone hits the Zoidberg service using the path "/foo/12/144" then the request will be forwarded to the DE using the path "/bar/144/12".
In addition to field mapping, the last component in a URL can be treated as optional and a default value may be specified in the target URL. Default values in the target URL are indicated by placing a vertical bar followed by the default value after the field name For example, if the source path is "/foo/{id}", the target path is "/bar/{id|27}" and someone hits the Zoidberg service using the path "/foo" then the request will be forwarded to the DE using the path "/bar/27".
-
in-progress: GET - gets one or more existing integrations from the OSM. If the TITO ID is specified in the query string then just that analysis is returned. If the user ID is specified in the query string then all of the integrations for that user are returned. Summary information can also be requested by adding the "summary" query string parameter. Examples:
curl 'http://services-1:31310/in-progress?user=dennis' curl 'http://services-1:31310/in-progress?tito=BEA5C47C-77EF-8775-40E8-7B3606F46A2E' curl 'http://services-1:31310/in-progress?user=dennis&summary=true'PUT - stores a new integration in the OSM. The request body should be in the same format as the preview JSON generated by TITO. This service has no query string arguments.
Example POST Body: { "groups":{ "id":"--root-PropertyGroupContainer--", "name":"", "label":"List multiple files", "description":"", "type":"", "isVisible":true, "groups":[ { "id":"6210175A-1519-BB35-3E84-149953A07E4B", "name":"", "label":"Options", "description":"", "type":"", "isVisible":true, "groups":[ ], "properties":[ { "id":"C72A151E-6970-2EF5-F13C-81BF04DFF881", "name":"-l ", "label":"Verbose", "description":"", "type":"Flag", "isVisible":true, "value":"true", "order":2 } ] } ] }, "input":[ { "id":"6B2A2560-3DF0-FEF4-B64A-AF4435A332DF", "name":"Files to list", "multiplicity":"Many", "type":"Input File", "cmdSwitch":"", "order":1, "file_info_type_id":"0900E992-3BBD-4F4B-8D2D-ED289CA4E4F1", "file_info_type":"File", "output_filename":"", "format_id":[ ], "format":[ ], "required":false } ], "output":[ ], "id":"CF11C0E6-5AED-59DE-BDB2-AB656321FEC2", "name":"List multiple files", "desc":"", "component_id":"c21b275668831473e82d4ee692da12a89", "component":"ls", "TYPE":"", "tito":"CF11C0E6-5AED-59DE-BDB2-AB656321FEC2", "user":"ipctest" } Note that the component ID specified in the JSON document must be known to the DE before the integration can be imported into the DE. To get a list of known component identifiers, use curl to hit the /components endpoint described below. Also note that the component ID does not affect the ability to PUT or POST to this endpoint; it only affects the ability to actually import the analysis into the DE via the /analysis-import endpoint. Here's an example curl command: curl -X PUT -d @/path/to/template.json http://services-1:31310/in-progressPOST - updates an existing integration in the OSM. The request body should be in the same format as the preview JSON generated by TITO. This service has no query string arguments. Because the Java libraries that we use don't allow DELETE requests to have bodies, if there is an "action" element in the request body that contains the value, "delete", the update is considered to be equivalent to a DELETE request.
The JSON format to use for POST requests is the same as the format for PUT requests except that the top-level JSON object may have an additional "action" key with a value of "delete" to indicate that the integration should be marked as deleted. Here's an example curl command: curl -d @/path/to/template.json http://services-1:31310/in-progressDELETE - removes an existing integration from the OSM. The request body should be in the same format as the preview JSON generated by TITO. Here's an example curl command:
curl -d @/path/to/template.json http://services-1:31310/in-progress -
/analysis-import
GET - publishes the analysis with the given TITO ID to the DE. The TITO ID is specified using the tito query string parameter.
curl 'http://services-1:31310/analysis-import?id=BEA5C47C-77EF-8775-40E8-7B3606F46A2E'PUT - this is a synonym for a POST request for this service.
POST - either creates or updates the analysis in the OSM and publishes the analysis to the DE. the request body should be in the same format as the preview JSON generated by TITO.
The format of the request body for this endpoint is the same as that of the /in-progress endpoint except that the "action" element in the top-level object is always ignored for this endpoint. -
/make-public
POST - marks an analysis as public in the OSM. TITO uses this flag to indicate whether or not the user is permitted to modify the analysis further. The TITO ID and user name are both provided in the query string. No request body is expected.
curl -X POST 'http://services-1:31310/make-public?user=ipctest&tito=4479508D-E3E1-41E6-8B24-54EE9B3E27A4' -
/components
GET - retrieves the list of deployed components from the DE. This is a proxy service for http://de-host:14445/get-workflow-elements/components.
-
/formats
GET - retrieves the list of known data formats from the DE. This is a proxy service for http://de-host:14445/get-workflow-elements/formats.
-
/info-types
GET - retrieves the list of known information types from the DE. This is a proxy service for http://de-host:14445/get-workflow-elements/info-types.
-
/meta-elements
GET - retrieves the lists of all known metadata elements from the DE. This is a proxy service for http://de-host:14445/get-workflow-elements/all
-
/property-types
GET - retrieves the list of all known property types from the DE. This is a proxy service for http://de-host:14445/get-workflow-elements/property-types
-
/rule-types
GET - retrieves the list of all known rule types from the DE. This is a proxy service for http://de-host:14445/get-workflow-elements/rule-types.
-
/preview
POST - converts JSON in the format that is consumed by the template import service to the format that is required by the DE. The request body format used by this endpoint is the same as that of the /in-progress and /analysis-import endpoints.
-
/only-analysis-groups/{email}
GET - retrieves the list of analysis categories without including the analyses contained in those analysis groups. The e-mail component of the path is optional. If the e-mail address is not provided then only the public analysis groups will be listed.