Skip to content

HTTPS clone URL

Subversion checkout URL

You can clone with
or
.
Download ZIP
Fetching contributors…

Cannot retrieve contributors at this time

451 lines (397 sloc) 16.785 kB
m4_include(/mcs/m4/worksp.lib.m4)
_NIMBUS_HEADER(2.10 - Developer reference)
_NIMBUS_HEADER2(n,n,y,n,n,n,n)
_NIMBUS_LEFT2_COLUMN
_NIMBUS_LEFT2_DEV1_SIDEBAR(n,n,n,n,n,n,n,n,y)
_NIMBUS_LEFT2_COLUMN_END
_NIMBUS_CENTER2_COLUMN
_NIMBUS_IS_DEPRECATED
<div>
<h2>Developer reference</h2>
<p>
This section is for miscellaneous information of use to Nimbus developers.
</p>
<ul>
<li>
<p>
<a href="#code-layout">Layout of the source directories</a>
</p>
</li>
<li>
<p>
<a href="#pilot-fake">Pilot development environment</a>
</p>
</li>
<li>
<p>
<a href="#change-wsdl">Changing the WSDL</a>
</p>
</li>
</ul>
<br />
<a name="code-layout"> </a>
<h2>Layout of the source directories _NAMELINK(code-layout)</h2>
<p>
The code lives under _PATH(workspace/vm) in CVS (see
<a href="sccs.html">here</a> for details), all directory names will be
referenced with the assumption that this is being prepended. So if
_PATH(backend/workspace) is referenced, in CVS that will be
_PATH(workspace/vm/backend/workspace)
</p>
<p>
The source code has been through many changes but in some cases the
decision was made to preserve the CVS history rather than moving the
directory to a more organized place. This is not a set in stone
decision... whatever works best.
</p>
<p>
While reading the following notes, it will help to have
<a href="../faq.html#nimbus-main-components">this picture</a>
in mind.
</p>
_MINI_SEP
<p>
The core of Nimbus is the <a href="../faq.html#rm-api">RM API</a>
which lives in the _PATH(service-api/java/source) directory.
</p>
<p>
If you navigate there you will find many things that are common to
Nimbus source directories.
</p>
<p>
There is a construct borrowed from other Globus services: the
component's hierarchy is first split by the language it is implemented
in. So the first directory here is _PATH(java). Then there is
a directory for source code and a directory for tests.
</p>
<p>
Under _PATH(source) are the _PATH(build.xml) and
_PATH(build.properties) files which are specific to the component.
Running "ant dist" in any Nimbus directory
with a _PATH(build.xml) file is usually all you need to do in order to
build that component.
</p>
<p>
Though a much better option is to use the _PATH(scripts) scripts for building.
The build file that the _PATH(scripts) scripts call is:
_PATH(scripts/lib/gt4.0/build/build.xml)
</p>
<p>
That file will build components with the proper dependencies. On many
projects, source trees have their build.xml files call on other things
to compile, etc., but this source tree's build.xml files -- if used
directly -- will only look for dependencies in other directories. If
those dependencies are not built, then the build fails. This lets the
developer do what he or she wants to do with maximum control when
mucking with build files directly. The scripts in the "bin" directory
are what both users and developers will use on a regular basis.
</p>
<p>
The RM API has no dependencies outside of this tree. It contains its
own "dummy" implementations of the APIs which do nothing. If you
instantiate the API (it bootstraps a
<a href="http://www.springframework.net/doc-latest/reference/html/objects.html">Spring
inversion of control</a> container), it will use the embedded
configuration as its default. This does nothing, it will just print
things to the logger (which can be useful in some situations, for
example when developing a new protocol implementation).
</p>
_MINI_SEP
<p>
Moving along, _PATH(service/service/java/source) contains the
<a href="../faq.html#workspace-service">Workspace Service</a>
site manager implementation.
</p>
<p>
That implements the RM API. Together, these two trees could be used
in any number of Java containers. There is no other dependency.
</p>
<p>
In the _PATH(service/service/java/source/etc/workspace-service/other/main.xml)
file you will find the Spring configurations that are used to instantiate
the workspace service. There are a number of internal plug points etc.,
this is how that is all arranged. The _PATH(.conf) files are never examined
directly by the service, they are sucked into the Spring configuration
using the magic of PropertyPlaceholderConfigurer. Open the
_PATH(main.conflocator.xml) file next to _PATH(main.xml) to see how that works.
</p>
<p>
The service may call on some components that are not contained in the
same tree but instead live in their own top level directories:
</p>
<ol>
<li>
<p>
_PATH(control) contains the standalone
<a href="../faq.html#wcontrol">workspace control</a> program
that is installed to the VMM nodes.
</p>
</li>
<li>
<p>
_PATH(pilot) contains the standalone
<a href="../faq.html#wpilot">workspace pilot</a> which is
submitted to a local batch scheduler (in some deployments) in
order to reserve resources (VMMs). <i><b>(advanced)</b></i>
</p>
</li>
<li>
<p>
_PATH(plugins) contains plugin implementations for the workspace
service. These are broken out from the service tree mainly
because they require dependencies (such as Jython) that are
bulky and/or licensed in ways that may not be acceptable to
people that need pure BSD/Apache style licensings.
<i><b>(advanced)</b></i>
</p>
</li>
</ol>
_MINI_SEP
<p>
Next thing to understand is the _PATH(messaging) directory.
</p>
<p>
While not necessary, the RM API (and the main workspace service
implementation of it) are intended to be used with a remote messaging
implementation. Some kind of protocol which is implemented by some kind
of WS/REST/binary conventions and hosted by some kind of container
technology (in all likelihood).
</p>
<p>
The _PATH(messaging) directory contains two protocol implementations, each
of them run in the Axis based Globus 4.0.x Java container. As you can
see from the subdirectories, the container glue and protocol implementations
are intertwined. When making a new permutation of protocol and
container, you typically need to worry about both at the same time to
make it work.
</p>
<p>
The _PATH(messaging/gt4.0/java/stubs) directory contains the auto-generated
Axis stub classes that are used to marshall/demarshall the WSRF protocol
based Nimbus messages.
</p>
<p>
The _PATH(messaging/gt4.0/java/msgbridge) directory contains everything
necessary to actually take messages off the wire, initially consume
them, and translate them into RM API calls.
</p>
<p>
The _PATH(messaging/gt4.0-elastic) directory contains similar things for the
EC2 WS protocol hosted in the Globus 4.0.x Java container.
</p>
<p>
The _PATH(messaging/gt4.0/java/gar-builder) directory contains the
scripts that produce the "final
packaging." That is, this is the master directory for building a
"GAR" file which is what is actually deployed into the Globus 4.0.x Java
container. Both the WSRF and "elastic" interfaces are stuffed into
this GAR creation process, as well as any dependency (i.e., the RM API
and any of its dependencies ... and the workspace service and any of
its dependencies).
</p>
<p>
So the final product for the service (the GAR files) that is deployed
is something of an onion. The layers around the outside are very
specific to the deployment and wire technology but as you peel things
away you become more and more generic.
</p>
<p>
A related thing to understand is that the initialization sequence works
in the same way. The container boots, the Axis service is initialized
(because the "loadOnStartup" configuration is true), the service is tied
in with the container's JNDI system, a JNDI configuration points to the
Spring configuration, and then the Spring configuration takes over. 99%
of the configuration is done via Spring, but something needs to kick
everything off and via Container->JNDI is how it works.
</p>
_MINI_SEP
<p>
Next up, something needs to call the service. The
_PATH(service/client/java/source) directory is where all of the client code
lives (EC2 clients work too).
</p>
<p>
Navigate to _PATH(service/client/java/source/src/org/globus/workspace).
There are several different layers here. _PATH(client_core) is where all of
the code is to make actual calls to the service, it is a collection of
convenience wrappers around the web services stubs and the returned
data structures (and notifications/polling helpers). These wrappers
make it easier to use the different operations as "building blocks" for
higher level actions.
</p>
<p>
The _PATH(client) package (sibling of _PATH(client_core)) is where the "reference
client" is implemented. This is a commandline wrapper around the
_PATH(client_core) basic actions. It has several "modes" which are each an
"orchestration" of various actions. This client can handle any operation
the service supports and provides many useful utilities for getting
things done in the expected order ("create, then monitor", etc.).
</p>
<p>
Both the _PATH(client) and the _PATH(client_core) classes are ripe for
inserting into portals and other new types of clients.
</p>
<p>
The _PATH(cloud) package, however, is not geared towards programmatic
re-use (parts of it could be, but they were not intentionally written
for this). This is another commandline program but
it is fully geared towards human use. There are many default parameters
and behaviors that are
simplifications of what is possible with the Nimbus API, but definitely
right for users that are getting started or only have the typical needs.
</p>
<p>
The cloud client is packaged separately, and this Java code is in it,
it gets installed (via "ant deploy" in the _PATH(service/client/java/source)
directory) into the embedded GLOBUS_LOCATION in the cloud client
(_PATH(lib/globus)).
</p>
_MINI_SEP
<p>
The _PATH(autoconfiguaration) directory contains programs and scripts that
run the configuration wizard for installing <i>and maintaining</i> the
server side, the "administrator wizard".
</p>
<p>
This is
contained in the GAR files when Nimbus is installed to a container, it
is installed into the _PATH($GLOBUS_LOCATION/share/nimbus-autoconfig)
directory and is a standalone thing. It asks questions and can alter
some of the configuration files, its sole purpose is to make it so the
user has to understand less to get things up and running initially.
</p>
<p>
The _PATH(autocommon) directory contains libraries and code that are
used by _PATH(nimbus-configure --autoconfig)
</p>
_MINI_SEP
<p>
That's the bird's eye view. Ask questions about any of this on the
<a href="_NIMBUS_WEBSITE/contact/#dev">developer's list</a>.
</p>
<br />
<br />
<br />
<a name="pilot-fake"> </a>
<h2>Pilot development environment _NAMELINK(pilot-fake)</h2>
<p>
The following steps will allow you to develop the <b>service</b> portion of the pilot setup without actually needing an LRM installed or hypervisors.
</p>
<ol>
<li>
Activate the pilot plugin, copy "resource-locator-pilot.xml" to "resource-locator-ACTIVE.xml"
</li>
<li>
Edit that, add any string here (e.g. "fake") in order to activate SSH based notifications:
<div class="screen">
&lt;property name="sshNotificationInfo" value="fake" /&gt;
</div>
</li>
<li>Edit "services/etc/nimbus/workspace-service/pilot.conf"
<div class="screen">
pbs.submit.path=/tmp/fakeqsub
pbs.delete.path=/bin/true
</div>
</li>
<li>
Create "/tmp/fakeqsub"
<div class="screen">
#!/bin/sh
sleep 3
echo "asdsadasd" # This is the LRM job ID.
</div>
</li>
<li>
Edit "services/container-log4j.properties"
<div class="screen">
log4j.category.org.nimbustools=DEBUG
log4j.category.org.globus.workspace=DEBUG
</div>
</li>
<li>
Edit "services/etc/nimbus/workspace-service/logging.conf"
<div class="screen">
log.state=on
log.trace=on
</div>
Or in some cases you might even want to enable the scheduler logging in
"services/etc/nimbus/workspace-service/other/main.xml"
<div class="screen">
&lt;property name="scheduler" value="on" /&gt;
</div>
</li>
<li>
Start service in fake mode (edit "services/etc/nimbus/workspace-service/other/common.conf" )
</li>
<li>Start a vm with cloud client</li>
<li>grep logs for "pilot command" and note the "-i" parameter, e.g. "1e372dca-de93-4fd9-a125-9d66842804cc". That is the SLOT ID assigned to the LRM job. If you are launching many pilots at once (i.e., a cluster of 50 VMs, but one LRM job), each pilot process will report back with the <b>same id</b> but different hostname.
</li>
<li>
Simulate reserved notification:
<div class="screen">
<b>$</b> ./services/var/nimbus/msg-sinks/pilotnotifications write 1e372dca-de93-4fd9-a125-9d66842804cc+++localhost pilot-reserved 0 2010-11-07-16-12-01
</div>
<ul>
<li>
<p>arg1 ("1e372dca-de93-4fd9-a125-9d66842804cc+++localhost")</p>
<p>"1e372dca-de93-4fd9-a125-9d66842804cc" is slot id from service logs</p>
<p>"+++" is separator</p>
<p>"localhost" is hostname where pilot ended up running</p>
</li>
<li><p>arg2 state, e.g. pilot-reserved</p></li>
<li><p>arg3, return code, e.g. 0</p></li>
<li><p>arg4, timestamp, YYYY-MM-DD-HH-MM-SS (always UTC/GMT)</p></li>
</ul>
</li>
</ol>
<br />
<a name="change-wsdl"> </a>
<h2>Changing the WSDL _NAMELINK(change-wsdl)</h2>
<p>
This section discusses making changes to WSDL and generating new stub
classes, etc. It assumes you have some understanding of the layers and
source code directions discussed in the <a href="#code-layout">code layout
section</a>.
</p>
<p>
First run _PATH(scripts/gt/all-clean.sh).
</p>
<p>
Then edit the "compact" WSDL in the
_PATH(messaging/gt4.0/schema/compact/workspace/) directory. Note that the
EC2 WSDL works the same way but we are using the WSRF ones as an example.
</p>
<p>
Compact WSDL
is a Globus convention, it facilitates an inheritance mechanism that allows
the final WSDL to contain common WSRF/WSN related operations. You edit the
compact WSDL only and then a program will "compile" that into the real
WSDL/schemas.
</p>
<p>
Once you have edited something, the changes will not be picked up by the
build system until you have run "ant" in the
_PATH(messaging/gt4.0/schema/compact/) directory. This triggers the default
ant target of "copyToDeployableComponent" which will take the "compact" files
and put the (generated) real WSDL/schemas under the
_PATH(messaging/gt4.0/schema/dist/) directory.
</p>
<p>
That directory is what the _PATH(stubs-build.sh) script uses to generate
the stubs (the Axis classes that are actually referenced by the Nimbus
code for un/marshalling of the web services).
</p>
<p>
That creates the new wsdl. Now you need to create the auto-generated "stub"
code Java jars. Do this by running "scripts/gt/stubs-build.sh"
</p>
<p>
Unless you added something entirely new, the corresponding messaging layer
(e.g., _PATH(messaging/gt4.0/java/msgbridge)) will probably not compile
now. So the next step is to move up the stack and make it work.
</p>
</div>
_NIMBUS_CENTER2_COLUMN_END
_NIMBUS_FOOTER1
_NIMBUS_FOOTER2
_NIMBUS_FOOTER3
Jump to Line
Something went wrong with that request. Please try again.