This repository has been archived by the owner on Sep 23, 2020. It is now read-only.
-
Notifications
You must be signed in to change notification settings - Fork 82
/
reference.html
450 lines (397 loc) · 16.4 KB
/
reference.html
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57
58
59
60
61
62
63
64
65
66
67
68
69
70
71
72
73
74
75
76
77
78
79
80
81
82
83
84
85
86
87
88
89
90
91
92
93
94
95
96
97
98
99
100
101
102
103
104
105
106
107
108
109
110
111
112
113
114
115
116
117
118
119
120
121
122
123
124
125
126
127
128
129
130
131
132
133
134
135
136
137
138
139
140
141
142
143
144
145
146
147
148
149
150
151
152
153
154
155
156
157
158
159
160
161
162
163
164
165
166
167
168
169
170
171
172
173
174
175
176
177
178
179
180
181
182
183
184
185
186
187
188
189
190
191
192
193
194
195
196
197
198
199
200
201
202
203
204
205
206
207
208
209
210
211
212
213
214
215
216
217
218
219
220
221
222
223
224
225
226
227
228
229
230
231
232
233
234
235
236
237
238
239
240
241
242
243
244
245
246
247
248
249
250
251
252
253
254
255
256
257
258
259
260
261
262
263
264
265
266
267
268
269
270
271
272
273
274
275
276
277
278
279
280
281
282
283
284
285
286
287
288
289
290
291
292
293
294
295
296
297
298
299
300
301
302
303
304
305
306
307
308
309
310
311
312
313
314
315
316
317
318
319
320
321
322
323
324
325
326
327
328
329
330
331
332
333
334
335
336
337
338
339
340
341
342
343
344
345
346
347
348
349
350
351
352
353
354
355
356
357
358
359
360
361
362
363
364
365
366
367
368
369
370
371
372
373
374
375
376
377
378
379
380
381
382
383
384
385
386
387
388
389
390
391
392
393
394
395
396
397
398
399
400
401
402
403
404
405
406
407
408
409
410
411
412
413
414
415
416
417
418
419
420
421
422
423
424
425
426
427
428
429
430
431
432
433
434
435
436
437
438
439
440
441
442
443
444
445
446
447
448
449
450
m4_include(/mcs/m4/worksp.lib.m4)
_NIMBUS_HEADER(2.8 - Developer reference)
_NIMBUS_HEADER2(n,n,y,n,n,n,n)
_NIMBUS_LEFT2_COLUMN
_NIMBUS_LEFT2_DEV1_SIDEBAR(n,n,n,n,n,n,n,n,y)
_NIMBUS_LEFT2_COLUMN_END
_NIMBUS_CENTER2_COLUMN
_NIMBUS_IS_DEPRECATED
<div>
<h2>Developer reference</h2>
<p>
This section is for miscellaneous information of use to Nimbus developers.
</p>
<ul>
<li>
<p>
<a href="#code-layout">Layout of the source directories</a>
</p>
</li>
<li>
<p>
<a href="#pilot-fake">Pilot development environment</a>
</p>
</li>
<li>
<p>
<a href="#change-wsdl">Changing the WSDL</a>
</p>
</li>
</ul>
<br />
<a name="code-layout"> </a>
<h2>Layout of the source directories _NAMELINK(code-layout)</h2>
<p>
The code lives under _PATH(workspace/vm) in CVS (see
<a href="sccs.html">here</a> for details), all directory names will be
referenced with the assumption that this is being prepended. So if
_PATH(backend/workspace) is referenced, in CVS that will be
_PATH(workspace/vm/backend/workspace)
</p>
<p>
The source code has been through many changes but in some cases the
decision was made to preserve the CVS history rather than moving the
directory to a more organized place. This is not a set in stone
decision... whatever works best.
</p>
<p>
While reading the following notes, it will help to have
<a href="../faq.html#nimbus-main-components">this picture</a>
in mind.
</p>
_MINI_SEP
<p>
The core of Nimbus is the <a href="../faq.html#rm-api">RM API</a>
which lives in the _PATH(service-api/java/source) directory.
</p>
<p>
If you navigate there you will find many things that are common to
Nimbus source directories.
</p>
<p>
There is a construct borrowed from other Globus services: the
component's hierarchy is first split by the language it is implemented
in. So the first directory here is _PATH(java). Then there is
a directory for source code and a directory for tests.
</p>
<p>
Under _PATH(source) are the _PATH(build.xml) and
_PATH(build.properties) files which are specific to the component.
Running "ant dist" in any Nimbus directory
with a _PATH(build.xml) file is usually all you need to do in order to
build that component.
</p>
<p>
Though a much better option is to use the _PATH(scripts) scripts for building.
The build file that the _PATH(scripts) scripts call is:
_PATH(scripts/lib/gt4.0/build/build.xml)
</p>
<p>
That file will build components with the proper dependencies. On many
projects, source trees have their build.xml files call on other things
to compile, etc., but this source tree's build.xml files -- if used
directly -- will only look for dependencies in other directories. If
those dependencies are not built, then the build fails. This lets the
developer do what he or she wants to do with maximum control when
mucking with build files directly. The scripts in the "bin" directory
are what both users and developers will use on a regular basis.
</p>
<p>
The RM API has no dependencies outside of this tree. It contains its
own "dummy" implementations of the APIs which do nothing. If you
instantiate the API (it bootstraps a
<a href="http://www.springframework.net/doc-latest/reference/html/objects.html">Spring
inversion of control</a> container), it will use the embedded
configuration as its default. This does nothing, it will just print
things to the logger (which can be useful in some situations, for
example when developing a new protocol implementation).
</p>
_MINI_SEP
<p>
Moving along, _PATH(service/service/java/source) contains the
<a href="../faq.html#workspace-service">Workspace Service</a>
site manager implementation.
</p>
<p>
That implements the RM API. Together, these two trees could be used
in any number of Java containers. There is no other dependency.
</p>
<p>
In the _PATH(service/service/java/source/etc/workspace-service/other/main.xml)
file you will find the Spring configurations that are used to instantiate
the workspace service. There are a number of internal plug points etc.,
this is how that is all arranged. The _PATH(.conf) files are never examined
directly by the service, they are sucked into the Spring configuration
using the magic of PropertyPlaceholderConfigurer. Open the
_PATH(main.conflocator.xml) file next to _PATH(main.xml) to see how that works.
</p>
<p>
The service may call on some components that are not contained in the
same tree but instead live in their own top level directories:
</p>
<ol>
<li>
<p>
_PATH(control) contains the standalone
<a href="../faq.html#wcontrol">workspace control</a> program
that is installed to the VMM nodes.
</p>
</li>
<li>
<p>
_PATH(pilot) contains the standalone
<a href="../faq.html#wpilot">workspace pilot</a> which is
submitted to a local batch scheduler (in some deployments) in
order to reserve resources (VMMs). <i><b>(advanced)</b></i>
</p>
</li>
<li>
<p>
_PATH(plugins) contains plugin implementations for the workspace
service. These are broken out from the service tree mainly
because they require dependencies (such as Jython) that are
bulky and/or licensed in ways that may not be acceptable to
people that need pure BSD/Apache style licensings.
<i><b>(advanced)</b></i>
</p>
</li>
</ol>
_MINI_SEP
<p>
Next thing to understand is the _PATH(messaging) directory.
</p>
<p>
While not necessary, the RM API (and the main workspace service
implementation of it) are intended to be used with a remote messaging
implementation. Some kind of protocol which is implemented by some kind
of WS/REST/binary conventions and hosted by some kind of container
technology (in all likelihood).
</p>
<p>
The _PATH(messaging) directory contains two protocol implementations, each
of them run in the Axis based Globus 4.0.x Java container. As you can
see from the subdirectories, the container glue and protocol implementations
are intertwined. When making a new permutation of protocol and
container, you typically need to worry about both at the same time to
make it work.
</p>
<p>
The _PATH(messaging/gt4.0/java/stubs) directory contains the auto-generated
Axis stub classes that are used to marshall/demarshall the WSRF protocol
based Nimbus messages.
</p>
<p>
The _PATH(messaging/gt4.0/java/msgbridge) directory contains everything
necessary to actually take messages off the wire, initially consume
them, and translate them into RM API calls.
</p>
<p>
The _PATH(messaging/gt4.0-elastic) directory contains similar things for the
EC2 WS protocol hosted in the Globus 4.0.x Java container.
</p>
<p>
The _PATH(messaging/gt4.0/java/gar-builder) directory contains the
scripts that produce the "final
packaging." That is, this is the master directory for building a
"GAR" file which is what is actually deployed into the Globus 4.0.x Java
container. Both the WSRF and "elastic" interfaces are stuffed into
this GAR creation process, as well as any dependency (i.e., the RM API
and any of its dependencies ... and the workspace service and any of
its dependencies).
</p>
<p>
So the final product for the service (the GAR files) that is deployed
is something of an onion. The layers around the outside are very
specific to the deployment and wire technology but as you peel things
away you become more and more generic.
</p>
<p>
A related thing to understand is that the initialization sequence works
in the same way. The container boots, the Axis service is initialized
(because the "loadOnStartup" configuration is true), the service is tied
in with the container's JNDI system, a JNDI configuration points to the
Spring configuration, and then the Spring configuration takes over. 99%
of the configuration is done via Spring, but something needs to kick
everything off and via Container->JNDI is how it works.
</p>
_MINI_SEP
<p>
Next up, something needs to call the service. The
_PATH(service/client/java/source) directory is where all of the client code
lives (EC2 clients work too).
</p>
<p>
Navigate to _PATH(service/client/java/source/src/org/globus/workspace).
There are several different layers here. _PATH(client_core) is where all of
the code is to make actual calls to the service, it is a collection of
convenience wrappers around the web services stubs and the returned
data structures (and notifications/polling helpers). These wrappers
make it easier to use the different operations as "building blocks" for
higher level actions.
</p>
<p>
The _PATH(client) package (sibling of _PATH(client_core)) is where the "reference
client" is implemented. This is a commandline wrapper around the
_PATH(client_core) basic actions. It has several "modes" which are each an
"orchestration" of various actions. This client can handle any operation
the service supports and provides many useful utilities for getting
things done in the expected order ("create, then monitor", etc.).
</p>
<p>
Both the _PATH(client) and the _PATH(client_core) classes are ripe for
inserting into portals and other new types of clients.
</p>
<p>
The _PATH(cloud) package, however, is not geared towards programmatic
re-use (parts of it could be, but they were not intentionally written
for this). This is another commandline program but
it is fully geared towards human use. There are many default parameters
and behaviors that are
simplifications of what is possible with the Nimbus API, but definitely
right for users that are getting started or only have the typical needs.
</p>
<p>
The cloud client is packaged separately, and this Java code is in it,
it gets installed (via "ant deploy" in the _PATH(service/client/java/source)
directory) into the embedded GLOBUS_LOCATION in the cloud client
(_PATH(lib/globus)).
</p>
_MINI_SEP
<p>
The _PATH(autoconfiguaration) directory contains programs and scripts that
run the configuration wizard for installing <i>and maintaining</i> the
server side, the "administrator wizard".
</p>
<p>
This is
contained in the GAR files when Nimbus is installed to a container, it
is installed into the _PATH($GLOBUS_LOCATION/share/nimbus-autoconfig)
directory and is a standalone thing. It asks questions and can alter
some of the configuration files, its sole purpose is to make it so the
user has to understand less to get things up and running initially.
</p>
<p>
The _PATH(autocommon) directory contains libraries and code that are
used by _PATH(nimbus-configure --autoconfig)
</p>
_MINI_SEP
<p>
That's the bird's eye view. Ask questions about any of this on the
<a href="_NIMBUS_WEBSITE/contact/#dev">developer's list</a>.
</p>
<br />
<br />
<br />
<a name="pilot-fake"> </a>
<h2>Pilot development environment _NAMELINK(pilot-fake)</h2>
<p>
The following steps will allow you to develop the <b>service</b> portion of the pilot setup without actually needing an LRM installed or hypervisors.
</p>
<ol>
<li>
Activate the pilot plugin, copy "resource-locator-pilot.xml" to "resource-locator-ACTIVE.xml"
</li>
<li>
Edit that, add any string here (e.g. "fake") in order to activate SSH based notifications:
<div class="screen">
<property name="sshNotificationInfo" value="fake" />
</div>
</li>
<li>Edit "services/etc/nimbus/workspace-service/pilot.conf"
<div class="screen">
pbs.submit.path=/tmp/fakeqsub
pbs.delete.path=/bin/true
</div>
</li>
<li>
Create "/tmp/fakeqsub"
<div class="screen">
#!/bin/sh
sleep 3
echo "asdsadasd" # This is the LRM job ID.
</div>
</li>
<li>
Edit "services/container-log4j.properties"
<div class="screen">
log4j.category.org.nimbustools=DEBUG
log4j.category.org.globus.workspace=DEBUG
</div>
</li>
<li>
Edit "services/etc/nimbus/workspace-service/logging.conf"
<div class="screen">
log.state=on
log.trace=on
</div>
Or in some cases you might even want to enable the scheduler logging in
"services/etc/nimbus/workspace-service/other/main.xml"
<div class="screen">
<property name="scheduler" value="on" />
</div>
</li>
<li>
Start service in fake mode (edit "services/etc/nimbus/workspace-service/other/common.conf" )
</li>
<li>Start a vm with cloud client</li>
<li>grep logs for "pilot command" and note the "-i" parameter, e.g. "1e372dca-de93-4fd9-a125-9d66842804cc". That is the SLOT ID assigned to the LRM job. If you are launching many pilots at once (i.e., a cluster of 50 VMs, but one LRM job), each pilot process will report back with the <b>same id</b> but different hostname.
</li>
<li>
Simulate reserved notification:
<div class="screen">
<b>$</b> ./services/var/nimbus/msg-sinks/pilotnotifications write 1e372dca-de93-4fd9-a125-9d66842804cc+++localhost pilot-reserved 0 2010-11-07-16-12-01
</div>
<ul>
<li>
<p>arg1 ("1e372dca-de93-4fd9-a125-9d66842804cc+++localhost")</p>
<p>"1e372dca-de93-4fd9-a125-9d66842804cc" is slot id from service logs</p>
<p>"+++" is separator</p>
<p>"localhost" is hostname where pilot ended up running</p>
</li>
<li><p>arg2 state, e.g. pilot-reserved</p></li>
<li><p>arg3, return code, e.g. 0</p></li>
<li><p>arg4, timestamp, YYYY-MM-DD-HH-MM-SS (always UTC/GMT)</p></li>
</ul>
</li>
</ol>
<br />
<a name="change-wsdl"> </a>
<h2>Changing the WSDL _NAMELINK(change-wsdl)</h2>
<p>
This section discusses making changes to WSDL and generating new stub
classes, etc. It assumes you have some understanding of the layers and
source code directions discussed in the <a href="#code-layout">code layout
section</a>.
</p>
<p>
First run _PATH(scripts/gt/all-clean.sh).
</p>
<p>
Then edit the "compact" WSDL in the
_PATH(messaging/gt4.0/schema/compact/workspace/) directory. Note that the
EC2 WSDL works the same way but we are using the WSRF ones as an example.
</p>
<p>
Compact WSDL
is a Globus convention, it facilitates an inheritance mechanism that allows
the final WSDL to contain common WSRF/WSN related operations. You edit the
compact WSDL only and then a program will "compile" that into the real
WSDL/schemas.
</p>
<p>
Once you have edited something, the changes will not be picked up by the
build system until you have run "ant" in the
_PATH(messaging/gt4.0/schema/compact/) directory. This triggers the default
ant target of "copyToDeployableComponent" which will take the "compact" files
and put the (generated) real WSDL/schemas under the
_PATH(messaging/gt4.0/schema/dist/) directory.
</p>
<p>
That directory is what the _PATH(stubs-build.sh) script uses to generate
the stubs (the Axis classes that are actually referenced by the Nimbus
code for un/marshalling of the web services).
</p>
<p>
That creates the new wsdl. Now you need to create the auto-generated "stub"
code Java jars. Do this by running "scripts/gt/stubs-build.sh"
</p>
<p>
Unless you added something entirely new, the corresponding messaging layer
(e.g., _PATH(messaging/gt4.0/java/msgbridge)) will probably not compile
now. So the next step is to move up the stack and make it work.
</p>
</div>
_NIMBUS_CENTER2_COLUMN_END
_NIMBUS_FOOTER1
_NIMBUS_FOOTER2
_NIMBUS_FOOTER3