Skip to content

HTTPS clone URL

Subversion checkout URL

You can clone with
or
.
Download ZIP

Loading…

Add copying credentials to worker node, and pulling VMs with them #12

Closed
wants to merge 7 commits into from

2 participants

@oldpatricka
Collaborator

So, this isn't polished yet, but I thought I'd push this up and have you guys take a look at it, and see what you think before I do any more work on it. All I wanted at this point was something that worked, and now want to get some other eyes on it to make sure this approach doesn't do anything crazy that you guys don't like.

The idea is that users might (well in my case users do) want to pull images from an https repository using an x509 proxy cert. To do this, we need some kind of delegation method, to get a credential on the service, and some method of copying the credential to the VMM. Originally, I planned on using the standard Globus delegation factory, but realized that this isn't included with Nimbus anymore, and probably wouldn't be portable when Nimbus moves away from WSRF. So I did the simplest thing I could think of, and just added a field to the optional XML file that workspace.sh uses. The idea is that users can just stuff their credential in there, and Nimbus can use it. The nimbus service then pushes this credential to the workspace control tmp directory.

There's also a new propagation adapter for https, which takes the --prop-extra-args parameter to get the filename of the credential it should use when pulling a file.

Any feedback?

oldpatricka added some commits
@oldpatricka oldpatricka Implement simple proxy copying in service and client 89f15e8
@oldpatricka oldpatricka Cleanup service changes for copying a proxy 1de3fd4
@oldpatricka oldpatricka Add https pulling with x509 credential to workspace control 6550fcb
@oldpatricka oldpatricka Add persistance and clean up credential copying.
Also do some cleanup.
bbe63d9
@oldpatricka oldpatricka Change arguments to prop-extra-args to play nicely with other uses
Just in case someone else ever uses prop-extra-args for anything. Other
wc arguments seem to use the 'argument;argument;argument' format, so
repeat that here.
f2288e0
@oldpatricka oldpatricka Refactor CustomizationNeed to FileCopyNeed.
This way it can also be used for files that need to be copied to the VMM
node at Propagate time (credentials).

This also means that files that are contexed in are copied at
propagation time, rather than start time, as in the past.
7b64f30
@oldpatricka oldpatricka Fix restarting gzipped images.
Since the files will have been unzipped in a previous start attempt,
they will fail on a second attempt unless we check that they've already
been unzipped.
b94dbc0
@oldpatricka
Collaborator

Huh. This diff looks strange, but it's a pretty simple patch.

@oldpatricka
Collaborator

So this is a much cleaned up from when this pull was first asked for. It renames the CustomizationNeed to FileCopyNeed, as this is used for copying credentials now. This also happens at propagate time, rather than start time.

As before, comments are welcome.

@labisso
Collaborator

Merged in cee6e2f

This issue was closed.
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Commits on Nov 30, 2010
  1. @oldpatricka
  2. @oldpatricka
  3. @oldpatricka
Commits on Dec 2, 2010
  1. @oldpatricka

    Add persistance and clean up credential copying.

    oldpatricka authored
    Also do some cleanup.
Commits on Dec 7, 2010
  1. @oldpatricka

    Change arguments to prop-extra-args to play nicely with other uses

    oldpatricka authored
    Just in case someone else ever uses prop-extra-args for anything. Other
    wc arguments seem to use the 'argument;argument;argument' format, so
    repeat that here.
  2. @oldpatricka

    Refactor CustomizationNeed to FileCopyNeed.

    oldpatricka authored
    This way it can also be used for files that need to be copied to the VMM
    node at Propagate time (credentials).
    
    This also means that files that are contexed in are copied at
    propagation time, rather than start time, as in the past.
Commits on Dec 8, 2010
  1. @oldpatricka

    Fix restarting gzipped images.

    oldpatricka authored
    Since the files will have been unzipped in a previous start attempt,
    they will fail on a second attempt unless we check that they've already
    been unzipped.
This page is out of date. Refresh to see the latest.
Showing with 555 additions and 193 deletions.
  1. +4 −0 control/etc/workspace-control/propagation.conf
  2. +14 −1 control/src/python/workspacecontrol/defaults/ImageEditing.py
  3. +17 −10 control/src/python/workspacecontrol/defaults/imageprocurement/propagate_common.py
  4. +139 −0 control/src/python/workspacecontrol/defaults/imageprocurement/propagate_https.py
  5. BIN  lib/generated/nimbus-ctx-stubs-gt4.0.jar
  6. BIN  lib/generated/nimbus-messaging-stubs-gt4.0-elastic.jar
  7. BIN  lib/generated/nimbus-messaging-stubs-gt4.0.jar
  8. +1 −0  messaging/gt4.0/java/msgbridge/src/org/nimbustools/messaging/gt4_0/factory/Translate.java
  9. +4 −0 messaging/gt4.0/schema/compact/workspace/workspace_types.xsd
  10. +4 −0 messaging/gt4.0/schema/dist/workspace/workspace_types.xsd
  11. +1 −0  service-api/java/source/src/org/nimbustools/api/_repr/_CreateRequest.java
  12. +1 −0  service-api/java/source/src/org/nimbustools/api/_repr/vm/_VM.java
  13. +9 −0 service-api/java/source/src/org/nimbustools/api/defaults/repr/DefaultCreateRequest.java
  14. +9 −0 service-api/java/source/src/org/nimbustools/api/defaults/repr/vm/DefaultVM.java
  15. +1 −0  service-api/java/source/src/org/nimbustools/api/repr/CreateRequest.java
  16. +7 −0 service/service/java/source/etc/workspace-service/other/main.xml
  17. +0 −1  service/service/java/source/share/lib/workspace_service_derby_dump.sql
  18. +7 −7 service/service/java/source/share/lib/workspace_service_derby_schema.sql
  19. +4 −4 service/service/java/source/src/org/globus/workspace/creation/defaults/DefaultCreation.java
  20. +1 −0  service/service/java/source/src/org/globus/workspace/persistence/DataConvert.java
  21. +3 −3 service/service/java/source/src/org/globus/workspace/persistence/PersistenceAdapter.java
  22. +15 −15 service/service/java/source/src/org/globus/workspace/persistence/PersistenceAdapterConstants.java
  23. +11 −11 service/service/java/source/src/org/globus/workspace/persistence/PersistenceAdapterImpl.java
  24. +15 −8 service/service/java/source/src/org/globus/workspace/persistence/impls/VirtualMachinePersistenceUtil.java
  25. +2 −2 service/service/java/source/src/org/globus/workspace/service/InstanceResource.java
  26. +28 −0 service/service/java/source/src/org/globus/workspace/service/binding/BindCredential.java
  27. +2 −2 service/service/java/source/src/org/globus/workspace/service/binding/BindCustomizations.java
  28. +2 −2 service/service/java/source/src/org/globus/workspace/service/binding/BindingAdapter.java
  29. +85 −0 service/service/java/source/src/org/globus/workspace/service/binding/defaults/DefaultBindCredential.java
  30. +6 −6 service/service/java/source/src/org/globus/workspace/service/binding/defaults/DefaultBindCustomizations.java
  31. +1 −1  service/service/java/source/src/org/globus/workspace/service/binding/defaults/DefaultBindDisks.java
  32. +12 −3 service/service/java/source/src/org/globus/workspace/service/binding/defaults/DefaultBindingAdapter.java
  33. +44 −38 ...e/java/source/src/org/globus/workspace/service/binding/vm/{CustomizationNeed.java → FileCopyNeed.java}
  34. +32 −21 service/service/java/source/src/org/globus/workspace/service/binding/vm/VirtualMachine.java
  35. +3 −3 service/service/java/source/src/org/globus/workspace/service/impls/InstanceResourceImpl.java
  36. +1 −1  service/service/java/source/src/org/globus/workspace/service/impls/StateTransition.java
  37. +11 −10 service/service/java/source/src/org/globus/workspace/xen/XenUtil.java
  38. +4 −4 service/service/java/source/src/org/globus/workspace/xen/xenlocal/Start.java
  39. +48 −3 service/service/java/source/src/org/globus/workspace/xen/xenssh/Propagate.java
  40. +4 −34 service/service/java/source/src/org/globus/workspace/xen/xenssh/Start.java
  41. +3 −3 service/service/java/tests/suites/basic/home/services/share/nimbus/lib/workspace_service_derby_schema.sql
View
4 control/etc/workspace-control/propagation.conf
@@ -28,6 +28,10 @@ scp: /usr/bin/scp
# has a configuration for the allowed hostnames that may be pulled from.
http: false
+# Set this to 'true' to enable https based propagation. The service node
+# has a configuration for the allowed hostnames that may be pulled from.
+# This allows you to authenticate with the server with an x509 credential.
+https: false
# Hadoop Distributed File System (hdfs)
# Make this point to the hadoop executable of your install. Environment
View
15 control/src/python/workspacecontrol/defaults/ImageEditing.py
@@ -218,7 +218,7 @@ def process_after_procurement(self, local_file_set):
"""
for lf in local_file_set.flist():
- if lf.path[-3:] == ".gz":
+ if lf.path.count(".gz") > 0 :
lf.path = self._gunzip_file_inplace(lf.path)
# disabled
@@ -325,6 +325,19 @@ def _gzip_file_inplace(self, path):
def _gunzip_file_inplace(self, path):
self.c.log.info("gunzipping '%s'" % path)
try:
+ # Since gunzip chokes when you give it a file not ending in .gz,
+ # remove anything after the last .gz (query string params etc)
+ gzindex = path.rfind(".gz")
+ clean_path = path[:gzindex] + ".gz"
+ if clean_path != path:
+ try:
+ shutil.move(path, clean_path)
+ path = clean_path
+ except:
+ errmsg = "problem renaming %s to %s" % (path, clean_path)
+ log.exception(errmsg)
+ raise UnexpectedError(errmsg)
+
cmd = "gunzip %s" % path
if self.c.dryrun:
self.c.log.debug("dryrun, command is: %s" % cmd)
View
27 control/src/python/workspacecontrol/defaults/imageprocurement/propagate_common.py
@@ -17,6 +17,7 @@
PROP_ADAPTER_GUC = "gsiftp"
PROP_ADAPTER_HDFS = "hdfs"
PROP_ADAPTER_HTTP = "http"
+PROP_ADAPTER_HTTPS = "https"
PROP_ADAPTER_LANTORRENT = "lantorrent"
class DefaultImageProcurement:
@@ -90,6 +91,11 @@ def validate(self):
import propagate_http
self.adapters[PROP_ADAPTER_HTTP] = propagate_http.propadapter(self.p, self.c)
+ https_enabled = self.p.get_conf_or_none("propagation", "https")
+ if https_enabled and https_enabled.strip().lower() == "true":
+ import propagate_https
+ self.adapters[PROP_ADAPTER_HTTPS] = propagate_https.propadapter(self.p, self.c)
+
if len(self.adapters) == 0:
self.c.log.warn("There are no propagation adapters configured, propagation is disabled")
return
@@ -612,24 +618,24 @@ def _one_imagestr(self, logstr, imgstr, unprop=False):
securedir_try = self._derive_instance_dir()
securedir_try = os.path.join(securedir_try, original)
+
+ # We need to check that the image hasn't previously been unzipped
+ gz_parts = original.rsplit(".gz",1)
+ securedir_lessgz_try = self._derive_instance_dir()
+ securedir_lessgz_try = os.path.join(securedir_lessgz_try, gz_parts[0])
localdir_try = os.path.join(self.localdir, original)
# important: try securedir first, it takes precedence
if os.path.exists(securedir_try):
- localdir_try = None
- elif os.path.exists(localdir_try):
- securedir_try = None
- else:
- raise InvalidInput("File specified by relative path ('%s' could resolve to either '%s' or '%s') but it does not exist" % (original, securedir_try, localdir_try))
-
- if securedir_try:
lf.path = securedir_try
- elif localdir_try:
+ elif os.path.exists(securedir_lessgz_try):
+ lf.path = securedir_lessgz_try
+ elif os.path.exists(localdir_try):
lf.path = localdir_try
else:
- raise ProgrammingError("must be relative to either securedir or localdir or it is invalid")
-
+ raise InvalidInput("File specified by relative path ('%s' could resolve to either '%s', '%s', or '%s') but it does not exist" % (original, securedir_try, securedir_lessgz_try, localdir_try))
+
# ---------------------------------------------------------------
elif imgstr[:14] == "blankcreate://":
@@ -715,6 +721,7 @@ def _one_imagestr_propagation(self, lf, imgstr, unprop):
fnameindex = string.rfind(imgstr, '/')
local_filename = imgstr[fnameindex+1:]
+
# lf.path is propagation target while in the module ... if this
# object is returned by the module, it is assumed to exist
lf.path = self._derive_instance_dir()
View
139 control/src/python/workspacecontrol/defaults/imageprocurement/propagate_https.py
@@ -0,0 +1,139 @@
+from commands import getstatusoutput
+import os
+import string
+from urlparse import urlparse
+import httplib
+import shutil
+from propagate_adapter import PropagationAdapter
+from workspacecontrol.api.exceptions import *
+import workspacecontrol.main.wc_args as wc_args
+
+class propadapter(PropagationAdapter):
+
+ def __init__(self, params, common):
+ PropagationAdapter.__init__(self, params, common)
+
+ def validate(self):
+ self.c.log.debug("validating https propagation adapter")
+ self._get_credential()
+ # Nothing to validate...
+
+ def validate_propagate_source(self, imagestr):
+
+ url = urlparse(imagestr)
+ #urlparse breaks the url into a tuple
+ if url[0] != "https":
+ raise InvalidInput("invalid url, not https:// " + remote)
+
+ def validate_unpropagate_target(self, imagestr):
+ raise InvalidInput("HTTPS unpropagation is not supported.")
+
+ def propagate(self, remote_source, local_absolute_target):
+ self.c.log.info("HTTPS propagation - remote source: %s" % remote_source)
+ self.c.log.info("HTTPS propagation - local target: %s" % local_absolute_target)
+
+ url = urlparse(remote_source)
+ scheme = url[0]
+ netloc = url[1]
+ path = url[2] + "?" + url[4]
+ host_port = netloc.split(":")
+ host = host_port[0]
+ try:
+ port = host_port[1]
+ except IndexError:
+ if scheme == 'http':
+ port = 80
+ else:
+ port = 443
+ credential = self._get_credential()
+ self.c.log.debug("server: %s port %s credential %s" % (host, port, credential))
+ if credential:
+ connection = httplib.HTTPSConnection(host, port, strict=False, key_file=credential, cert_file=credential)
+ else:
+ connection = httplib.HTTPSConnection(host, port)
+
+ try:
+ response = self._get_handle_redirects(connection, path)
+ except:
+ errmsg = "HTTP propagation - Couldn't get image"
+ self.c.log.error(errmsg)
+ raise
+ else:
+ if response.status != 200:
+ errmsg = "HTTP propagation: Got status %s from web server. Can't download image" % response.status
+ else:
+ try:
+ shutil.copyfileobj(response, open(local_absolute_target, 'w'))
+ except:
+ self.c.log.exception("Couldn't save image file to %s." % local_absolute_target)
+ raise
+
+ self.c.log.info("Transfer complete.")
+
+ def unpropagate(self, local_absolute_source, remote_target):
+ raise InvalidInput("HTTP unpropagation is not supported.")
+
+ def _get_credential(self):
+ extra_args = self.p.get_arg_or_none(wc_args.EXTRA_ARGS)
+ if extra_args == None:
+ return None
+
+ # unpack extra-args in format arg=value;arg=value;...;arg=value
+ credential_name = None
+ for extra_arg in extra_args.split(";"):
+ try:
+ parts = extra_arg.split("=")
+ if parts[0] == "credential":
+ credential_name = parts[1]
+ except:
+ continue
+
+ if not credential_name:
+ return None
+
+ tmpdir = self.p.get_conf_or_none("mount", "tmpdir")
+ tmpdir = self.c.resolve_var_dir(tmpdir)
+ credential = tmpdir + "/" + credential_name
+
+ # If the file is readable, we assume it's good. Otherwise we'll fail on propagate
+ if not (os.path.exists(credential) and os.access(credential, os.R_OK)):
+ raise InvalidInput("Cannot read credential '%s'" % credential)
+
+ return credential
+
+ def _get_handle_redirects(self, connection, path):
+ """
+ _handle_redirects -- takes an httplib.connection object, follows
+ redirects if there are any, then returns a new connection object
+
+ """
+ connection.request("GET", path)
+ response = connection.getresponse()
+ status_class = response.status / 100
+
+ # all 3xx return codes are redirects
+ if status_class != 3:
+ return response
+ else:
+ try:
+ redirect_url = response.getheader("Location")
+ except:
+ errmsg = "Got a redirect, but couldn't follow redirect from: %s" % response.msg
+ self.c.log.exception(errmsg)
+ raise UnexpectedError(errmsg)
+
+ self.c.log.info("Redirected to %s" % redirect_url)
+
+ # parse new URL
+ url = urlparse(redirect_url)
+ netloc = url[1]
+ host_port = netloc.split(":")
+ host = host_port[0]
+ redirect_path = url[2] + "?" + url[4]
+
+ if host != connection.host:
+ errmsg = "Cannot follow cross-server redirect from %s to %s" % (connection.host, host)
+ self.c.log.error(errmsg)
+ raise UnexpectedError(errmsg)
+
+ return self._get_handle_redirects(connection, redirect_path)
View
BIN  lib/generated/nimbus-ctx-stubs-gt4.0.jar
Binary file not shown
View
BIN  lib/generated/nimbus-messaging-stubs-gt4.0-elastic.jar
Binary file not shown
View
BIN  lib/generated/nimbus-messaging-stubs-gt4.0.jar
Binary file not shown
View
1  messaging/gt4.0/java/msgbridge/src/org/nimbustools/messaging/gt4_0/factory/Translate.java
@@ -183,6 +183,7 @@ public _CreateRequest getCreateRequest(WorkspaceCreateRequest_Type wsreq)
if (optional != null) {
customizes = optional.getFilewrite();
req.setMdUserData(optional.getMdServerUserdata());
+ req.setCredential(optional.getCredentialToCopy());
} else {
customizes = null;
}
View
4 messaging/gt4.0/schema/compact/workspace/workspace_types.xsd
@@ -229,6 +229,10 @@
<xs:element name="mdServerUserdata"
type="xs:string"
minOccurs="0" />
+
+ <xs:element name="credentialToCopy"
+ type="xs:string"
+ minOccurs="0" />
</xs:sequence>
</xs:complexType>
View
4 messaging/gt4.0/schema/dist/workspace/workspace_types.xsd
@@ -229,6 +229,10 @@
<xs:element name="mdServerUserdata"
type="xs:string"
minOccurs="0" />
+
+ <xs:element name="credentialToCopy"
+ type="xs:string"
+ minOccurs="0" />
</xs:sequence>
</xs:complexType>
View
1  service-api/java/source/src/org/nimbustools/api/_repr/_CreateRequest.java
@@ -42,6 +42,7 @@
public void setShutdownType(String type);
public void setInitialStateRequest(String state);
public void setContext(Context context);
+ public void setCredential(String credential);
public void setMdUserData(String mdUserData);
public void setSshKeyName(String keyName);
}
View
1  service-api/java/source/src/org/nimbustools/api/_repr/vm/_VM.java
@@ -37,5 +37,6 @@
public void setCreator(Caller creator);
public void setLaunchIndex(int launchIndex);
public void setMdUserData(String mdUserData);
+ public void setCredentialName(String credentialName);
public void setSshKeyName(String keyName);
}
View
9 service-api/java/source/src/org/nimbustools/api/defaults/repr/DefaultCreateRequest.java
@@ -49,6 +49,7 @@
private String initialStateRequest;
private Context context;
private String mdUserData;
+ private String credential;
private String sshKeyName;
@@ -116,6 +117,10 @@ public String getMdUserData() {
return this.mdUserData;
}
+ public String getCredential() {
+ return this.credential;
+ }
+
public String getSshKeyName() {
return sshKeyName;
}
@@ -184,6 +189,10 @@ public void setMdUserData(String mdUserData) {
this.mdUserData = mdUserData;
}
+ public void setCredential(String credential) {
+ this.credential = credential;
+ }
+
public void setSshKeyName(String sshKeyName) {
this.sshKeyName = sshKeyName;
}
View
9 service-api/java/source/src/org/nimbustools/api/defaults/repr/vm/DefaultVM.java
@@ -43,6 +43,7 @@
private Caller creator;
private int launchIndex;
private String mdUserData;
+ private String credentialName;
private String sshKeyName;
@@ -146,6 +147,14 @@ public String getMdUserData() {
return this.mdUserData;
}
+ public void setCredentialName(String credentialName) {
+ this.credentialName = credentialName;
+ }
+
+ public String getCredentialName() {
+ return this.credentialName;
+ }
+
public void setSshKeyName(String sshKeyName) {
this.sshKeyName = sshKeyName;
}
View
1  service-api/java/source/src/org/nimbustools/api/repr/CreateRequest.java
@@ -50,5 +50,6 @@
public String getInitialStateRequest();
public Context getContext();
public String getMdUserData();
+ public String getCredential();
public String getSshKeyName();
}
View
7 service/service/java/source/etc/workspace-service/other/main.xml
@@ -177,6 +177,13 @@
</bean>
+ <bean id="nimbus-rm.service.binding.BindCredential"
+ class="org.globus.workspace.service.binding.defaults.DefaultBindCredential">
+
+ <constructor-arg ref="nimbus-rm.PathConfigs" />
+
+ </bean>
+
<bean id="nimbus-rm.service.binding.BindKernel"
class="org.globus.workspace.service.binding.defaults.DefaultBindKernel" />
View
1  service/service/java/source/share/lib/workspace_service_derby_dump.sql
@@ -18,6 +18,5 @@
-- select * from counter;
-- select * from notification_position;
-
disconnect;
exit;
View
14 service/service/java/source/share/lib/workspace_service_derby_schema.sql
@@ -61,7 +61,8 @@ kernel_parameters VARCHAR(128),
vmm VARCHAR(32),
vmm_version VARCHAR(32),
assocs_needed VARCHAR(256),
-md_user_data VARCHAR(30720)
+md_user_data VARCHAR(30720),
+credential_name VARCHAR(128)
);
--
@@ -117,14 +118,14 @@ PRIMARY KEY(association,ipaddress)
);
--
--- Persistence for file customization tasks
+-- Persistence for file copy tasks
-CREATE TABLE vm_customization
+CREATE TABLE file_copy
(
vmid INT NOT NULL,
-sourcepath VARCHAR(32) NOT NULL,
-destpath VARCHAR(512) NOT NULL,
-sent SMALLINT NOT NULL
+sourcepath VARCHAR(36) NOT NULL,
+destpath VARCHAR(512),
+on_image SMALLINT NOT NULL
);
--
@@ -157,7 +158,6 @@ CREATE TABLE default_scheduler_done_ensemb
coschedid CHAR(36) NOT NULL
);
-
-- using REAL for memory attributs to allow
-- real division operations in ORDER BY statements
View
8 service/service/java/source/src/org/globus/workspace/creation/defaults/DefaultCreation.java
@@ -44,7 +44,7 @@
import org.globus.workspace.service.binding.GlobalPolicies;
import org.globus.workspace.service.binding.vm.VirtualMachine;
import org.globus.workspace.service.binding.vm.VirtualMachineDeployment;
-import org.globus.workspace.service.binding.vm.CustomizationNeed;
+import org.globus.workspace.service.binding.vm.FileCopyNeed;
import org.nimbustools.api._repr._CreateResult;
import org.nimbustools.api._repr._Advertised;
@@ -891,10 +891,10 @@ protected VM createOne(int idx,
try {
final String newContent =
addIPs(context.getBootstrapText(), vm);
- final CustomizationNeed need =
- this.binding.newCustomizationNeed(
+ final FileCopyNeed need =
+ this.binding.newFileCopyNeed(
newContent, context.getBootstrapPath());
- resource.newCustomizationNeed(need);
+ resource.newFileCopyNeed(need);
} catch (Exception e) {
logger.error(e.getMessage());
}
View
1  service/service/java/source/src/org/globus/workspace/persistence/DataConvert.java
@@ -206,6 +206,7 @@ public VM getVM(InstanceResource resource) throws CannotTranslateException {
vm.setLaunchIndex(resource.getLaunchIndex());
vm.setNics(this.getNICs(resource.getVM()));
vm.setMdUserData(resource.getVM().getMdUserData());
+ vm.setCredentialName(resource.getVM().getCredentialName());
vm.setVMFiles(this.getStorage(resource.getVM()));
vm.setResourceAllocation(this.getRA(resource));
vm.setSchedule(this.getSchedule(resource));
View
6 service/service/java/source/src/org/globus/workspace/persistence/PersistenceAdapter.java
@@ -25,7 +25,7 @@
import org.globus.workspace.service.CoschedResource;
import org.globus.workspace.service.GroupResource;
import org.globus.workspace.service.InstanceResource;
-import org.globus.workspace.service.binding.vm.CustomizationNeed;
+import org.globus.workspace.service.binding.vm.FileCopyNeed;
import org.nimbustools.api.services.rm.DoesNotExistException;
/**
@@ -62,7 +62,7 @@ public void setRootUnpropTarget(int id, String path)
throws WorkspaceDatabaseException;
- public void setCustomizeTaskSent(int id, CustomizationNeed need)
+ public void setFileCopyOnImage(int id, FileCopyNeed need)
throws WorkspaceDatabaseException;
@@ -172,7 +172,7 @@ public boolean removeResourcepoolEntry(String hostname)
throws WorkspaceDatabaseException;
- public void addCustomizationNeed(int id, CustomizationNeed need)
+ public void addCustomizationNeed(int id, FileCopyNeed need)
throws WorkspaceDatabaseException;
View
30 ...service/java/source/src/org/globus/workspace/persistence/PersistenceAdapterConstants.java
@@ -58,8 +58,8 @@
"UPDATE vm_partitions SET alternate_unprop=? " +
"WHERE vmid=? AND rootdisk=1";
- public static final String SQL_SET_VM_CUSTOMIZATION_SENT =
- "UPDATE vm_customization SET sent=? " +
+ public static final String SQL_SET_FILE_COPY_ON_IMAGE =
+ "UPDATE file_copy SET on_image=? " +
"WHERE vmid=? AND sourcepath=? AND destpath=?";
public static final String SQL_SET_STARTTIME =
@@ -83,8 +83,8 @@
public static final String SQL_DELETE_VM_DEPLOYMENT =
"DELETE from vm_deployment WHERE vmid=?";
- public static final String SQL_DELETE_VM_CUSTOMIZATION =
- "DELETE from vm_customization WHERE vmid=?";
+ public static final String SQL_DELETE_FILE_COPY =
+ "DELETE from file_copy WHERE vmid=?";
public static final String SQL_INSERT_RESOURCE =
"INSERT INTO resources VALUES(?,?,?,?,?,?,?,?,?,?,?,?,?,?,?)";
@@ -93,7 +93,7 @@
"INSERT INTO groupresources VALUES(?,?)";
public static final String SQL_INSERT_VM =
- "INSERT INTO vms VALUES(?,?,?,?,?,?,?,?,?,?,?)";
+ "INSERT INTO vms VALUES(?,?,?,?,?,?,?,?,?,?,?,?)";
public static final String SQL_INSERT_VM_PARTITION =
"INSERT INTO vm_partitions VALUES(?,?,?,?,?,?,?,?,?)";
@@ -101,8 +101,8 @@
public static final String SQL_INSERT_VM_DEPLOYMENT =
"INSERT INTO vm_deployment VALUES(?,?,?,?,?,?)";
- public static final String SQL_INSERT_VM_CUSTOMIZATION =
- "INSERT INTO vm_customization VALUES(?,?,?,?)";
+ public static final String SQL_INSERT_FILE_COPY =
+ "INSERT INTO file_copy VALUES(?,?,?,?)";
public static final String SQL_LOAD_RESOURCE =
"SELECT name, state, target_state, term_time, ops_enabled, " +
@@ -120,7 +120,7 @@
public static final String SQL_LOAD_VM =
"SELECT name, node, prop_required, unprop_required, network, " +
"kernel_parameters, vmm, vmm_version, assocs_needed, " +
- "md_user_data " +
+ "md_user_data, credential_name " +
"FROM vms WHERE id=?";
public static final String SQL_LOAD_VM_PARTITIONS =
@@ -133,9 +133,9 @@
"ind_physmem, ind_physcpu " +
"FROM vm_deployment WHERE vmid=?";
- public static final String SQL_LOAD_VM_CUSTOMIZATION =
- "SELECT sourcepath, destpath, sent " +
- "FROM vm_customization WHERE vmid=?";
+ public static final String SQL_LOAD_FILE_COPY =
+ "SELECT sourcepath, destpath, on_image " +
+ "FROM file_copy WHERE vmid=?";
public static final String SQL_UPDATE_ASSOCIATION_ENTRY =
"UPDATE association_entries SET used=? " +
@@ -203,7 +203,7 @@
SQL_SET_HOSTNAME,
SQL_SET_ROOT_UNPROP_TARGET,
SQL_UNSET_ROOT_UNPROP_TARGET,
- SQL_SET_VM_CUSTOMIZATION_SENT,
+ SQL_SET_FILE_COPY_ON_IMAGE,
SQL_SET_STARTTIME,
SQL_SET_TERMTIME,
SQL_DELETE_RESOURCE,
@@ -211,19 +211,19 @@
SQL_DELETE_VM,
SQL_DELETE_VM_PARTITIONS,
SQL_DELETE_VM_DEPLOYMENT,
- SQL_DELETE_VM_CUSTOMIZATION,
+ SQL_DELETE_FILE_COPY,
SQL_INSERT_RESOURCE,
SQL_INSERT_VM,
SQL_INSERT_VM_PARTITION,
SQL_INSERT_VM_DEPLOYMENT,
- SQL_INSERT_VM_CUSTOMIZATION,
+ SQL_INSERT_FILE_COPY,
SQL_LOAD_RESOURCE,
SQL_LOAD_RESOURCE_NAME,
SQL_LOAD_GROUP_RESOURCE,
SQL_LOAD_VM,
SQL_LOAD_VM_PARTITIONS,
SQL_LOAD_VM_DEPLOYMENT,
- SQL_LOAD_VM_CUSTOMIZATION,
+ SQL_LOAD_FILE_COPY,
SQL_UPDATE_ASSOCIATION_ENTRY,
SQL_DELETE_ALL_ASSOCIATIONS,
SQL_DELETE_ALL_ASSOCIATION_ENTRIES,
View
22 service/service/java/source/src/org/globus/workspace/persistence/PersistenceAdapterImpl.java
@@ -41,7 +41,7 @@
import org.globus.workspace.service.CoschedResource;
import org.globus.workspace.service.GroupResource;
import org.globus.workspace.service.InstanceResource;
-import org.globus.workspace.service.binding.vm.CustomizationNeed;
+import org.globus.workspace.service.binding.vm.FileCopyNeed;
import org.globus.workspace.service.binding.vm.VirtualMachine;
import org.globus.workspace.service.binding.vm.VirtualMachinePartition;
import org.nimbustools.api.services.rm.DoesNotExistException;
@@ -489,11 +489,11 @@ public void setRootUnpropTarget(int id, String path)
}
}
- public void addCustomizationNeed(int id, CustomizationNeed need)
+ public void addCustomizationNeed(int id, FileCopyNeed need)
throws WorkspaceDatabaseException {
if (this.dbTrace) {
- logger.trace("addCustomizationNeed(): " + Lager.id(id));
+ logger.trace("addFileCopyNeed(): " + Lager.id(id));
}
if (need == null) {
@@ -504,12 +504,12 @@ public void addCustomizationNeed(int id, CustomizationNeed need)
PreparedStatement pstmt = null;
try {
c = getConnection();
- pstmt = c.prepareStatement(SQL_INSERT_VM_CUSTOMIZATION);
+ pstmt = c.prepareStatement(SQL_INSERT_FILE_COPY);
pstmt.setInt(1, id);
pstmt.setString(2, need.sourcePath);
pstmt.setString(3, need.destPath);
- if (need.isSent()) {
+ if (need.onImage()) {
pstmt.setInt(4, 1);
} else {
pstmt.setInt(4, 0);
@@ -538,20 +538,20 @@ public void addCustomizationNeed(int id, CustomizationNeed need)
}
}
- public void setCustomizeTaskSent(int id, CustomizationNeed need)
+ public void setFileCopyOnImage(int id, FileCopyNeed need)
throws WorkspaceDatabaseException {
if (this.dbTrace) {
- logger.trace("setCustomizeTaskSent(): " + Lager.id(id) +
- ", sent = " + need.isSent());
+ logger.trace("setFileCopyOnImage(): " + Lager.id(id) +
+ ", on image = " + need.onImage());
}
Connection c = null;
PreparedStatement pstmt = null;
try {
c = getConnection();
- pstmt = c.prepareStatement(SQL_SET_VM_CUSTOMIZATION_SENT);
- if (need.isSent()) {
+ pstmt = c.prepareStatement(SQL_SET_FILE_COPY_ON_IMAGE);
+ if (need.onImage()) {
pstmt.setInt(1, 1);
} else {
pstmt.setInt(1, 0);
@@ -1430,7 +1430,7 @@ public void load(int id, InstanceResource resource)
}
} else {
do {
- vm.addCustomizationNeed(
+ vm.addFileCopyNeed(
VirtualMachinePersistenceUtil.getNeed(rs));
} while (rs.next());
}
View
23 ...java/source/src/org/globus/workspace/persistence/impls/VirtualMachinePersistenceUtil.java
@@ -20,10 +20,10 @@
import org.globus.workspace.persistence.PersistenceAdapterConstants;
import org.globus.workspace.persistence.WorkspaceDatabaseException;
import org.globus.workspace.service.InstanceResource;
+import org.globus.workspace.service.binding.vm.FileCopyNeed;
import org.globus.workspace.service.binding.vm.VirtualMachine;
import org.globus.workspace.service.binding.vm.VirtualMachineDeployment;
import org.globus.workspace.service.binding.vm.VirtualMachinePartition;
-import org.globus.workspace.service.binding.vm.CustomizationNeed;
import org.nimbustools.api.services.rm.ManageException;
import java.sql.Connection;
@@ -105,6 +105,12 @@
pstmt.setNull(11, Types.VARCHAR);
}
+ if (vm.getCredentialName() != null) {
+ pstmt.setString(12, vm.getCredentialName());
+ } else {
+ pstmt.setNull(12, Types.VARCHAR);
+ }
+
PreparedStatement pstmt2 = null;
VirtualMachineDeployment dep = vm.getDeployment();
@@ -186,15 +192,15 @@
}
}
- final CustomizationNeed[] needs = vm.getCustomizationNeeds();
+ final FileCopyNeed[] needs = vm.getFileCopyNeeds();
if (needs != null) {
for (int i = 0; i < needs.length; i++) {
final PreparedStatement custStmt =
- c.prepareStatement(SQL_INSERT_VM_CUSTOMIZATION);
+ c.prepareStatement(SQL_INSERT_FILE_COPY);
custStmt.setInt(1, id);
custStmt.setString(2, needs[i].sourcePath);
custStmt.setString(3, needs[i].destPath);
- if (needs[i].isSent()) {
+ if (needs[i].onImage()) {
custStmt.setInt(4, 1);
} else {
custStmt.setInt(4, 0);
@@ -243,7 +249,7 @@
deletes.add(pstmt3);
final PreparedStatement pstmt4 =
- c.prepareStatement(SQL_DELETE_VM_CUSTOMIZATION);
+ c.prepareStatement(SQL_DELETE_FILE_COPY);
pstmt4.setInt(1, id);
deletes.add(pstmt4);
@@ -268,7 +274,7 @@
pstmt3.setInt(1, id);
final PreparedStatement pstmt4 =
- c.prepareStatement(SQL_LOAD_VM_CUSTOMIZATION);
+ c.prepareStatement(SQL_LOAD_FILE_COPY);
pstmt4.setInt(1, id);
final PreparedStatement[] selects = new PreparedStatement[4];
@@ -297,6 +303,7 @@ public static VirtualMachine newVM(int id, ResultSet rs)
vm.setVmmVersion(rs.getString(8));
vm.setAssociationsNeeded(rs.getString(9));
vm.setMdUserData(rs.getString(10));
+ vm.setCredentialName(rs.getString(11));
return vm;
}
@@ -329,14 +336,14 @@ public static VirtualMachinePartition getPartition(ResultSet rs)
return partition;
}
- public static CustomizationNeed getNeed(ResultSet rs)
+ public static FileCopyNeed getNeed(ResultSet rs)
throws WorkspaceDatabaseException {
try {
final String src = rs.getString(1);
final String dst = rs.getString(2);
final boolean sent = rs.getBoolean(3);
- return new CustomizationNeed(src, dst, sent);
+ return new FileCopyNeed(src, dst, sent);
} catch (Exception e) {
throw new WorkspaceDatabaseException(e.getMessage(), e);
}
View
4 service/service/java/source/src/org/globus/workspace/service/InstanceResource.java
@@ -17,8 +17,8 @@
package org.globus.workspace.service;
import org.globus.workspace.LockAcquisitionFailure;
+import org.globus.workspace.service.binding.vm.FileCopyNeed;
import org.globus.workspace.service.binding.vm.VirtualMachine;
-import org.globus.workspace.service.binding.vm.CustomizationNeed;
import org.nimbustools.api.services.rm.OperationDisabledException;
import org.nimbustools.api.services.rm.DoesNotExistException;
import org.nimbustools.api.services.rm.ManageException;
@@ -161,7 +161,7 @@ public void reboot(ShutdownTasks tasks)
public void newNetwork(String network);
- public void newCustomizationNeed(CustomizationNeed need);
+ public void newFileCopyNeed(FileCopyNeed need);
/**
* Don't call unless you are managing the instance cache (or not using
View
28 service/service/java/source/src/org/globus/workspace/service/binding/BindCredential.java
@@ -0,0 +1,28 @@
+/*
+ * Copyright 1999-2008 University of Chicago
+ *
+ * Licensed under the Apache License, Version 2.0 (the "License"); you may not
+ * use this file except in compliance with the License. You may obtain a copy
+ * of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS, WITHOUT
+ * WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. See the
+ * License for the specific language governing permissions and limitations
+ * under the License.
+ */
+
+package org.globus.workspace.service.binding;
+
+import org.globus.workspace.WorkspaceException;
+import org.globus.workspace.service.binding.vm.VirtualMachine;
+import org.nimbustools.api.services.rm.CreationException;
+
+public interface BindCredential {
+
+ public void consume(VirtualMachine vm,
+ final String credential)
+ throws CreationException;
+}
View
4 service/service/java/source/src/org/globus/workspace/service/binding/BindCustomizations.java
@@ -17,7 +17,7 @@
package org.globus.workspace.service.binding;
import org.globus.workspace.service.binding.vm.VirtualMachine;
-import org.globus.workspace.service.binding.vm.CustomizationNeed;
+import org.globus.workspace.service.binding.vm.FileCopyNeed;
import org.globus.workspace.WorkspaceException;
import org.nimbustools.api.repr.CustomizationRequest;
import org.nimbustools.api.services.rm.CreationException;
@@ -29,7 +29,7 @@ public void consume(VirtualMachine vm,
CustomizationRequest[] reqs)
throws CreationException, ResourceRequestDeniedException;
- public CustomizationNeed newCustomizationNeedImpl(String srcContent,
+ public FileCopyNeed newFileCopyNeedImpl(String srcContent,
String dstPath)
throws WorkspaceException;
}
View
4 service/service/java/source/src/org/globus/workspace/service/binding/BindingAdapter.java
@@ -17,8 +17,8 @@
package org.globus.workspace.service.binding;
import org.globus.workspace.WorkspaceException;
+import org.globus.workspace.service.binding.vm.FileCopyNeed;
import org.globus.workspace.service.binding.vm.VirtualMachine;
-import org.globus.workspace.service.binding.vm.CustomizationNeed;
import org.nimbustools.api.repr.CreateRequest;
import org.nimbustools.api.services.rm.ResourceRequestDeniedException;
import org.nimbustools.api.services.rm.CreationException;
@@ -67,7 +67,7 @@ public void backOutAllocations(VirtualMachine vm)
public void backOutAllocations(VirtualMachine[] vms)
throws WorkspaceException;
- public CustomizationNeed newCustomizationNeed(String srcContent,
+ public FileCopyNeed newFileCopyNeed(String srcContent,
String dstPath)
throws WorkspaceException;
}
View
85 .../java/source/src/org/globus/workspace/service/binding/defaults/DefaultBindCredential.java
@@ -0,0 +1,85 @@
+/*
+ * Copyright 1999-2008 University of Chicago
+ *
+ * Licensed under the Apache License, Version 2.0 (the "License"); you may not
+ * use this file except in compliance with the License. You may obtain a copy
+ * of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS, WITHOUT
+ * WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. See the
+ * License for the specific language governing permissions and limitations
+ * under the License.
+ */
+
+package org.globus.workspace.service.binding.defaults;
+
+import org.apache.commons.logging.Log;
+import org.apache.commons.logging.LogFactory;
+import org.globus.workspace.PathConfigs;
+import org.globus.workspace.service.binding.BindCredential;
+import org.globus.workspace.service.binding.vm.FileCopyNeed;
+import org.globus.workspace.service.binding.vm.VirtualMachine;
+import org.nimbustools.api.services.rm.CreationException;
+import org.safehaus.uuid.UUIDGenerator;
+
+import java.io.File;
+import java.io.FileOutputStream;
+
+public class DefaultBindCredential implements BindCredential {
+
+ private static final Log logger =
+ LogFactory.getLog(DefaultBindCredential.class.getName());
+
+ protected final PathConfigs paths;
+ protected final UUIDGenerator uuidGen = UUIDGenerator.getInstance();
+
+ public DefaultBindCredential(PathConfigs paths) {
+ if (paths == null) {
+ throw new IllegalArgumentException("paths may not be null");
+ }
+ this.paths = paths;
+ }
+
+ public void consume(VirtualMachine vm, final String credential)
+ throws CreationException {
+
+ if (vm == null) {
+ throw new IllegalArgumentException("vm may not be null");
+ }
+ else if (credential == null) {
+ // return early and leave credential null
+ return;
+ }
+
+ final String localTempDirectory = this.paths.getLocalTempDirPath();
+ final String credentialName = this.uuidGen.generateRandomBasedUUID().toString();
+ final String localPath = localTempDirectory + "/" + credentialName;
+
+
+ try {
+ FileOutputStream out = new FileOutputStream(localPath);
+ out.write(credential.getBytes());
+ out.flush();
+ out.close();
+ } catch (Exception e) {
+ throw new CreationException("Couldn't save credential to " + localTempDirectory
+ + ". " + e.getMessage());
+ }
+
+ final FileCopyNeed need;
+ try {
+ // FileCopyNeed expects a file in nimbus's tmp, not a full path
+ need = new FileCopyNeed(credentialName);
+ vm.addFileCopyNeed(need);
+ } catch (Exception e) {
+ final String err = "problem setting up file copy for credential: " +
+ credentialName + " : " + e.getMessage();
+ throw new CreationException(err);
+ }
+
+ vm.setCredentialName(credentialName);
+ }
+}
View
12 ...a/source/src/org/globus/workspace/service/binding/defaults/DefaultBindCustomizations.java
@@ -18,13 +18,13 @@
import org.apache.commons.logging.Log;
import org.apache.commons.logging.LogFactory;
+import org.globus.workspace.service.binding.vm.FileCopyNeed;
import org.nimbustools.api.repr.CustomizationRequest;
import org.nimbustools.api.services.rm.CreationException;
import org.nimbustools.api.services.rm.ResourceRequestDeniedException;
import org.globus.workspace.PathConfigs;
import org.globus.workspace.WorkspaceException;
import org.globus.workspace.service.binding.BindCustomizations;
-import org.globus.workspace.service.binding.vm.CustomizationNeed;
import org.globus.workspace.service.binding.vm.VirtualMachine;
import org.safehaus.uuid.UUIDGenerator;
@@ -110,7 +110,7 @@ public void consume(VirtualMachine vm,
}
}
- public CustomizationNeed newCustomizationNeedImpl(String srcContent,
+ public FileCopyNeed newFileCopyNeedImpl(String srcContent,
String dstPath)
throws WorkspaceException {
@@ -185,12 +185,12 @@ protected void bindFileWrites(VirtualMachine vm,
final String src = filewrites[i].getContent();
final String dstPath = filewrites[i].getPathOnVM();
- vm.addCustomizationNeed(this.newNeed(src, dstPath));
+ vm.addFileCopyNeed(this.newNeed(src, dstPath));
}
}
- protected CustomizationNeed newNeed(String src,
+ protected FileCopyNeed newNeed(String src,
String dstPath) throws Exception {
// already checked, this is for object extenders
@@ -217,9 +217,9 @@ protected CustomizationNeed newNeed(String src,
final String srcPath = this.newSrcPath();
- final CustomizationNeed need;
+ final FileCopyNeed need;
try {
- need = new CustomizationNeed(srcPath, dstPath);
+ need = new FileCopyNeed(srcPath, dstPath);
} catch (Exception e) {
if (logger.isDebugEnabled()) {
logger.error(e.getMessage(), e);
View
2  ...rvice/java/source/src/org/globus/workspace/service/binding/defaults/DefaultBindDisks.java
@@ -154,7 +154,7 @@ protected VirtualMachinePartition rootPartition(VirtualMachine vm,
final String rootDiskScheme = uri.getScheme();
final boolean local = "file".equals(rootDiskScheme);
- final boolean http = "http".equals(rootDiskScheme);
+ final boolean http = "http".equals(rootDiskScheme) || "https".equals(rootDiskScheme);
if (!propagationEnabled && !local) {
final String err = "cannot propagate: supplied image '" +
View
15 .../java/source/src/org/globus/workspace/service/binding/defaults/DefaultBindingAdapter.java
@@ -18,6 +18,8 @@
import org.apache.commons.logging.Log;
import org.apache.commons.logging.LogFactory;
+import org.globus.workspace.service.binding.*;
+import org.globus.workspace.service.binding.vm.FileCopyNeed;
import org.nimbustools.api.repr.CreateRequest;
import org.nimbustools.api.repr.vm.ResourceAllocation;
import org.nimbustools.api.services.rm.CreationException;
@@ -34,7 +36,6 @@
import org.globus.workspace.service.binding.BindDisks;
import org.globus.workspace.service.binding.BindVMM;
import org.globus.workspace.service.binding.BindNetwork;
-import org.globus.workspace.service.binding.vm.CustomizationNeed;
import org.globus.workspace.service.binding.vm.VirtualMachine;
import org.globus.workspace.service.binding.vm.VirtualMachineDeployment;
@@ -57,6 +58,7 @@
protected final BindInitialState bindInitialState;
protected final BindShutdownMechanism bindShutdownMechanism;
protected final BindCustomizations bindCustomizations;
+ protected final BindCredential bindCredential;
protected final BindKernel bindKernel;
protected final BindResourceRequest bindResourceRequest;
protected final BindDisks bindDisks;
@@ -72,6 +74,7 @@ public DefaultBindingAdapter(BindSchedule bindScheduleImpl,
BindInitialState bindInitialStateImpl,
BindShutdownMechanism bindShutdownImpl,
BindCustomizations bindCustomizationsImpl,
+ BindCredential bindCredentialImpl,
BindKernel bindKernelImpl,
BindDisks bindDisksImpl,
BindResourceRequest bindResourceRequestImpl,
@@ -98,6 +101,11 @@ public DefaultBindingAdapter(BindSchedule bindScheduleImpl,
}
this.bindCustomizations = bindCustomizationsImpl;
+ if (bindCredentialImpl == null) {
+ throw new IllegalArgumentException("bindCredentialImpl may not be null");
+ }
+ this.bindCredential = bindCredentialImpl;
+
if (bindKernelImpl == null) {
throw new IllegalArgumentException("bindKernelImpl may not be null");
}
@@ -165,6 +173,7 @@ public DefaultBindingAdapter(BindSchedule bindScheduleImpl,
this.bindResourceRequest.consume(dep, req.getRequestedRA());
this.bindVMM.consume(vm, req.getRequiredVMM());
this.bindCustomizations.consume(vm, req.getCustomizationRequests());
+ this.bindCredential.consume(vm, req.getCredential());
// all in group get the same data
if (req.getMdUserData() != null) {
@@ -204,12 +213,12 @@ public void backOutAllocations(VirtualMachine[] vms)
// OTHER
// -------------------------------------------------------------------------
- public CustomizationNeed newCustomizationNeed(String srcContent,
+ public FileCopyNeed newFileCopyNeed(String srcContent,
String dstPath)
throws WorkspaceException {
return this.bindCustomizations
- .newCustomizationNeedImpl(srcContent, dstPath);
+ .newFileCopyNeedImpl(srcContent, dstPath);
}
}
View
82 ...service/binding/vm/CustomizationNeed.java → ...pace/service/binding/vm/FileCopyNeed.java
@@ -21,7 +21,7 @@
* is 32 chars (UUID), destpath is is 512 chars and is the path ON the VM.
* On the VMM, the file is also the UUID.
*/
-public class CustomizationNeed {
+public class FileCopyNeed {
public final static int srcMax = 36;
public final static int dstMax = 512;
@@ -33,16 +33,22 @@
legalDestCharsString.toCharArray();
public final String sourcePath;
+ // A null destPath indicates that the file won't be copied to the VM
public final String destPath;
- boolean wasSent;
+ boolean onImage;
- public CustomizationNeed(String src, String dst)
+ public FileCopyNeed(String src)
+ throws Exception {
+ this(src, null, false);
+ }
+
+ public FileCopyNeed(String src, String dst)
throws Exception {
this(src, dst, false);
}
- public CustomizationNeed(String src, String dst, boolean sent)
+ public FileCopyNeed(String src, String dst, boolean onImage)
throws Exception {
if (src == null) {
@@ -50,56 +56,56 @@ public CustomizationNeed(String src, String dst, boolean sent)
}
if (src.length() > srcMax) {
throw new Exception(
- "customization source path is too long: " +
+ "file copy source path is too long: " +
src.length() + " > " + srcMax +
". Path: '" + src + "'");
}
- if (dst == null) {
- throw new IllegalArgumentException(
- "destination path may not be null");
- }
- if (dst.length() > dstMax) {
- throw new Exception(
- "customization destination path is too long: " +
- dst.length() + " > " + dstMax +
- ". Path: '" + dst + "'");
- }
- // the mount tool would catch this too, but failfast
- final char[] dstChars = dst.toCharArray();
- for (int i = 0; i < dstChars.length; i++) {
- if (!legalChar(dstChars[i])) {
+ // If dst is null, file won't be copied onto the VM
+ if (dst != null) {
+ if (dst.length() > dstMax) {
throw new Exception(
- "customization destination path contains illegal " +
- "character '" + dstChars[i] + "'. Path: '" + dst + "'");
+ "file copy destination path is too long: " +
+ dst.length() + " > " + dstMax +
+ ". Path: '" + dst + "'");
}
- }
- if (dst.indexOf("../") >= 0) {
- throw new Exception(
- "customization destination path contains illegal " +
- "path expansion, for example '../'. Path: '" + dst + "'");
+ // the mount tool would catch this too, but failfast
+ final char[] dstChars = dst.toCharArray();
+ for (int i = 0; i < dstChars.length; i++) {
+ if (!legalChar(dstChars[i])) {
+ throw new Exception(
+ "file copy destination path contains illegal " +
+ "character '" + dstChars[i] + "'. Path: '" + dst + "'");
+ }
+ }
+
+ if (dst.indexOf("../") >= 0) {
+ throw new Exception(
+ "file copy destination path contains illegal " +
+ "path expansion, for example '../'. Path: '" + dst + "'");
+ }
}
this.sourcePath = src;
this.destPath = dst;
- this.wasSent = sent;
+ this.onImage = onImage;
}
// for clone only
// 'fake' is just there to get around java limitation
- private CustomizationNeed(String src, String dst, boolean sent, int fake) {
+ private FileCopyNeed(String src, String dst, boolean onImage, int fake) {
this.sourcePath = src;
this.destPath = dst;
- this.wasSent = sent;
+ this.onImage = onImage;
}
- public synchronized boolean isSent() {
- return this.wasSent;
+ public synchronized boolean onImage() {
+ return this.onImage;
}
- public synchronized void setSent(boolean sent) {
- this.wasSent = sent;
+ public synchronized void setOnImage(boolean onImage) {
+ this.onImage = onImage;
}
private static boolean legalChar(char c) {
@@ -111,28 +117,28 @@ private static boolean legalChar(char c) {
return false;
}
- public static CustomizationNeed[] cloneArray(CustomizationNeed[] cur)
+ public static FileCopyNeed[] cloneArray(FileCopyNeed[] cur)
throws Exception {
if (cur == null) {
return null;
}
- final CustomizationNeed[] newArr = new CustomizationNeed[cur.length];
+ final FileCopyNeed[] newArr = new FileCopyNeed[cur.length];
for (int i = 0; i < cur.length; i++) {
newArr[i] = cloneOne(cur[i]);
}
return newArr;
}
- public static CustomizationNeed cloneOne(CustomizationNeed cur)
+ public static FileCopyNeed cloneOne(FileCopyNeed cur)
throws Exception {
if (cur == null) {
return null;
}
- return new CustomizationNeed(
- cur.sourcePath, cur.destPath, cur.wasSent, 0);
+ return new FileCopyNeed(
+ cur.sourcePath, cur.destPath, cur.onImage, 0);
}
}
View
53 service/service/java/source/src/org/globus/workspace/service/binding/vm/VirtualMachine.java
@@ -46,7 +46,9 @@
private VirtualMachinePartition[] partitions;
- private CustomizationNeed[] customizationNeeds;
+ private FileCopyNeed[] fileCopyNeeds;
+
+ private String credentialName;
private String mdUserData;
//requested vmm type
@@ -144,30 +146,38 @@ public void setMdUserData(String mdUserData) {
this.mdUserData = mdUserData;
}
- public synchronized void addCustomizationNeed(CustomizationNeed need) {
- if (this.customizationNeeds == null) {
- this.customizationNeeds = new CustomizationNeed[1];
- this.customizationNeeds[0] = need;
+ public String getCredentialName() {
+ return this.credentialName;
+ }
+
+ public void setCredentialName(String credentialName) {
+ this.credentialName = credentialName;
+ }
+
+ public synchronized void addFileCopyNeed(FileCopyNeed need) {
+ if (this.fileCopyNeeds == null) {
+ this.fileCopyNeeds = new FileCopyNeed[1];
+ this.fileCopyNeeds[0] = need;
} else {
- final int curlen = this.customizationNeeds.length;
- final CustomizationNeed[] src = this.customizationNeeds;
- final CustomizationNeed[] dst = new CustomizationNeed[curlen+1];
+ final int curlen = this.fileCopyNeeds.length;
+ final FileCopyNeed[] src = this.fileCopyNeeds;
+ final FileCopyNeed[] dst = new FileCopyNeed[curlen+1];
System.arraycopy(src, 0, dst, 0, curlen);
dst[curlen] = need;
- this.customizationNeeds = dst;
+ this.fileCopyNeeds = dst;
}
}
- public synchronized CustomizationNeed[] getCustomizationNeeds() {
- return this.customizationNeeds;
+ public synchronized FileCopyNeed[] getFileCopyNeeds() {
+ return this.fileCopyNeeds;
}
- public synchronized boolean isCustomizationAllDone() {
- if (this.customizationNeeds == null) {
+ public synchronized boolean isFileCopyAllDone() {
+ if (this.fileCopyNeeds == null) {
return true;
}
- for (int i = 0; i < this.customizationNeeds.length; i++) {
- if (!this.customizationNeeds[i].isSent()) {
+ for (int i = 0; i < this.fileCopyNeeds.length; i++) {
+ if (!this.fileCopyNeeds[i].onImage()) {
return false;
}
}
@@ -258,8 +268,8 @@ public String toString() {
}
int custLen = 0;
- if (this.customizationNeeds != null) {
- custLen = this.customizationNeeds.length;
+ if (this.fileCopyNeeds != null) {
+ custLen = this.fileCopyNeeds.length;
}
boolean userDataPresent = this.mdUserData != null;
@@ -283,8 +293,8 @@ public String toString() {
}
public boolean isPropagateStartOK() {
- return this.customizationNeeds == null ||
- this.customizationNeeds.length <= 0;
+ return this.fileCopyNeeds == null ||
+ this.fileCopyNeeds.length <= 0;
}
// part of the instantiation interface, nothing about VM deployment
@@ -335,10 +345,11 @@ public static VirtualMachine cloneOne(final VirtualMachine vm)
VirtualMachineDeployment.cloneOne(vm.deployment);
}
- newvm.customizationNeeds =
- CustomizationNeed.cloneArray(vm.customizationNeeds);
+ newvm.fileCopyNeeds =
+ FileCopyNeed.cloneArray(vm.fileCopyNeeds);
newvm.mdUserData = vm.mdUserData;
+ newvm.credentialName = vm.credentialName;
return newvm;
}
View
6 service/service/java/source/src/org/globus/workspace/service/impls/InstanceResourceImpl.java
@@ -31,7 +31,7 @@
import org.globus.workspace.service.binding.authorization.CreationAuthorizationCallout;
import org.globus.workspace.service.binding.authorization.Decision;
import org.globus.workspace.service.binding.authorization.PostTaskAuthorization;
-import org.globus.workspace.service.binding.vm.CustomizationNeed;
+import org.globus.workspace.service.binding.vm.FileCopyNeed;
import org.globus.workspace.service.binding.vm.VirtualMachine;
import org.nimbustools.api.repr.ShutdownTasks;
@@ -321,11 +321,11 @@ public void setInitialVMMaccessOK(boolean accessOK) {
this.vmmAccessOK = accessOK;
}
- public synchronized void newCustomizationNeed(CustomizationNeed need) {
+ public synchronized void newFileCopyNeed(FileCopyNeed need) {
if (this.vm == null) {
throw new IllegalStateException("vm is null");
}
- this.vm.addCustomizationNeed(need);
+ this.vm.addFileCopyNeed(need);
try {
this.persistence.addCustomizationNeed(this.id, need);
} catch (ManageException e) {
View
2  service/service/java/source/src/org/globus/workspace/service/impls/StateTransition.java
@@ -779,7 +779,7 @@ private boolean propagate(final StatefulResourceImpl resource,
final WorkspaceRequestContext requestContext =
new WorkspaceRequestContext(id, resource.getName(),
this.locator, this.lager);
-
+
requestContext.setVm(resource.getVM());
requestContext.setGroupID(resource.getGroupId());
requestContext.setGroupSize(resource.getGroupSize());
View
21 service/service/java/source/src/org/globus/workspace/xen/XenUtil.java
@@ -29,7 +29,7 @@
import org.globus.workspace.service.binding.vm.VirtualMachine;
import org.globus.workspace.service.binding.vm.VirtualMachineDeployment;
import org.globus.workspace.service.binding.vm.VirtualMachinePartition;
-import org.globus.workspace.service.binding.vm.CustomizationNeed;
+import org.globus.workspace.service.binding.vm.FileCopyNeed;
import org.nimbustools.api.repr.vm.NIC;
@@ -471,22 +471,24 @@ public static ArrayList constructCreateCommand(VirtualMachine vm,
cmd.add(notificationInfo);
}
- final CustomizationNeed[] needs = vm.getCustomizationNeeds();
+ final FileCopyNeed[] needs = vm.getFileCopyNeeds();
if (needs != null) {
- if (!vm.isCustomizationAllDone()) {
+ if (!vm.isFileCopyAllDone()) {
- boolean oneBeingSent = false;
+ boolean oneBeingSentToImage = false;
final StringBuffer tasks = new StringBuffer("'");
for (int i = 0; i < needs.length; i++) {
- if (!needs[i].isSent()) {
+ // Note that we don't copy needs with no destination path
+ // these are filecopies that are used at propagation time
+ if (!needs[i].onImage() && needs[i].destPath != null) {
- if (oneBeingSent) {
+ if (oneBeingSentToImage) {
tasks.append(WC_GROUP_SEPARATOR);
}
- oneBeingSent = true;
+ oneBeingSentToImage = true;
tasks.append(needs[i].sourcePath)
.append(WC_FIELD_SEPARATOR)
@@ -501,7 +503,7 @@ public static ArrayList constructCreateCommand(VirtualMachine vm,
tasks.append("'");
- if (oneBeingSent) {
+ if (oneBeingSentToImage) {
cmd.add("--mnttasks");
cmd.add(tasks.toString());
}
@@ -777,7 +779,7 @@ private static void filePush(VirtualMachine vm,
throw new Exception("backendTargetDir needed but missing");
}
- final CustomizationNeed[] needs = vm.getCustomizationNeeds();
+ final FileCopyNeed[] needs = vm.getFileCopyNeeds();
if (needs == null || needs.length == 0) {
logger.warn("file push: nothing to do?");
return;
@@ -814,5 +816,4 @@ private static void filePush(VirtualMachine vm,
WorkspaceUtil.runCommand(send, eventLog, traceLog, vm.getID().intValue());
}
}
-
}
View
8 service/service/java/source/src/org/globus/workspace/xen/xenlocal/Start.java
@@ -19,8 +19,8 @@
import org.globus.workspace.WorkspaceException;
import org.globus.workspace.PathConfigs;
import org.globus.workspace.persistence.WorkspaceDatabaseException;
+import org.globus.workspace.service.binding.vm.FileCopyNeed;
import org.globus.workspace.service.binding.vm.VirtualMachine;
-import org.globus.workspace.service.binding.vm.CustomizationNeed;
import org.globus.workspace.xen.XenTask;
import org.globus.workspace.xen.XenUtil;
@@ -55,7 +55,7 @@ protected Exception preExecute(boolean fake) {
// init would have thrown exception if null
final VirtualMachine vm = this.ctx.getVm();
- final CustomizationNeed[] needs = vm.getCustomizationNeeds();
+ final FileCopyNeed[] needs = vm.getFileCopyNeeds();
if (needs == null || needs.length == 0) {
if (traceLog) {
logger.debug("customization file push: nothing to do");
@@ -82,9 +82,9 @@ protected Exception preExecute(boolean fake) {
final int vmid = vm.getID().intValue();
for (int i = 0; i < needs.length; i++) {
try {
- needs[i].setSent(true);
+ needs[i].setOnImage(true);
this.ctx.getLocator().getPersistenceAdapter().
- setCustomizeTaskSent(vmid, needs[i]);
+ setFileCopyOnImage(vmid, needs[i]);
} catch (WorkspaceDatabaseException e) {
logger.error("", e);
}
View
51 service/service/java/source/src/org/globus/workspace/xen/xenssh/Propagate.java
@@ -16,11 +16,15 @@
package org.globus.workspace.xen.xenssh;
+import org.globus.workspace.PathConfigs;
import org.globus.workspace.WorkspaceException;
import org.globus.workspace.cmdutils.SSHUtil;
+import org.globus.workspace.persistence.WorkspaceDatabaseException;
+import org.globus.workspace.service.binding.vm.FileCopyNeed;
import org.globus.workspace.service.binding.vm.VirtualMachine;
import org.globus.workspace.service.impls.site.PropagationAdapter;
import org.globus.workspace.xen.XenTask;
+import org.globus.workspace.xen.XenUtil;
import java.util.ArrayList;
@@ -37,6 +41,14 @@ protected void init() throws WorkspaceException {
final ArrayList exe = this.ctx.getLocator().
getPropagationAdapter().constructPropagateCommand(vm);
ssh.addAll(exe);
+
+ final String credentialName = vm.getCredentialName();
+ if (credentialName != null) {
+
+ ssh.add("--prop-extra-args");
+ ssh.add("'credential=" + credentialName + "'");
+ }
+
this.cmd = (String[]) ssh.toArray(new String[ssh.size()]);
} else {
throw new WorkspaceException("no VirtualMachine in request " +
@@ -44,9 +56,42 @@ protected void init() throws WorkspaceException {
}
}
- protected Exception preExecute() {
- return _preExecute(
- this.ctx.getLocator().getGlobalPolicies().isFake(),
+ protected Exception preExecute(boolean fake) {
+
+ final boolean eventLog = this.ctx.lager().eventLog;
+ final boolean traceLog = this.ctx.lager().traceLog;
+
+ if (traceLog) {
+ logger.trace("Beginning start pre-execute");
+ }
+
+ // init would have thrown exception if null
+ final VirtualMachine vm = this.ctx.getVm();
+
+ final FileCopyNeed[] needs = vm.getFileCopyNeeds();
+ if (needs == null || needs.length == 0) {
+ if (traceLog) {
+ logger.debug("FileCopy push: nothing to do");
+ }
+ return null;
+ }
+
+ final PathConfigs paths = this.ctx.getLocator().getPathConfigs();
+ final String backendDirectory = paths.getBackendTempDirPath();
+ final String localDirectory = paths.getLocalTempDirPath();
+
+ try {
+ XenUtil.doFilePushRemoteTarget(vm,
+ localDirectory,
+ backendDirectory,
+ fake,
+ eventLog,
+ traceLog);
+ } catch (Exception e) {
+ return e;
+ }
+
+ return _preExecute(fake,
this.ctx.getLocator().getPropagationAdapter());
}
View
38 service/service/java/source/src/org/globus/workspace/xen/xenssh/Start.java
@@ -20,8 +20,8 @@
import org.globus.workspace.PathConfigs;
import org.globus.workspace.persistence.WorkspaceDatabaseException;
import org.globus.workspace.cmdutils.SSHUtil;
+import org.globus.workspace.service.binding.vm.FileCopyNeed;
import org.globus.workspace.service.binding.vm.VirtualMachine;
-import org.globus.workspace.service.binding.vm.CustomizationNeed;
import org.globus.workspace.xen.XenTask;
import org.globus.workspace.xen.XenUtil;
@@ -47,46 +47,16 @@ protected void init() throws WorkspaceException {
protected Exception preExecute(boolean fake) {
- final boolean eventLog = this.ctx.lager().eventLog;
- final boolean traceLog = this.ctx.lager().traceLog;
-
- if (traceLog) {
- logger.trace("Beginning start pre-execute");
- }
-
- // init would have thrown exception if null
final VirtualMachine vm = this.ctx.getVm();
-
- final CustomizationNeed[] needs = vm.getCustomizationNeeds();
- if (needs == null || needs.length == 0) {
- if (traceLog) {
- logger.debug("customization file push: nothing to do");
- }
- return null;
- }
-
- final PathConfigs paths = this.ctx.getLocator().getPathConfigs();
- final String backendDirectory = paths.getBackendTempDirPath();
- final String localDirectory = paths.getLocalTempDirPath();
-
- try {
- XenUtil.doFilePushRemoteTarget(vm,
- localDirectory,
- backendDirectory,
- fake,
- eventLog,
- traceLog);
- } catch (Exception e) {
- return e;
- }
+ final FileCopyNeed[] needs = vm.getFileCopyNeeds();
// todo: do not like this concept (waiting for ORM overhaul)
final int vmid = vm.getID().intValue();
for (int i = 0; i < needs.length; i++) {
try {
- needs[i].setSent(true);
+ needs[i].setOnImage(true);
this.ctx.getLocator().getPersistenceAdapter().
- setCustomizeTaskSent(vmid, needs[i]);
+ setFileCopyOnImage(vmid, needs[i]);
} catch (WorkspaceDatabaseException e) {
logger.error("", e);
}
View
6 ...java/tests/suites/basic/home/services/share/nimbus/lib/workspace_service_derby_schema.sql
@@ -117,14 +117,14 @@ PRIMARY KEY(association,ipaddress)
);
--
--- Persistence for file customization tasks
+-- Persistence for file copy tasks
-CREATE TABLE vm_customization
+CREATE TABLE file_copy
(
vmid INT NOT NULL,
sourcepath VARCHAR(32) NOT NULL,
destpath VARCHAR(512) NOT NULL,
-sent SMALLINT NOT NULL
+on_image SMALLINT NOT NULL
);
--
Something went wrong with that request. Please try again.