Skip to content

HTTPS clone URL

Subversion checkout URL

You can clone with HTTPS or Subversion.

Download ZIP
Browse files

merge result

Merge branch 'master' of github.com:nimbusproject/nimbus

Conflicts:
	docs/src/index.html
  • Loading branch information...
commit d9fd28319e3c41a0eb8907d462ccb603cc6ccb1b 2 parents 5ecdbe8 + 59a3688
John Bresnahan buzztroll authored
88 docs/src/changelog.html
View
@@ -22,8 +22,13 @@
<ul>
<li>
<p>
-Image propagation performance enhancing features have been introduced in
-2.8. Propagation by means of a file system copy has been introduced. This
+ This release contains many important bug fixes as well as a few new features.
+ </p>
+ </li>
+ <li>
+ <p>
+Image propagation performance enhancing features have been introduced in.
+Propagation by means of a file system copy has been introduced. This
can greatly decrease the boot time of VMs on systems where a fast shared
file system is used. We have also added support for a VM image cache on
all VMMs. This greatly increases the performance for clouds with a
@@ -36,41 +41,22 @@
<ul>
<li>
<p>
- libvirt template support added. A cloud administrator can
- now completely control the options sent to libvirt when starting
- a virtual machine by editing the template file
- /opt/nimbus/etc/workspace-control/libvirt_template.xml
- on the VMM nodes.
- </p>
- </li>
- <li>
- <p>
- A cache of propagated images can now be kept on each VMM. Before
- an image is propagated the cache is checked for an image with a
- matching checksum. If found that image is used and no propagation
- is needed which can save a significant amount of time.
- </p>
- </li>
- <li>
- <p>
- The cp (copy) propagation driver has been introduced in this release.
- This prepares an image for use by a VMM by coping it directly out
- of the Cumulus data store and into a temporary location from which it
- will be booted. For users with shared fast file systems this can
- bring great performance benefits.
+ Internally the Nimbus team has introduced a build and test system
+ with <a href="http://build.nimbusproject.org">Jenkins</a>.
+ This has helped us create a more robust and
+ well tested system as well as making it easier for the community
+ to make contributions.
</p>
</li>
<li>
<p>
- Internally the Nimbus team has introduced a build and test system
- with <i>Jenkins</i>. This has helped us create a more robust and
- well tested system as well as making it easier for the community
- to make contributions.
+ The Nimbus Context Broker has been factored out into a separate tarball install
+ in addition to being part of the default IaaS installation.
</p>
</li>
<li>
<p>
- Documentation added to the <i>zero to cloud guide</i> describing
+ Documentation added to the <a href="http://www.nimbusproject.org/docs/latest/admin/z2c/">zero to cloud guide</a> describing
how to setup Nimbus to work with KVM.
</p>
</li>
@@ -82,6 +68,11 @@
<ul>
<li>
<p>
+ <a href="https://github.com/nimbusproject/nimbus/issues/59">ehcache unreliable for critical persistence</a>
+ </p>
+ </li>
+ <li>
+ <p>
<a href="https://github.com/nimbusproject/nimbus/issues/56">Termination should be retried if there is a fatal issue</a>
</p>
</li>
@@ -115,11 +106,6 @@
<a href="https://github.com/nimbusproject/nimbus/issues/36">Implement EC2 ImportKeyPair</a>
</p>
</li>
- <li>
- <p>
- <a href="https://github.com/nimbusproject/nimbus/issues/32">Pilot updates</a>
- </p>
- </li>
</ul>
</li>
</ul>
@@ -130,7 +116,7 @@
<p>
In this release LANTorrent was repackaged for more pythonic
distribution mechanisms and an important bug was fixed that
- cased long term stability problems.
+ caused long term stability problems.
</p>
</li>
<li>
@@ -153,6 +139,11 @@
<a href="https://github.com/nimbusproject/nimbus/issues/51">Add lantorrent tests to test suite.</a>
</p>
</li>
+ <li>
+ <p>
+ <a href="https://github.com/nimbusproject/nimbus/issues/32">Pilot updates</a>
+ </p>
+ </li>
</ul>
</li>
</ul>
@@ -162,8 +153,29 @@
<ul>
<li>
<p>
+ libvirt template support added. A cloud administrator can
+ now completely control the options sent to libvirt when starting
+ a virtual machine by editing the template file
+ <tt class="literal">/opt/nimbus/etc/workspace-control/libvirt_template.xml</tt>
+ on the VMM nodes.
+ </p>
+ </li>
+ <li>
+ <p>
+ A cache of propagated images can now be kept on each VMM. Before
+ an image is propagated the cache is checked for an image with a
+ matching checksum. If found that image is used and no propagation
+ is needed which can save a significant amount of time.
+ </p>
+ </li>
+ <li>
+ <p>
+ The cp (copy) propagation driver has been introduced in this release.
+ This prepares an image for use by a VMM by copying it directly out
+ of the Cumulus data store and into a temporary location from which it
+ will be booted. For users with shared fast file systems this can
+ bring great performance benefits.
</p>
-
</li>
<li>
<p>
@@ -189,6 +201,10 @@
<ul>
<li>
<p>
+ List of
+ <a href="http://github.com/nimbusproject/nimbus/compare/nimbus-release-2.7...nimbus-release-2.8RC1">all
+ commits</a>
+ between Nimbus 2.7 final and Nimbus 2.8 RC1.
</p>
</li>
</ul>
77 service/service/java/source/share/lib/workspace_service_derby_schema.sql
View
@@ -264,12 +264,83 @@ creator_is_superuser SMALLINT,
group_id VARCHAR(512),
ssh_key_name VARCHAR(512),
creation_time BIGINT,
-nics VARCHAR(512)
+nics VARCHAR(512),
+status VARCHAR(512)
);
-- Persistence for AsyncRequest list of NICs
+--CREATE TABLE async_requests_vms
+--(
+--id VARCHAR(512),
+--vmid INT
+--);
+
+--
+-- Persistence for async virtual machines:
+
CREATE TABLE async_requests_vms
(
-id VARCHAR(512),
-vmid INT
+async_id VARCHAR(512) NOT NULL,
+binding_index INT NOT NULL,
+id INT NOT NULL,
+name VARCHAR(128) NOT NULL,
+node VARCHAR(128),
+prop_required SMALLINT NOT NULL,
+unprop_required SMALLINT NOT NULL,
+network VARCHAR(1024),
+kernel_parameters VARCHAR(128),
+vmm VARCHAR(32),
+vmm_version VARCHAR(32),
+assocs_needed VARCHAR(256),
+md_user_data VARCHAR(30720),
+preemptable SMALLINT,
+credential_name VARCHAR(128),
+PRIMARY KEY (async_id, binding_index, id)
+);
+
+--
+-- Persistence for async vm deployment-time data:
+
+CREATE TABLE async_requests_vm_deployment
+(
+async_id VARCHAR(512) NOT NULL,
+binding_index INT NOT NULL,
+vmid INT NOT NULL,
+requested_state SMALLINT,
+requested_shutdown SMALLINT,
+min_duration INT,
+ind_physmem INT,
+ind_physcpu INT
+);
+
+--
+-- async VM partitions
+
+CREATE TABLE async_requests_vm_partitions
+(
+async_id VARCHAR(512) NOT NULL,
+binding_index INT NOT NULL,
+vmid INT NOT NULL,
+image VARCHAR(4096) NOT NULL,
+imagemount VARCHAR(128) NOT NULL,
+readwrite SMALLINT NOT NULL,
+rootdisk SMALLINT NOT NULL,
+blankspace INT NOT NULL,
+prop_required SMALLINT NOT NULL,
+unprop_required SMALLINT NOT NULL,
+alternate_unprop VARCHAR(128)
+);
+
+--
+-- Persistence for async vm file copy tasks
+
+CREATE TABLE async_requests_vm_file_copy
+(
+async_id VARCHAR(512) NOT NULL,
+binding_index INT NOT NULL,
+vmid INT NOT NULL,
+sourcepath VARCHAR(36) NOT NULL,
+destpath VARCHAR(512),
+on_image SMALLINT NOT NULL
);
+
2  service/service/java/source/src/org/globus/workspace/async/AsyncRequestStatus.java
View
@@ -24,6 +24,6 @@ public boolean isCancelled() {
public boolean isFailed() {
return this.equals(FAILED);
- }
+ }
}
42 service/service/java/source/src/org/globus/workspace/persistence/PersistenceAdapterConstants.java
View
@@ -245,23 +245,51 @@
"DELETE FROM idempotency WHERE creator_dn=? AND client_token=?";
public static final String SQL_INSERT_ASYNC_REQUEST =
- "INSERT INTO async_requests (id, max_bid, spot, group_id, persistent, creator_dn, creator_is_superuser, ssh_key_name, creation_time, nics) " +
- " VALUES (?,?,?,?,?,?,?,?,?,?)";
+ "INSERT INTO async_requests (id, max_bid, spot, group_id, persistent, creator_dn, creator_is_superuser, ssh_key_name, creation_time, nics, status) " +
+ " VALUES (?,?,?,?,?,?,?,?,?,?,?)";
public static final String SQL_LOAD_ASYNC_REQUEST =
- "SELECT id, max_bid, spot, group_id, persistent, creator_dn, creator_is_superuser, ssh_key_name, creation_time, nics FROM async_requests WHERE id=?";
+ "SELECT id, max_bid, spot, group_id, persistent, creator_dn, creator_is_superuser, ssh_key_name, creation_time, nics, status FROM async_requests WHERE id=?";
public static final String SQL_LOAD_ALL_ASYNC_REQUESTS =
- "SELECT id, max_bid, spot, group_id, persistent, creator_dn, creator_is_superuser, ssh_key_name, creation_time, nics FROM async_requests";
+ "SELECT id, max_bid, spot, group_id, persistent, creator_dn, creator_is_superuser, ssh_key_name, creation_time, nics, status FROM async_requests";
public static final String SQL_LOAD_ASYNC_REQUESTS_VMS =
- "SELECT vmid FROM async_requests_vms WHERE id=?";
+ "SELECT async_id, binding_index, id, name, node, prop_required, unprop_required, network, kernel_parameters, vmm, vmm_version, assocs_needed, md_user_data, preemptable, credential_name FROM async_requests_vms WHERE async_id=?";
public static final String SQL_INSERT_ASYNC_REQUESTS_VMS =
- "INSERT INTO async_requests_vms (id,vmid) VALUES (?,?)";
+ "INSERT INTO async_requests_vms " +
+ "(async_id, binding_index, id, name, node, prop_required, unprop_required, network, kernel_parameters, vmm, vmm_version, assocs_needed, md_user_data, preemptable, credential_name) " +
+ " VALUES (?,?,?,?,?,?,?,?,?,?,?,?,?,?,?)";
public static final String SQL_UPDATE_ASYNC_REQUEST =
- "UPDATE async_requests SET id=?";
+ "UPDATE async_requests SET id=?, max_bid=?, spot=?, group_id=?, persistent=?, creator_dn=?, creator_is_superuser=?, ssh_key_name=?, creation_time=?, nics=?, status=?";
+
+ public static final String SQL_LOAD_ASYNC_REQUESTS_VM_DEPLOYMENT =
+ "SELECT requested_state, requested_shutdown, min_duration, " +
+ "ind_physmem, ind_physcpu " +
+ "FROM async_requests_vm_deployment WHERE async_id=? AND binding_index=?";
+
+ public static final String SQL_LOAD_ASYNC_REQUESTS_VM_PARTITIONS =
+ "SELECT image, imagemount, readwrite, rootdisk, blankspace, " +
+ "prop_required, unprop_required, alternate_unprop " +
+ "FROM async_requests_vm_partitions WHERE async_id=? AND binding_index=?";
+
+ public static final String SQL_LOAD_ASYNC_REQUESTS_VM_FILE_COPY =
+ "SELECT sourcepath, destpath, on_image " +
+ "FROM async_requests_vm_file_copy WHERE async_id=? AND binding_index=?";
+
+ public static final String SQL_DELETE_ASYNC_REQUESTS_VMS =
+ "DELETE FROM async_requests_vms WHERE async_id=?";
+
+ public static final String SQL_DELETE_ASYNC_REQUESTS_VM_DEPLOYMENT =
+ "DELETE FROM async_requests_vm_deployment WHERE async_id=?";
+
+ public static final String SQL_DELETE_ASYNC_REQUESTS_VM_PARTITIONS =
+ "DELETE FROM async_requests_vm_partitions WHERE async_id=?";
+
+ public static final String SQL_DELETE_ASYNC_REQUESTS_VM_FILE_COPY =
+ "DELETE FROM async_requests_vm_file_copy WHERE async_id=?";
public static final String[] PREPARED_STATEMENTS = {
SQL_SELECT_RESOURCES,
68 service/service/java/source/src/org/globus/workspace/persistence/PersistenceAdapterImpl.java
View
@@ -1504,13 +1504,12 @@ public void load(int id, InstanceResource resource)
}
}
- public VirtualMachine loadVM(int id) throws SQLException, DoesNotExistException, WorkspaceDatabaseException {
+ public VirtualMachine loadVM(int id, Connection c) throws SQLException, DoesNotExistException, WorkspaceDatabaseException {
if (this.dbTrace) {
logger.trace(Lager.id(id) + ": load virtual machine");
}
- Connection c = getConnection();
PreparedStatement[] pstmts = VirtualMachinePersistenceUtil.getVMQuery(id, c);
ResultSet rs = pstmts[0].executeQuery();
if (rs == null || !rs.next()) {
@@ -3235,38 +3234,40 @@ public void addAsyncRequest(AsyncRequest asyncRequest)
PreparedStatement[] pstmts = null;
try {
c = getConnection();
- logger.debug("SETTING spot = " + asyncRequest.isSpotRequest());
- pstmt = AsyncRequestMapPersistenceUtil.getInsertAsyncRequest(asyncRequest, this.repr, c);
- pstmt.executeUpdate();
- VirtualMachine[] bindings = asyncRequest.getBindings();
- if (bindings != null) {
+ AsyncRequest oldAsyncRequest = getAsyncRequest(asyncRequest.getId());
- for (VirtualMachine vm : asyncRequest.getBindings()) {
+ if (oldAsyncRequest != null) {
+ logger.debug("Updating old request " + oldAsyncRequest.getId());
+ pstmt = AsyncRequestMapPersistenceUtil.getUpdateAsyncRequest(asyncRequest, this.repr, c);
+ pstmt.executeUpdate();
+ pstmt.close();
- pstmts = VirtualMachinePersistenceUtil.
- getInsertVM(vm, vm.getID(), c);
- if (this.dbTrace) {
- logger.trace("creating VirtualMachine db " +
- "entry for " + Lager.id(vm.getID()) + ": " +
- pstmts.length + " inserts");
- }
+ // We will later need to persist modified versions of our persisted VMs,
+ // so we need to remove all of our earlier copies
+ pstmts = AsyncRequestMapPersistenceUtil.getRemoveAsyncVMs(asyncRequest, c);
+ for (PreparedStatement p : pstmts) {
- for (int i = 0; i < pstmts.length; i++) {
- pstmts[i].executeUpdate();
- }
-
- pstmt = AsyncRequestMapPersistenceUtil.getInsertAsyncRequestVM(asyncRequest.getId(), vm.getID(), c);
- pstmt.executeUpdate();
+ p.executeUpdate();
}
+
+ pstmt = AsyncRequestMapPersistenceUtil.getDeleteAsyncRequestVMs(asyncRequest, c);
+ pstmt.executeUpdate();
}
+ else {
+ logger.debug("Persisting request: " + asyncRequest.getId());
+ pstmt = AsyncRequestMapPersistenceUtil.getInsertAsyncRequest(asyncRequest, this.repr, c);
+ pstmt.executeUpdate();
+ }
+
+ AsyncRequestMapPersistenceUtil.putAsyncRequestBindings(asyncRequest, c);
c.commit();
- } catch (ManageException e) {
+ } catch (SQLException e) {
logger.error("",e);
throw new WorkspaceDatabaseException(e);
- } catch (SQLException e) {
+ } catch (ManageException e) {
logger.error("",e);
throw new WorkspaceDatabaseException(e);
} finally {
@@ -3311,19 +3312,12 @@ public AsyncRequest getAsyncRequest(String id)
}
asyncRequest = AsyncRequestMapPersistenceUtil.rsToAsyncRequest(rs, this.repr, c);
- ArrayList<VirtualMachine> bindings = new ArrayList<VirtualMachine>();
- for (int vmid : AsyncRequestMapPersistenceUtil.getVMIDs(asyncRequest.getId(), c)) {
- bindings.add(loadVM(vmid));
- }
- VirtualMachine[] newBindings = bindings.toArray(new VirtualMachine[bindings.size()]);
- asyncRequest.setBindings(newBindings);
+ VirtualMachine[] bindings = AsyncRequestMapPersistenceUtil.getAsyncVMs(asyncRequest.getId(), c);
+ asyncRequest.setBindings(bindings);
} catch (SQLException e) {
logger.error("",e);
throw new WorkspaceDatabaseException(e);
- } catch (DoesNotExistException e) {
- logger.error("",e);
- throw new WorkspaceDatabaseException(e);
} catch (CannotTranslateException e) {
logger.error("",e);
throw new WorkspaceDatabaseException(e);
@@ -3343,6 +3337,7 @@ public AsyncRequest getAsyncRequest(String id)
return asyncRequest;
}
+
public ArrayList<AsyncRequest> getAllAsyncRequests()
throws WorkspaceDatabaseException {
@@ -3363,9 +3358,14 @@ public AsyncRequest getAsyncRequest(String id)
do {
String id = rs.getString("id");
- asyncRequests.add(getAsyncRequest(id));
+ AsyncRequest asyncRequest = AsyncRequestMapPersistenceUtil.rsToAsyncRequest(rs, this.repr, c);
+ VirtualMachine[] bindings = AsyncRequestMapPersistenceUtil.getAsyncVMs(asyncRequest.getId(), c);
+ asyncRequest.setBindings(bindings);
+ asyncRequests.add(asyncRequest);
} while(rs.next());
-
+ } catch (CannotTranslateException e) {
+ logger.error("",e);
+ throw new WorkspaceDatabaseException(e);
} catch (SQLException e) {
logger.error("",e);
throw new WorkspaceDatabaseException(e);
245 service/service/java/source/src/org/globus/workspace/persistence/impls/AsyncRequestMapPersistenceUtil.java
View
@@ -17,10 +17,16 @@
package org.globus.workspace.persistence.impls;
import org.globus.workspace.async.AsyncRequest;
+import org.globus.workspace.async.AsyncRequestStatus;
import org.globus.workspace.network.Association;
import org.globus.workspace.network.AssociationEntry;
import org.globus.workspace.persistence.DataConvert;
import org.globus.workspace.persistence.PersistenceAdapterConstants;
+import org.globus.workspace.persistence.WorkspaceDatabaseException;
+import org.globus.workspace.service.binding.vm.FileCopyNeed;
+import org.globus.workspace.service.binding.vm.VirtualMachine;
+import org.globus.workspace.service.binding.vm.VirtualMachineDeployment;
+import org.globus.workspace.service.binding.vm.VirtualMachinePartition;
import org.nimbustools.api._repr._Caller;
import org.nimbustools.api.repr.Caller;
import org.nimbustools.api.repr.CannotTranslateException;
@@ -29,13 +35,11 @@
import org.nimbustools.api.repr.vm.NIC;
import org.nimbustools.api.services.rm.ManageException;
-import javax.xml.crypto.Data;
import java.sql.Connection;
import java.sql.PreparedStatement;
import java.sql.ResultSet;
import java.sql.SQLException;
import java.util.*;
-import java.util.logging.Logger;
public class AsyncRequestMapPersistenceUtil
implements PersistenceAdapterConstants {
@@ -77,6 +81,49 @@ public static PreparedStatement getInsertAsyncRequest(AsyncRequest asyncRequest,
DataConvert dataConvert = new DataConvert(repr);
String nics = dataConvert.nicsAsString(asyncRequest.getRequestedNics());
pstmt.setString(10, nics);
+
+ pstmt.setString(11, asyncRequest.getStatus().toString());
+ return pstmt;
+ }
+
+ public static PreparedStatement getUpdateAsyncRequest(AsyncRequest asyncRequest, ReprFactory repr, Connection c)
+ throws SQLException {
+
+ final PreparedStatement pstmt = c.prepareStatement(SQL_UPDATE_ASYNC_REQUEST);
+ pstmt.setString(1, asyncRequest.getId());
+ pstmt.setDouble(2, asyncRequest.getMaxBid());
+ if (asyncRequest.isSpotRequest()) {
+ pstmt.setInt(3, 1);
+ }
+ else {
+ pstmt.setInt(3, 0);
+ }
+ pstmt.setString(4, asyncRequest.getGroupID());
+ if (asyncRequest.isPersistent()) {
+ pstmt.setInt(5, 1);
+ }
+ else {
+ pstmt.setInt(5, 0);
+ }
+ pstmt.setString(6, asyncRequest.getCaller().getIdentity());
+ if (asyncRequest.getCaller().isSuperUser()) {
+ pstmt.setInt(7, 1);
+ }
+ else {
+ pstmt.setInt(7, 0);
+ }
+ pstmt.setString(8, asyncRequest.getSshKeyName());
+
+ if (asyncRequest.getCreationTime() != null) {
+ pstmt.setLong(9, asyncRequest.getCreationTime().getTimeInMillis());
+ }
+ else {
+ pstmt.setInt(9,0);
+ }
+ DataConvert dataConvert = new DataConvert(repr);
+ String nics = dataConvert.nicsAsString(asyncRequest.getRequestedNics());
+ pstmt.setString(10, nics);
+ pstmt.setString(11, asyncRequest.getStatus().toString());
return pstmt;
}
@@ -97,6 +144,14 @@ public static PreparedStatement getAllAsyncRequests(Connection c)
return pstmt;
}
+ public static PreparedStatement getDeleteAsyncRequestVMs(AsyncRequest asyncRequest, Connection c)
+ throws SQLException {
+
+ final PreparedStatement pstmt = c.prepareStatement(SQL_DELETE_ASYNC_REQUESTS_VMS);
+ pstmt.setString(1, asyncRequest.getId());
+ return pstmt;
+ }
+
public static AsyncRequest rsToAsyncRequest(ResultSet rs, ReprFactory repr, Connection c)
throws SQLException, CannotTranslateException {
@@ -122,29 +177,182 @@ public static AsyncRequest rsToAsyncRequest(ResultSet rs, ReprFactory repr, Conn
DataConvert dataConvert = new DataConvert(repr);
NIC[] nics = null;
nics = dataConvert.getNICs(nicsAsString);
+ AsyncRequestStatus status = AsyncRequestStatus.valueOf(rs.getString("status"));
//public AsyncRequest(String id, boolean spotinstances, Double spotPrice, boolean persistent, Caller caller, String groupID, VirtualMachine[] bindings, Context context, NIC[] requestedNics, String sshKeyName, Calendar creationTime) {
//AsyncRequest testRequest = new AsyncRequest(testID, testSpotinstances, testMaxBid, false, null, testGroupID, null, null, null, null, null);
- return new AsyncRequest(id, isSpotInstance, maxBid, isPersistent, caller, groupID, null, context, nics, sshKeyName, creationTime);
+ AsyncRequest asyncRequest = new AsyncRequest(id, isSpotInstance, maxBid, isPersistent, caller, groupID, null, context, nics, sshKeyName, creationTime);
+
+ asyncRequest.setStatus(status);
+ return asyncRequest;
}
- public static ArrayList<Integer> getVMIDs(String id, Connection c) throws SQLException{
+ public static VirtualMachine[] getAsyncVMs(String asyncID, Connection c) throws SQLException, WorkspaceDatabaseException{
final PreparedStatement pstmt = c.prepareStatement(SQL_LOAD_ASYNC_REQUESTS_VMS);
- pstmt.setString(1, id);
+ pstmt.setString(1, asyncID);
ResultSet rs = pstmt.executeQuery();
- ArrayList<Integer> ids = new ArrayList<Integer>();
+ ArrayList<VirtualMachine> vms = new ArrayList<VirtualMachine>();
if (rs == null || !rs.next()) {
- return ids;
+ return vms.toArray(new VirtualMachine[vms.size()]);
}
do {
- ids.add(rs.getInt(1));
+ VirtualMachine newVM = new VirtualMachine();
+ newVM.setID(rs.getInt("id"));
+ newVM.setName(rs.getString("name"));
+ newVM.setNode(rs.getString("node"));
+ newVM.setPropagateRequired(rs.getBoolean("prop_required"));
+ newVM.setUnPropagateRequired(rs.getBoolean("unprop_required"));
+ newVM.setNetwork(rs.getString("network"));
+ newVM.setKernel(rs.getString("kernel_parameters"));
+ newVM.setVmm(rs.getString("vmm"));
+ newVM.setVmmVersion(rs.getString("vmm_version"));
+ newVM.setAssociationsNeeded(rs.getString("assocs_needed"));
+ newVM.setMdUserData(rs.getString("md_user_data"));
+ newVM.setPreemptable(rs.getBoolean("preemptable"));
+ newVM.setCredentialName(rs.getString("credential_name"));
+
+ int binding_index = rs.getInt("binding_index");
+
+ addDeployment(newVM, asyncID, binding_index, c);
+ addPartitions(newVM, asyncID, binding_index, c);
+ addFileCopies(newVM, asyncID, binding_index, c);
+
+ vms.add(newVM);
+ } while (rs.next());
+
+ rs.close();
+ pstmt.close();
+
+ VirtualMachine[] final_vms = vms.toArray(new VirtualMachine[vms.size()]);
+ return final_vms;
+ }
+
+ public static void addDeployment(VirtualMachine vm, String asyncID, int binding_index, Connection c) throws SQLException {
+
+ final PreparedStatement pstmt = c.prepareStatement(SQL_LOAD_ASYNC_REQUESTS_VM_DEPLOYMENT);
+ pstmt.setString(1, asyncID);
+ pstmt.setInt(2, binding_index);
+ ResultSet rs = pstmt.executeQuery();
+
+ if (rs == null || !rs.next()) {
+ //No deployment for this VM
+ return;
+ }
+
+ final VirtualMachineDeployment dep = new VirtualMachineDeployment();
+ dep.setRequestedState(rs.getInt("requested_state"));
+ dep.setRequestedShutdown(rs.getInt("requested_shutdown"));
+ dep.setMinDuration(rs.getInt("min_duration"));
+ dep.setIndividualPhysicalMemory(rs.getInt("ind_physmem"));
+ dep.setIndividualCPUCount(rs.getInt("ind_physcpu"));
+ vm.setDeployment(dep);
+ }
+
+ public static void addPartitions(VirtualMachine vm, String asyncID, int binding_index, Connection c) throws SQLException {
+
+ final PreparedStatement pstmt = c.prepareStatement(SQL_LOAD_ASYNC_REQUESTS_VM_PARTITIONS);
+ pstmt.setString(1, asyncID);
+ pstmt.setInt(2, binding_index);
+ ResultSet rs = pstmt.executeQuery();
+
+ if (rs == null || !rs.next()) {
+ //No partitions for this VM
+ return;
+ }
+
+ final ArrayList partitions = new ArrayList(8);
+
+ do {
+ final VirtualMachinePartition partition =
+ new VirtualMachinePartition();
+ partition.setImage(rs.getString("image"));
+ partition.setImagemount(rs.getString("imagemount"));
+ partition.setReadwrite(rs.getBoolean("readwrite"));
+ partition.setRootdisk(rs.getBoolean("rootdisk"));
+ partition.setBlankspace(rs.getInt("blankspace"));
+ partition.setPropRequired(rs.getBoolean("prop_required"));
+ partition.setUnPropRequired(rs.getBoolean("unprop_required"));
+ partition.setAlternateUnpropTarget(rs.getString("alternate_unprop"));
+
+ partitions.add(partition);
} while (rs.next());
- return ids;
+ VirtualMachinePartition[] final_partitions = new VirtualMachinePartition[partitions.size()];
+ vm.setPartitions(final_partitions);
+ }
+
+ public static void addFileCopies(VirtualMachine vm, String asyncID, int binding_index, Connection c) throws SQLException, WorkspaceDatabaseException {
+
+ final PreparedStatement pstmt = c.prepareStatement(SQL_LOAD_ASYNC_REQUESTS_VM_FILE_COPY);
+ pstmt.setString(1, asyncID);
+ pstmt.setInt(2, binding_index);
+ ResultSet rs = pstmt.executeQuery();
+
+ if (rs == null || !rs.next()) {
+ //No file copies for this VM
+ return;
+ }
+
+ do {
+ String sourcePath = rs.getString("sourcepath");
+ String destPath = rs.getString("destpath");
+ boolean onImage = rs.getBoolean("on_image");
+ try {
+ FileCopyNeed need = new FileCopyNeed(sourcePath, destPath, onImage);
+ vm.addFileCopyNeed(need);
+ } catch(Exception e) {
+ throw new WorkspaceDatabaseException("", e);
+ }
+ } while(rs.next());
+ }
+
+ public static void putAsyncRequestBindings(AsyncRequest asyncRequest, Connection c) throws SQLException {
+
+ VirtualMachine[] bindings = asyncRequest.getBindings();
+ for (int i=0; i<bindings.length; i++) {
+
+ VirtualMachine binding = bindings[i];
+ final PreparedStatement pstmt = c.prepareStatement(SQL_INSERT_ASYNC_REQUESTS_VMS);
+ pstmt.setString(1, asyncRequest.getId());
+ pstmt.setInt(2, i);
+ pstmt.setInt(3, binding.getID());
+ pstmt.setString(4, binding.getName());
+ pstmt.setString(5, binding.getNode());
+ boolean propRequired = binding.isPropagateRequired();
+ if (propRequired) {
+ pstmt.setInt(6, 1);
+ }
+ else {
+ pstmt.setInt(6, 0);
+ }
+ boolean unpropRequired = binding.isUnPropagateRequired();
+ if (unpropRequired) {
+ pstmt.setInt(7, 1);
+ }
+ else {
+ pstmt.setInt(7, 0);
+ }
+ pstmt.setString(8, binding.getNetwork());
+ pstmt.setString(9, binding.getKernelParameters());
+ pstmt.setString(10, binding.getVmm());
+ pstmt.setString(11, binding.getVmmVersion());
+ pstmt.setString(12, binding.getAssociationsNeeded());
+ pstmt.setString(13, binding.getMdUserData());
+ boolean preemptable = binding.isPreemptable();
+ if (preemptable) {
+ pstmt.setInt(14, 1);
+ }
+ else {
+ pstmt.setInt(14, 0);
+ }
+ pstmt.setString(15, binding.getCredentialName());
+ pstmt.executeUpdate();
+ pstmt.close();
+ }
}
public static PreparedStatement getInsertAsyncRequestVM(String id, int vmid, Connection c) throws SQLException {
@@ -154,4 +362,23 @@ public static PreparedStatement getInsertAsyncRequestVM(String id, int vmid, Con
pstmt.setInt(2, vmid);
return pstmt;
}
+
+ public static PreparedStatement[] getRemoveAsyncVMs(AsyncRequest asyncRequest, Connection c) throws SQLException {
+
+ PreparedStatement[] pstmts = new PreparedStatement[4];
+
+ pstmts[0] = c.prepareStatement(SQL_DELETE_ASYNC_REQUESTS_VMS);
+ pstmts[0].setString(1, asyncRequest.getId());
+
+ pstmts[1] = c.prepareStatement(SQL_DELETE_ASYNC_REQUESTS_VM_DEPLOYMENT);
+ pstmts[1].setString(1, asyncRequest.getId());
+
+ pstmts[2] = c.prepareStatement(SQL_DELETE_ASYNC_REQUESTS_VM_PARTITIONS);
+ pstmts[2].setString(1, asyncRequest.getId());
+
+ pstmts[3] = c.prepareStatement(SQL_DELETE_ASYNC_REQUESTS_VM_FILE_COPY);
+ pstmts[3].setString(1, asyncRequest.getId());
+
+ return pstmts;
+ }
}
77 service/service/java/tests/suites/basic/home/services/share/nimbus/lib/workspace_service_derby_schema.sql
View
@@ -264,12 +264,83 @@ creator_is_superuser SMALLINT,
group_id VARCHAR(512),
ssh_key_name VARCHAR(512),
creation_time BIGINT,
-nics VARCHAR(512)
+nics VARCHAR(512),
+status VARCHAR(512)
);
-- Persistence for AsyncRequest list of NICs
+--CREATE TABLE async_requests_vms
+--(
+--id VARCHAR(512),
+--vmid INT
+--);
+
+--
+-- Persistence for async virtual machines:
+
CREATE TABLE async_requests_vms
(
-id VARCHAR(512),
-vmid INT
+async_id VARCHAR(512) NOT NULL,
+binding_index INT NOT NULL,
+id INT NOT NULL,
+name VARCHAR(128) NOT NULL,
+node VARCHAR(128),
+prop_required SMALLINT NOT NULL,
+unprop_required SMALLINT NOT NULL,
+network VARCHAR(1024),
+kernel_parameters VARCHAR(128),
+vmm VARCHAR(32),
+vmm_version VARCHAR(32),
+assocs_needed VARCHAR(256),
+md_user_data VARCHAR(30720),
+preemptable SMALLINT,
+credential_name VARCHAR(128),
+PRIMARY KEY (async_id, binding_index, id)
+);
+
+--
+-- Persistence for async vm deployment-time data:
+
+CREATE TABLE async_requests_vm_deployment
+(
+async_id VARCHAR(512) NOT NULL,
+binding_index INT NOT NULL,
+vmid INT NOT NULL,
+requested_state SMALLINT,
+requested_shutdown SMALLINT,
+min_duration INT,
+ind_physmem INT,
+ind_physcpu INT
+);
+
+--
+-- async VM partitions
+
+CREATE TABLE async_requests_vm_partitions
+(
+async_id VARCHAR(512) NOT NULL,
+binding_index INT NOT NULL,
+vmid INT NOT NULL,
+image VARCHAR(4096) NOT NULL,
+imagemount VARCHAR(128) NOT NULL,
+readwrite SMALLINT NOT NULL,
+rootdisk SMALLINT NOT NULL,
+blankspace INT NOT NULL,
+prop_required SMALLINT NOT NULL,
+unprop_required SMALLINT NOT NULL,
+alternate_unprop VARCHAR(128)
+);
+
+--
+-- Persistence for async vm file copy tasks
+
+CREATE TABLE async_requests_vm_file_copy
+(
+async_id VARCHAR(512) NOT NULL,
+binding_index INT NOT NULL,
+vmid INT NOT NULL,
+sourcepath VARCHAR(36) NOT NULL,
+destpath VARCHAR(512),
+on_image SMALLINT NOT NULL
);
+
1  .../service/java/tests/suites/basic/src/org/globus/workspace/testing/suites/basic/AsyncRequestMapSuite.java
View
@@ -136,6 +136,7 @@ public void persistOne() throws Exception {
asyncRequestMap.addOrReplace(testRequest);
allRequests = asyncRequestMap.getAll();
+ assert(allRequests != null);
logger.debug("You have " + allRequests.size() + " requests.");
assert(allRequests.size() == 1);
18 service/service/java/tests/suites/failure/home/services/share/nimbus/lib/workspace_service_derby_schema.sql
View
@@ -255,5 +255,21 @@ instance_mem SMALLINT NOT NULL
-- Persistence for AsyncRequests
CREATE TABLE async_requests
(
-id VARCHAR(512) NOT NULL PRIMARY KEY
+id VARCHAR(512) NOT NULL PRIMARY KEY,
+max_bid DOUBLE,
+spot SMALLINT,
+persistent SMALLINT,
+creator_dn VARCHAR(512),
+creator_is_superuser SMALLINT,
+group_id VARCHAR(512),
+ssh_key_name VARCHAR(512),
+creation_time BIGINT,
+nics VARCHAR(512)
+);
+
+-- Persistence for AsyncRequest list of NICs
+CREATE TABLE async_requests_vms
+(
+id VARCHAR(512),
+vmid INT
);
2  service/service/java/tests/suites/spotinstances/home/services/etc/nimbus/workspace-service/other/main.xml
View
@@ -600,7 +600,7 @@
<bean id="nimbus-rm.si.pricingmodel" class="$ASYNC{si.pricingmodel}" />
<bean id="nimbus-rm.si.asyncreqmap" class="org.globus.workspace.async.AsyncRequestMap">
- <constructor-arg value="$COMMON{caches.dir}" />
+ <constructor-arg ref="nimbus-rm.persistence.PersistenceAdapter" />
</bean>
<bean id="nimbus-rm.async.manager"
94 ...ervice/java/tests/suites/spotinstances/home/services/share/nimbus/lib/workspace_service_derby_schema.sql
View
@@ -250,3 +250,97 @@ site_capacity INT NOT NULL,
repo_user VARCHAR(512) NOT NULL,
instance_mem SMALLINT NOT NULL
);
+
+--
+-- Persistence for AsyncRequests
+CREATE TABLE async_requests
+(
+id VARCHAR(512) NOT NULL PRIMARY KEY,
+max_bid DOUBLE,
+spot SMALLINT,
+persistent SMALLINT,
+creator_dn VARCHAR(512),
+creator_is_superuser SMALLINT,
+group_id VARCHAR(512),
+ssh_key_name VARCHAR(512),
+creation_time BIGINT,
+nics VARCHAR(512),
+status VARCHAR(512)
+);
+
+-- Persistence for AsyncRequest list of NICs
+--CREATE TABLE async_requests_vms
+--(
+--id VARCHAR(512),
+--vmid INT
+--);
+
+--
+-- Persistence for async virtual machines:
+
+CREATE TABLE async_requests_vms
+(
+async_id VARCHAR(512) NOT NULL,
+binding_index INT NOT NULL,
+id INT NOT NULL,
+name VARCHAR(128) NOT NULL,
+node VARCHAR(128),
+prop_required SMALLINT NOT NULL,
+unprop_required SMALLINT NOT NULL,
+network VARCHAR(1024),
+kernel_parameters VARCHAR(128),
+vmm VARCHAR(32),
+vmm_version VARCHAR(32),
+assocs_needed VARCHAR(256),
+md_user_data VARCHAR(30720),
+preemptable SMALLINT,
+credential_name VARCHAR(128),
+PRIMARY KEY (async_id, binding_index, id)
+);
+
+--
+-- Persistence for async vm deployment-time data:
+
+CREATE TABLE async_requests_vm_deployment
+(
+async_id VARCHAR(512) NOT NULL,
+binding_index INT NOT NULL,
+vmid INT NOT NULL,
+requested_state SMALLINT,
+requested_shutdown SMALLINT,
+min_duration INT,
+ind_physmem INT,
+ind_physcpu INT
+);
+
+--
+-- async VM partitions
+
+CREATE TABLE async_requests_vm_partitions
+(
+async_id VARCHAR(512) NOT NULL,
+binding_index INT NOT NULL,
+vmid INT NOT NULL,
+image VARCHAR(4096) NOT NULL,
+imagemount VARCHAR(128) NOT NULL,
+readwrite SMALLINT NOT NULL,
+rootdisk SMALLINT NOT NULL,
+blankspace INT NOT NULL,
+prop_required SMALLINT NOT NULL,
+unprop_required SMALLINT NOT NULL,
+alternate_unprop VARCHAR(128)
+);
+
+--
+-- Persistence for async vm file copy tasks
+
+CREATE TABLE async_requests_vm_file_copy
+(
+async_id VARCHAR(512) NOT NULL,
+binding_index INT NOT NULL,
+vmid INT NOT NULL,
+sourcepath VARCHAR(36) NOT NULL,
+destpath VARCHAR(512),
+on_image SMALLINT NOT NULL
+);
+
Please sign in to comment.
Something went wrong with that request. Please try again.