elclients.html

m4_include(/mcs/m4/worksp.lib.m4)
_NIMBUS_HEADER(EC2 Client Notes)
_NIMBUS_HEADER2(n,n,y,n,n,n,n)
_NIMBUS_CENTER3_COLUMN
_NIMBUS_IS_DEPRECATED


<h2>EC2 Client Notes</h2>

<p>This section explains how to use 3rd-party EC2 and S3 clients with the Nimbus EC2 frontends, both SOAP and Query.</p>

<ul>
    <li><p><a href="#s3boto">Uploading VM Images to Cumulus with Boto</a></p></li>
    <li><p><a href="#s3cmd">Uploading VM Images to Cumulus with s3cmd</a></p></li>
    <li><p><a href="#boto">Using the EC2 Query frontend from Python with Boto</a></p></li>
    <li><p><a href="#botoex">A Full Example with Boto</a></p></li>
    <li><p><a href="#ec2-api-tools">Using the EC2 SOAP frontend from the console</a></p></li>
</ul>

<a name="s3boto"> </a>
<h2>Uploading VM Images to Cumulus with Boto _NAMELINK(s3boto)</h2>
<p>
The python library boto can be used to upload images to 
<a href="faq.html#cumulus">Cumulus</a>.  Once a VM is staged into Cumulus
it can be submitted for execution.  
</p>
<p>
In order for Nimbus to efficiently find VM images in Cumulus a naming convention
is used.  Three pieces of information are needed to follow this convention:
</p>
<ol>
    <li>The repository bucket name</li>By default this is
    <tt class="literal">Repo</tt>
    <li>The image name prefix</li>By default this is 
    <tt class="literal">VMS</tt>
    <li>Your canonical ID</li>The site admin must provide this to you at 
    the time of your account creation.  It can also be found in your 
    cloud clients configuration file (<a href="clouds/cloudquickstart.html">More Info</a>)
</ol>
<p>
That information is used to form a Cumulus url in the following way:
</p>
<pre class="panel">
cumulus://&lt;cumulus hostname&gt;/&lt;bucket name&gt;/&lt;prefix&gt;/&lt;canonical ID&gt;/&lt;image name&gt;
</pre>
<p>
Once you have the proper url formed you can use that information and 
boto to upload the image into the storage repository.
</p>
<pre class="panel">
from boto.s3.key import Key
from boto.s3.connection import OrdinaryCallingFormat
from boto.s3.connection import S3Connection

s3id="&lt;Cumulus ID&gt;"
s3pw="&lt;Cumulus password&gt;"
host="&lt;Cumulus hostname&gt;"
port=&lt;Cumulus port&gt;
canonical_id="&lt;Canonical ID&gt;"

cf = OrdinaryCallingFormat()
s3conn = S3Connection(s3id, s3pw, host=host, port=port, is_secure=False, calling_format=cf)

bucket = s3conn.get_bucket("Repo")
k = Key(bucket)
k.key = "VMS/" + canonical_id + "/" + image_name
k.set_contents_from_filename(path_to_image)
</pre>
<p>
At this point your image will be ready for submission using the name <i>image_name</i>
</p>
<div class="note"><p class="note-title">Image Names</p>
<p>
You may upload an image to any Cumulus name you wish and still submit it
for run.  This is described below.  However, the above naming convention
is strongly recommended in order to have a smooth and natural experience.
</p>
</div>
<a name="s3cmd"> </a>
<h2>Uploading VM Images to Cumulus with s3cmd _NAMELINK(s3cmd)</h2>
<p>
There is more information <a href="admin/reference.html#s3cmd">here.</a>
</p>
<a name="boto"> </a>
<h2>Using the EC2 Query frontend from Python with Boto _NAMELINK(ec2-api-tools)</h2>

<p>
    Nimbus 2.6 supports for the EC2 Query interface, which is used
    by the excellent Python client, <a href="http://code.google.com/p/boto/">boto</a>.
    In order to use this interface, you must have Query credentials from the cloud
    administrator. These are composed of an <i>access identifier</i> (usually the hash
    of your DN) and a shared <i>secret key</i>.
</p>
<p>
    You also need the URL of the Query interface on the Nimbus service node. See the
    <a href="_NIMBUS_WEBSITE/nimbus_cloud">University of Chicago cloud</a> for example.
<p>
Before you can launch a VM you must first upload it to Cumulus.  That process
is described <a href="#s3boto">here</a>
</p>

<div class="panel"><pre>
import boto
from boto.ec2.regioninfo import RegionInfo

# first create a region object and connection
region = RegionInfo(name="nimbus", endpoint="service.hostname.com")
conn =  boto.connect_ec2("YOUR_ACCESS_ID_HERE", "YOUR_SECRET_KEY_HERE",
                         port=SERVICE_PORT_NUMBER, region=region)

# then do something with the connection
conn.run_instances("image_name")
</pre></div>

<p>Please refer to the boto documentation for more details on usage.</p>
<div class="note"><p class="note-title">Image Names</p>
<p>Image names can be a full cumulus:// url, or simply the image name 
portion of the naming convention cumulus://&lt;cumulus hostname&gt;/&lt;bucket name&gt;/&lt;prefix&gt;/&lt;canonical ID&gt;/&lt;image name&gt;
</p>
</div>

<a name="botoex"> </a>
<h2>A Full Example with Boto _NAMELINK(botoex)</h2>
<div class="panel"><pre>
import boto
from boto.s3.key import Key
import time
from boto.s3.connection import OrdinaryCallingFormat
from boto.s3.connection import S3Connection
from boto.ec2.connection import EC2Connection
from boto.ec2.regioninfo import RegionInfo
    
access_id="&lt;Cumulus ID&gt;"
access_secret="&lt;Cumulus password&gt;"
host="&lt;Cumulus hostname&gt;"
port=&lt;Service port: 8444&gt;
cumulusport=&lt;Cumulus port: 8888&gt;
canonical_id="&lt;Canonical ID&gt;"

region = RegionInfo(name="nimbus", endpoint=host)
ec2conn = boto.connect_ec2(access_id, access_secret, region=region, port=port)
cf = OrdinaryCallingFormat()
s3conn = S3Connection(access_id, access_secret, host=host, port=cumulusport, is_secure=False, calling_format=cf)

# upload the file according to the Nimbus naming convention
bucket = s3conn.get_bucket("Repo")
k = boto.s3.key.Key(bucket)
k.key = "VMS/%s/%s" % (canonical_id, "my_image_name")
k.set_contents_from_filename(&lt;path to image file&gt;)

# now run the VM
reservation = ec2conn.run_instances("my_image_name")
instance = reservation.instances[0]

# poll for status
while instance.state != 'running':
    print 'instance is %s' % instance.state
    time.sleep(30)
    instance.update()

# terminate the instance
reservation.instances[0].terminate()

</pre></div>


<a name="ec2-api-tools"> </a>
<h2>Using the EC2 SOAP frontend from the console _NAMELINK(ec2-api-tools)</h2>

<p>
    When using a cloud running the <a href="faq.html#ec2-frontend">EC2
    frontend</a>, you can download this
    <a href="http://s3.amazonaws.com/ec2-downloads/ec2-api-tools-1.3-42584.zip">EC2
    client</a> from Amazon or try a number of different client that are
    <a href="http://www.google.com/search?hl=en&q=ec2%20client">out there</a>.
</p>

<div class="note"><p class="note-title">EC2 Upgrades</p>
<p>
    Amazon EC2 upgrades happen without warning and so there is sometimes
    a sync error between their default tools and the tools needed to work with
    particular Nimbus elastic services.
</p>
</div>
<ul>
	<li>
	    Nimbus 2.3+ supports the "2009-08-15" WSDL
        (<a href="http://s3.amazonaws.com/ec2-downloads/ec2-api-tools-1.3-42584.zip">this EC2
        client</a>) .
	</li>
    <li>
        Nimbus TP2.2 supports the "2008-05-05" WSDL
        (<a href="http://s3.amazonaws.com/ec2-downloads/ec2-api-tools-1.3-24159.zip">this EC2
        client</a>).
    </li>
    <li>
        The older Nimbus TP2.0 supports the "2008-02-01" WSDL
        (<a href="http://s3.amazonaws.com/ec2-downloads/ec2-api-tools-1.3-19403.zip">this EC2
        client</a>).
    </li>
    <li>
        So <a href="http://s3.amazonaws.com/ec2-downloads/ec2-api-tools.zip">the
        default client</a> may not always be the one to use.  See a specific cloud's
        documentation for the definitive tools URL (for example, the
        <a href="/nimbus_cloud">Nimbus cloud</a>).
        And see
        <a href="http://bugzilla.globus.org/globus/show_bug.cgi?id=6558">enhancement 6558</a>.
    </li>
</ul>

<br />
<hr />
<p>
    With the client from Amazon, you need to adjust the following environment variables
    (using other clients, the configurations can vary).
</p>
<ul>
    <li>
        <i>EC2_HOME</i> - Set the base directory (such that $EC2_HOME/bin is valid)
    </li>
    <li>
        <i>EC2_URL</i> - Set the service endpoint to the Nimbus based service
    </li>
    <li>
        <i>EC2_CERT</i> - Set the public credential to your public .pem
        based cert
    </li>
    <li>
        <i>EC2_PRIVATE_KEY</i> - Set the private credential to an unencrypted
        .pem based key
    </li>
</ul>

<p>
    Example:
</p>
_EXAMPLE_GENERICCMD_BEGIN
export EC2_HOME=`pwd`
_EXAMPLE_CMD_END
_EXAMPLE_GENERICCMD_BEGIN
export EC2_URL=https://HOST:PORT/wsrf/services/ElasticNimbusService
_EXAMPLE_CMD_END
_EXAMPLE_GENERICCMD_BEGIN
export EC2_CERT=/tmp/ec2clientcert.pem
_EXAMPLE_CMD_END
_EXAMPLE_GENERICCMD_BEGIN
export EC2_PRIVATE_KEY=/tmp/ec2clientkey.pem
_EXAMPLE_CMD_END


<p>
    The URL for <i>EC2_URL</i> will be provided to you by an administrator or
    some documentation
    (<i>HOST:PORT</i> in the example
    is a placeholder for an actual hostname and port number)
</p>
<p>
   For example, see the
    <a href="_SCIENCECLOUDS_WEBSITE/clouds/nimbus.html">Nimbus cloud</a>
    docs for running EC2 clients.
</p>
<p>
    If you have a problem, consult the
    <a href="admin/troubleshooting.html">troubleshooting</a> guide.
</p>

<br />
<hr />

<p>
    Before being able to launch an instance, you will need to register a keypair
    like so:
</p>
<ul>
<li>
_EXAMPLE_GENERICCMD_BEGIN
MYPUBKEY=`cat .ssh/id_rsa.pub`
_EXAMPLE_CMD_END
</li>
<li>
<p>
    Check to make sure those contents look OK:
</p>
_EXAMPLE_GENERICCMD_BEGIN
echo $MYPUBKEY
_EXAMPLE_CMD_END
</li>

<li>
<p>
    Send the public key value to the service, labelling it "mykey"
</p>
_EXAMPLE_GENERICCMD_BEGIN
ec2-add-keypair "mykey||$MYPUBKEY"
_EXAMPLE_CMD_END
</li>
</ul>

<hr />
        
<p>
    List the available images in your personal directory:
</p>
_EXAMPLE_GENERICCMD_BEGIN
ec2-describe-images
_EXAMPLE_CMD_END

<p>
    Pick one, for example "hello-cloud".  And with the "-k" argument, use the
    ssh key label you used above to register an ssh public key.
</p>
_EXAMPLE_GENERICCMD_BEGIN
ec2-run-instances -k mykey hello-cloud
_EXAMPLE_CMD_END

<p>
    Check status:
</p>
_EXAMPLE_GENERICCMD_BEGIN
ec2-describe-instances
_EXAMPLE_CMD_END

<p>
    After seeing 'pending' change to 'running', log in at the address printed:
</p>
_EXAMPLE_GENERICCMD_BEGIN
ssh <b>root</b>@abc.def.com
_EXAMPLE_CMD_END

<p>
    Terminate
    using the "instance ID" printed when you first launched (and also available
    from <i>ec2-describe-instances</i>).  It looks like "i-4662834e":
</p>
_EXAMPLE_GENERICCMD_BEGIN
ec2-terminate-instances i-4662834e
_EXAMPLE_CMD_END

<br/><br/>

<br />
<br />
<br />
<br />
<br />
<br />
<br />
<br />
        
_NIMBUS_CENTER3_COLUMN_END
_NIMBUS_FOOTER1
_NIMBUS_FOOTER2
_NIMBUS_FOOTER3