grempe / amazon-ec2

Click here to lend your support to: amazon-ec2 and make a donation at www.pledgie.com !

A Ruby Gem that gives you full access to several of the Amazon Web Services API from your Ruby/Ruby on Rails apps

This URL has Read+Write access

Updated gemspec for perfools.
grempe (author)
Sat Jun 13 11:16:56 -0700 2009
commit  97796041091a7ef43fdaffa2e36e71354b9437a3
tree    ef2e6460d65fb844a70aacee080656e308a65eb3
parent  bf7b7a80c370a7ee37a489f678b28b2534fad89b
amazon-ec2 / README.rdoc
raw normal view history
d84bb93d » grempe 2007-06-03 r1567@macbook-pro: glenn ... 1 = Amazon Web Services Elastic Compute Cloud (EC2) Ruby Gem
2
3 == About amazon-ec2
4
dc68752c » grempe 2008-04-25 Updated README 5 Amazon Web Services offers a compute power on demand capability known as the Elastic Compute Cloud (EC2). The server resources in the cloud can be provisioned on demand by making HTTP Query API calls to EC2.
d84bb93d » grempe 2007-06-03 r1567@macbook-pro: glenn ... 6
6385fd42 » grempe 2009-06-06 Smart quote issue in README. 7 This 'amazon-ec2' Ruby Gem is an interface library that can be used to interact with the Amazon EC2 system and control server resources on demand from your Ruby scripts, or from applications written in your Ruby framework of choice (Ruby on Rails, Merb, etc.).
64cf916a » grempe 2008-04-25 Update the readme to reflec... 8
dc68752c » grempe 2008-04-25 Updated README 9 For the most complete and up-to date README information please visit the project homepage at:
10
11 http://github.com/grempe/amazon-ec2/tree/master
12
13 or the EC2 website at http://aws.amazon.com/ec2
d84bb93d » grempe 2007-06-03 r1567@macbook-pro: glenn ... 14
15
16 == Installation
17
18 This gem follows the standard conventions for installation on any system with Ruby and RubyGems installed. If you have worked with gems before this will look very familiar.
19
dc68752c » grempe 2008-04-25 Updated README 20 === Get an AWS account
d84bb93d » grempe 2007-06-03 r1567@macbook-pro: glenn ... 21
22 Before you can make use of this gem you will need an Amazon Web Services developer account which you can sign up for at https://aws-portal.amazon.com/gp/aws/developer/registration/index.html. This account must also be specifically enabled for Amazon EC2 usage. AWS will provide you with an 'AWS Access Key ID' and a 'Secret Access Key' which will allow you to authenticate any API calls you make and ensure correct billing to you for usage of the service. Take note of these (and keep them secret!).
23
dc68752c » grempe 2008-04-25 Updated README 24 === Install required gem pre-requisites
25
16322a0b » grempe 2009-06-11 Damn smart quotes. Screw u... 26 The following gems should be installed automatically as part of your install of amazon-ec2. Most of them are needed for testing build dependencies but they should be painless to install even if you don't plan on running the tests or building this gem manually on your own.
dc68752c » grempe 2008-04-25 Updated README 27
28 XmlSimple (required)
29 Mocha (optional for testing)
30 Rcov (optional for testing)
31 Test-Spec (optional for testing)
32
33 === Install the amazon-ec2 gem
d84bb93d » grempe 2007-06-03 r1567@macbook-pro: glenn ... 34
64cf916a » grempe 2008-04-25 Update the readme to reflec... 35 # ONE TIME ONLY
36 # Execute this on each machine where you install gems to add GitHub as a gem source
37 # Do this only once or you'll end up with multiple entries in 'gem sources'
38 gem sources -a http://gems.github.com/
d84bb93d » grempe 2007-06-03 r1567@macbook-pro: glenn ... 39
38637ea0 » grempe 2008-08-20 Better install instructions... 40 # Install the stable release gem from RubyForge
41 sudo gem install amazon-ec2
d84bb93d » grempe 2007-06-03 r1567@macbook-pro: glenn ... 42
38637ea0 » grempe 2008-08-20 Better install instructions... 43 # OR
44
45 # Install the gem from the GitHub bleeding edge as a normal runtime install
64cf916a » grempe 2008-04-25 Update the readme to reflec... 46 sudo gem install grempe-amazon-ec2
d84bb93d » grempe 2007-06-03 r1567@macbook-pro: glenn ... 47
38637ea0 » grempe 2008-08-20 Better install instructions... 48 # OR
49
50 # Install the gem from the GitHub bleeding edge with all developer dependencies
51 sudo gem install grempe-amazon-ec2 --development
52
fd2e66af » grempe 2009-06-11 Don't require rubygems anym... 53 # OR
54
55 # Install using Rip (hellorip.com) instead of RubyGems
56 rip install git://github.com/grempe/amazon-ec2.git
57
dc68752c » grempe 2008-04-25 Updated README 58 == Using amazon-ec2
d84bb93d » grempe 2007-06-03 r1567@macbook-pro: glenn ... 59
0e0470ec » grempe 2007-06-19 r1285@mac-mini: glenn | 2... 60 The library exposes one main interface class EC2::Base. It is through an instance of this class that you will perform all the operations for using the EC2 service including query string header signing.
d84bb93d » grempe 2007-06-03 r1567@macbook-pro: glenn ... 61
0e0470ec » grempe 2007-06-19 r1285@mac-mini: glenn | 2... 62 The public methods on EC2::Base closely mirror the EC2 Query API, and as such the Query API Reference in the EC2 Developer Guide ( http://developer.amazonwebservices.com/connect/kbcategory.jspa?categoryID=84 ) will prove helpful.
d84bb93d » grempe 2007-06-03 r1567@macbook-pro: glenn ... 63
64
dc68752c » grempe 2008-04-25 Updated README 65 === Setting up
66
16322a0b » grempe 2009-06-11 Damn smart quotes. Screw u... 67 The 'ec2sh' and 'ec2-gem-example.rb' scripts which will be introduced to you shortly expect your AWS EC2 credentials to be stored as shell environment variables which are accessible to those scripts. This makes them convenient to use whenever you need to do a quick query to see what images you have available to you, what's running now, or to start or stop an instance on EC2. You'll find 'ec2sh' to be a very handy tool. I'll describe only the OS X route for setting up (of course the setup steps will vary depending on your particular system and preferred shell). If you don't want to do it this way, feel free to copy these scripts from the gem dir to any location where you can run them from and modify them directly to include your credentials.
dc68752c » grempe 2008-04-25 Updated README 68
69 Edit the file ~/.bash_login and add the following to the existing contents:
70
71 export RUBYOPT="rubygems"
72
fd2e66af » grempe 2009-06-11 Don't require rubygems anym... 73 Why 'export RUBYOPT'? Because we leave loading libs up to you... See : http://gist.github.com/54177
74
dc68752c » grempe 2008-04-25 Updated README 75 # For amazon-ec2 and amazon s3 ruby gems
76 export AMAZON_ACCESS_KEY_ID="YOUR_ACCESS_KEY_ID"
77 export AMAZON_SECRET_ACCESS_KEY="YOUR_SECRET_ACCESS_KEY_ID"
78
9ade9517 » kunley 2009-01-30 Use EC2_URL env variable to... 79 If you are using EC2 in the EU region, make sure you also set:
80
81 export EC2_URL="https://eu-west-1.ec2.amazonaws.com"
82
ce362be6 » grempe 2009-01-31 Typo 83 (which you have already if you configured standard EC2 command line tools to work with this region).
9ade9517 » kunley 2009-01-30 Use EC2_URL env variable to... 84
16322a0b » grempe 2009-06-11 Damn smart quotes. Screw u... 85 Once you save the file you should close and re-open your terminal so the new variables are made available. You'll need to do this close/re-open step for each terminal window you have open (or issue the 'source ~/.bash_login' command in each). Make sure that this file is only readable by your user so you don't inadvertently expose your credentials to other users on your system.
dc68752c » grempe 2008-04-25 Updated README 86
16322a0b » grempe 2009-06-11 Damn smart quotes. Screw u... 87 You can verify that this setup is complete by running the 'set' in a command window and seeing that your credentials are in the list of shell variables.
dc68752c » grempe 2008-04-25 Updated README 88
89 === The basics
90
91 The library exposes one main interface module
92
93 EC2::Base
94
95 This method requires arguments which include your AWS credentials and it will return an object that you can use to make method calls directly against EC2. All the operations for using the EC2 service, including query string header signing, are handled automatically for you. The connection string will look something like this:
96
97 @ec2 = EC2::Base.new(:access_key_id => ACCESS_KEY_ID, :secret_access_key => SECRET_ACCESS_KEY)
98
99 I've tried to keep the public methods on 'amazon-ec2' as close as possible to the AWS EC2 Query API. This similarity allows you to reference the Query API Reference in the EC2 Developer Guide and be able to get started right away. In most cases the methods names only differ in how they are presented. e.g. 'DescribeImages' becomes '#describe_images()' in Ruby. Feel free to browse the full RDoc documentation for all classes and methods of 'amazon-ec2' if you want more details.
100
101
102 === Examples
103
104 The best way to become familiar with 'amazon-ec2' is to take it for a test drive. We have provided a few simple ways to get you started. There is also some sample code below that should help out in using 'amazon-ec2' with a plain Ruby script, or as part of a Ruby on Rails application.
19ee86d8 » grempe 2007-07-03 r1346@mac-mini: glenn | 2... 105
dc68752c » grempe 2008-04-25 Updated README 106 ==== Using the 'ec2-gem-example.rb' sample test script
64cf916a » grempe 2008-04-25 Update the readme to reflec... 107
dc68752c » grempe 2008-04-25 Updated README 108 An example Ruby script which exercises the library a bit more is installed for you to check out when you install this gem. You can run this script to verify that everything is setup and working correctly in your environment. Consult the file which is installed at :
64cf916a » grempe 2008-04-25 Update the readme to reflec... 109
dc68752c » grempe 2008-04-25 Updated README 110 [your amazon-ec2 gem dir]/examples/ec2-example.rb
64cf916a » grempe 2008-04-25 Update the readme to reflec... 111
16322a0b » grempe 2009-06-11 Damn smart quotes. Screw u... 112 Since we also package this sample file in the gem's bin/ dir you should also be able to run it from anywhere on your shell path (once you have set your environment variables as described above).
64cf916a » grempe 2008-04-25 Update the readme to reflec... 113
dc68752c » grempe 2008-04-25 Updated README 114 ==== Using the 'ec2sh' command shell
115
16322a0b » grempe 2009-06-11 Damn smart quotes. Screw u... 116 The 'ec2sh' command shell is actually a standard 'irb' Ruby shell, with the main difference being we read your AWS credentials from your environment and pre-configure a connection string for you. This lets you run any EC2 command very simply. This has proven to be a valuable tool during the development of this gem and you should try it out. Since we install this tool in your system path as part of the installation of this gem, you should be able to simply run 'ec2sh' from any terminal command prompt on your local system. You'll see some basic instructions for use, and a few examples when you start 'ec2sh'. Go ahead and try it out now. We'll wait...
dc68752c » grempe 2008-04-25 Updated README 117
16322a0b » grempe 2009-06-11 Damn smart quotes. Screw u... 118 If you're not in front of a terminal shell now (perhaps you're browsing this site on your iPhone) this is what you would see:
dc68752c » grempe 2008-04-25 Updated README 119
120
121 hostname:/tmp/rails/amazon_test glenn$ ec2sh
122
123 'ec2sh' usage :
124 This is an interactive 'irb' command shell that allows you to use all
125 commands available to the amazon-ec2 gem. You'll find this to be a
126 great tool to help you debug issues and practice running commands
127 against the live EC2 servers prior to putting them in your code.
128
129 The EC2 connection is wired to the class instance '@ec2'. Make method calls
130 on this to execute commands on EC2. Adding a #to_s
131 at the end of any command should give you a full String representation of the
132 response. The #xml data is available for each response
133 which allows you to view the full and complete XML response returned by
134 EC2 without any parsing applied. This is useful for viewing the
135 hierarchy of an entire response in a friendly way (if XML is friendly
136 to you!). Understanding the hierarchy of the XML response is critical
137 to making effective use of this library.
138
139 Examples to try:
140
141 returns : all ec2 public methods
142 >> @ec2.methods.sort
143
144 returns : a string representation of ALL images
145 >> @ec2.describe_images.to_s
146
147 returns : an Array of EC2::Response objects, each an EC2 image and its data
148 >> @ec2.describe_images.imagesSet.item
fd2e66af » grempe 2009-06-11 Don't require rubygems anym... 149 >> @ec2.describe_images.imagesSet.item[0] (a hash representing a single item in that array)
150 >> @ec2.describe_images.imagesSet.item[0].to_s (a String representation of that item)
dc68752c » grempe 2008-04-25 Updated README 151
152 >> @ec2.describe_images.imagesSet.item[0].to_s
153 => "#<EC2::Response:0x100A465B4 imageId=\"ami-018e6b68\" imageLocation=\"rbuilder-online/phonehome-1.5.6-x86_10132.img.manifest.xml\" imageOwnerId=\"099034111737\" imageState=\"available\" isPublic=\"true\" parent=#<EC2::Response:0x100A469A6 ...>>"
154
155
156 === Ruby script usage example:
157
16322a0b » grempe 2009-06-11 Damn smart quotes. Screw u... 158 Try out the following bit of code. This should walk through each image returned by a call to #describe_images and print out its key data. Note in the example below that you cannot walk through the results of the #describe_images call with the '.each' iterator (You'll get errors if you try). You need to instead walk through the Array of items which are in the 'imagesSet' embedded in the response. This reflects exactly the XML hierarchy of data returned from EC2 which we parse to Ruby OpenStruct objects (EC2::Response).
dc68752c » grempe 2008-04-25 Updated README 159
160 #!/usr/bin/env ruby
38637ea0 » grempe 2008-08-20 Better install instructions... 161
dc68752c » grempe 2008-04-25 Updated README 162 require 'rubygems'
8589b082 » grempe 2009-06-06 Fixed require statement in ... 163 require 'EC2'
38637ea0 » grempe 2008-08-20 Better install instructions... 164
dc68752c » grempe 2008-04-25 Updated README 165 ACCESS_KEY_ID = '--YOUR AWS ACCESS KEY ID--'
166 SECRET_ACCESS_KEY = '--YOUR AWS SECRET ACCESS KEY--'
38637ea0 » grempe 2008-08-20 Better install instructions... 167
dc68752c » grempe 2008-04-25 Updated README 168 ec2 = EC2::Base.new(:access_key_id => ACCESS_KEY_ID, :secret_access_key => SECRET_ACCESS_KEY)
38637ea0 » grempe 2008-08-20 Better install instructions... 169
dc68752c » grempe 2008-04-25 Updated README 170 puts "----- listing images owned by 'amazon' -----"
171 ec2.describe_images(:owner_id => "amazon").imagesSet.item.each do |image|
172 # OpenStruct objects have members!
173 image.members.each do |member|
174 puts "#{member} => #{image[member]}"
175 end
176 end
d84bb93d » grempe 2007-06-03 r1567@macbook-pro: glenn ... 177
178 === Ruby on Rails usage example:
179
180 <b>config/environment.rb</b>
181
fd2e66af » grempe 2009-06-11 Don't require rubygems anym... 182 Rails::Initializer.run do |config|
183 ...
184 # INSERT THIS LINE IN YOUR CONFIG
185 config.gem "amazon-ec2", :lib => "EC2"
186 ...
187 end
188
d84bb93d » grempe 2007-06-03 r1567@macbook-pro: glenn ... 189
190 <b>app/controllers/my_controller.rb</b>
191
fd2e66af » grempe 2009-06-11 Don't require rubygems anym... 192 [some controller code ...]
64cf916a » grempe 2008-04-25 Update the readme to reflec... 193
fd2e66af » grempe 2009-06-11 Don't require rubygems anym... 194 ec2 = EC2::Base.new(:access_key_id => "YOUR_AWS_ACCESS_KEY_ID", :secret_access_key => "YOUR_AWS_SECRET_ACCESS_KEY")
64cf916a » grempe 2008-04-25 Update the readme to reflec... 195
fd2e66af » grempe 2009-06-11 Don't require rubygems anym... 196 # get ALL public images
197 @ec2_images = ec2.describe_images().imagesSet.item
64cf916a » grempe 2008-04-25 Update the readme to reflec... 198
fd2e66af » grempe 2009-06-11 Don't require rubygems anym... 199 # Get info on all public EC2 images created by the Amazon EC2 team.
200 @ec2_images_amazon = ec2.describe_images(:owner_id => "amazon").imagesSet.item
64cf916a » grempe 2008-04-25 Update the readme to reflec... 201
fd2e66af » grempe 2009-06-11 Don't require rubygems anym... 202 [some more controller code ...]
19ee86d8 » grempe 2007-07-03 r1346@mac-mini: glenn | 2... 203
64cf916a » grempe 2008-04-25 Update the readme to reflec... 204
d84bb93d » grempe 2007-06-03 r1567@macbook-pro: glenn ... 205 <b>app/views/my/index.rhtml</b>
206
fd2e66af » grempe 2009-06-11 Don't require rubygems anym... 207 <h1>EC2 Test#index</h1>
64cf916a » grempe 2008-04-25 Update the readme to reflec... 208
fd2e66af » grempe 2009-06-11 Don't require rubygems anym... 209 <h1>Sample 1 - debug() view</h1>
64cf916a » grempe 2008-04-25 Update the readme to reflec... 210
fd2e66af » grempe 2009-06-11 Don't require rubygems anym... 211 <%= debug(@ec2_images_amazon) %>
64cf916a » grempe 2008-04-25 Update the readme to reflec... 212
fd2e66af » grempe 2009-06-11 Don't require rubygems anym... 213 <h1>Sample 2 - Build a table</h1>
64cf916a » grempe 2008-04-25 Update the readme to reflec... 214
fd2e66af » grempe 2009-06-11 Don't require rubygems anym... 215 <table border='1'>
216 <tr>
217 <th>image.imageId</th>
218 <th>image.imageLocation</th>
219 <th>image.imageOwnerId</th>
220 <th>image.imageState</th>
221 <th>image.isPublic</th>
222 </tr>
64cf916a » grempe 2008-04-25 Update the readme to reflec... 223
fd2e66af » grempe 2009-06-11 Don't require rubygems anym... 224 <% for image in @ec2_images_amazon %>
225 <tr>
226 <td><%=h image.imageId %></td>
227 <td><%=h image.imageLocation %></td>
228 <td><%=h image.imageOwnerId %></td>
229 <td><%=h image.imageState %></td>
230 <td><%=h image.isPublic %></td>
231 </tr>
232 <% end %>
233 </table>
64cf916a » grempe 2008-04-25 Update the readme to reflec... 234
fd2e66af » grempe 2009-06-11 Don't require rubygems anym... 235 <h1>Sample 3 - Iterate</h1>
64cf916a » grempe 2008-04-25 Update the readme to reflec... 236
fd2e66af » grempe 2009-06-11 Don't require rubygems anym... 237 <% @ec2_images_amazon.each do |image| %>
238 <% image.each_pair do |key, value| %>
239 <% unless key == 'parent' %>
240 <%= "#{key} => #{value}" %><br />
241 <% end %>
242 <% end %>
243 <br />
244 <% end %>
64cf916a » grempe 2008-04-25 Update the readme to reflec... 245
d84bb93d » grempe 2007-06-03 r1567@macbook-pro: glenn ... 246
dc68752c » grempe 2008-04-25 Updated README 247 === Important notes regarding the structure of EC2::Response Objects
248
fd2e66af » grempe 2009-06-11 Don't require rubygems anym... 249 One of the key benefits of this new version of the library is that all responses from EC2 are bundled up in a real data structure and no longer require parsing of text. The hash returned is populated directly from the XML given to us by EC2 in response to any command we issue. This means that future changes to the API and what is returned by EC2 will be handled transparently by the gem. This is a huge benefit. What this means though, is that you may have to do a little homework on what actually gets returned by EC2 as XML. For example, when you make a #describe_images call in 'ec2sh' what AWS returns behind the scenes looks like:
38637ea0 » grempe 2008-08-20 Better install instructions... 250
dc68752c » grempe 2008-04-25 Updated README 251 <?xml version="1.0"?>
252 <DescribeImagesResponse xmlns="http://ec2.amazonaws.com/doc/2007-01-19/">
253 <imagesSet>
254 <item>
255 <imageId>ami-20b65349</imageId>
256 <imageLocation>ec2-public-images/fedora-core4-base.manifest.xml</imageLocation>
257 <imageState>available</imageState>
258 <imageOwnerId>amazon</imageOwnerId>
259 <isPublic>true</isPublic>
260 </item>
261 <item>
262 <imageId>ami-22b6534b</imageId>
263 <imageLocation>ec2-public-images/fedora-core4-mysql.manifest.xml</imageLocation>
264 <imageState>available</imageState>
265 <imageOwnerId>amazon</imageOwnerId>
266 <isPublic>true</isPublic>
267 </item>
268 <item>
269 <imageId>ami-23b6534a</imageId>
270 <imageLocation>ec2-public-images/fedora-core4-apache.manifest.xml</imageLocation>
271 <imageState>available</imageState>
272 <imageOwnerId>amazon</imageOwnerId>
273 <isPublic>true</isPublic>
274 </item>
275 <item>
276 <imageId>ami-25b6534c</imageId>
277 <imageLocation>ec2-public-images/fedora-core4-apache-mysql.manifest.xml</imageLocation>
278 <imageState>available</imageState>
279 <imageOwnerId>amazon</imageOwnerId>
280 <isPublic>true</isPublic>
281 </item>
282 <item>
283 <imageId>ami-26b6534f</imageId>
284 <imageLocation>ec2-public-images/developer-image.manifest.xml</imageLocation>
285 <imageState>available</imageState>
286 <imageOwnerId>amazon</imageOwnerId>
287 <isPublic>true</isPublic>
288 </item>
289 <item>
290 <imageId>ami-2bb65342</imageId>
291 <imageLocation>ec2-public-images/getting-started.manifest.xml</imageLocation>
292 <imageState>available</imageState>
293 <imageOwnerId>amazon</imageOwnerId>
294 <isPublic>true</isPublic>
295 </item>
296 </imagesSet>
297 </DescribeImagesResponse>
298
98fdd65a » grempe 2009-01-11 Removed references to the a... 299 You can see here the XML the structure that you will need to follow when constructing queries for information and parsing responses from EC2.
dc68752c » grempe 2008-04-25 Updated README 300
301 So, for example, if you wanted to get the image ID of the third image listed in the response above you would need to do:
302
303 >> puts @ec2.describe_images(:owner_id => 'amazon').imagesSet.item[2].imageId
304 ami-23b6534a
305
16322a0b » grempe 2009-06-11 Damn smart quotes. Screw u... 306 EC2 will typically return sets of things (imagesSet, reservationSet, etc.) which we map to ruby Arrays (.imagesSet.item in the example above). If you want to iterate over a response set you will need to iterate over this array. The Arrays will typically contain additional EC2::Response objects that represent each individual item. You'll find that you can use the 'ec2sh' to help you understand the structure more completely if you try issuing commands there as a way to practice seeing what will be returned and making sure you get exactly what you want.
dc68752c » grempe 2008-04-25 Updated README 307
308
d84bb93d » grempe 2007-06-03 r1567@macbook-pro: glenn ... 309 == Additional Resources
310
64cf916a » grempe 2008-04-25 Update the readme to reflec... 311 === Project Websites
d84bb93d » grempe 2007-06-03 r1567@macbook-pro: glenn ... 312
64cf916a » grempe 2008-04-25 Update the readme to reflec... 313 * Project Home : http://github.com/grempe/amazon-ec2/tree/master
314 * Discussion Group : http://groups.google.com/group/amazon-ec2
abd30941 » grempe 2009-06-06 Updated link for issue trac... 315 * Report Bugs / Request Features : http://github.com/grempe/amazon-ec2/issues
d84bb93d » grempe 2007-06-03 r1567@macbook-pro: glenn ... 316 * Amazon Web Services : http://aws.amazon.com
317
318 == Credits
319
64cf916a » grempe 2008-04-25 Update the readme to reflec... 320 The original code for this library was provided by Amazon Web Services, LLC as sample code. Thanks to them for providing all of us with something to get us started.
d84bb93d » grempe 2007-06-03 r1567@macbook-pro: glenn ... 321
322 == Contact
323
fd2e66af » grempe 2009-06-11 Don't require rubygems anym... 324 Comments, patches, Git pull requests and bug reports are welcome. Send an email to mailto:glenn@rempe.us or use the Google Groups forum for this project.
d84bb93d » grempe 2007-06-03 r1567@macbook-pro: glenn ... 325
326 Enjoy!
327
328 Glenn Rempe
fd2e66af » grempe 2009-06-11 Don't require rubygems anym... 329