Join GitHub today
GitHub is home to over 28 million developers working together to host and review code, manage projects, and build software together.Sign up
A small bit of code to make the Boto library for Amazon's AWS services work in an asynchronous (and extremely hacky) manner with Tornado. For a probably better approach see: http://blog.joshhaas.com/2011/06/marrying-boto-to-tornado-greenlets-bring-them-together/ http://almostobsolete.net
Fetching latest commit…
Cannot retrieve the latest commit at this time.
|Failed to load latest commit information.|
AsyncBoto Thomas Parslow 2011 firstname.lastname@example.org http://almostobsolete.net/ twitter: @almostobsolete Make Boto operate in an Asynchronous way using Tornado. I've only tested with SimpleDB so far (and then only a little bit) but you should be able to do the same with all the other types of connection in Boto. ****THIS REALLY ISN'T SUITABLE FOR PRODUCTION**** DON'T BE STUPID AND USE THIS FOR SOMETHING IMPORTANT. THANKS :) Please check out Josh Haas's version using Python Greenlets: http://blog.joshhaas.com/2011/06/marrying-boto-to-tornado-greenlets-bring-them-together/ How to use it ============= Create an asynchronous version of a Boto connection object my adding in the mixin, which needs to be supplied as the first base class so that it overrides the methods in the second. For example, to make an asynchronous version of the SimpleDB connection you would do this: import boto.sdb.connection import asyncboto class AsyncSDBConnection(asyncboto.AsyncConnectionMixin, boto.sdb.connection.SDBConnection): pass The connection will act just like a normal version of the connection except it now has a new method you can call. The async_call method takes a function containing the stuff you want to do and a callback. It returns immediately (ish), the result will be sent back to you later via the callback. The "do stuff" function can make calls on the connection and the objects it returns as normal. WARNING: The function you pass in will be called multiple times as part of the way this works, so don't do anything with side effects in their! In fact, you're best of just confining yourself to Boto calls within that function, save the rest for the callback. Here's an example of a call: sdb_conn = AsyncSDBConnection(aws_access_key_id, aws_access_key_secret) def callback(ret): print "create_domain returned:", ret sdb_conn.call_async(lambda : sdb_conn.create_domain("mytest"), callback=callback) Of course you'd need to start the Tornado IOLoop to actually see it run: tornado.ioloop.IOLoop.instance().start() Known Problems ============== Request retries won't work. In fact time the same HTTP request is made in a loop it won't work. Performance, a lot of the code in a Boto call will get executed multiple times. I have no idea how much of an effect this actually has on performance right now. That's it for now. This is really early (I only just wrote this) but I'm interested in what people think about the approach. Please do get in touch with any comments, suggestions or whatever. I'm on Twitter as @almostobsolete or my email is email@example.com