mamba

Distributed awesome, for Python.

Overview

Mamba is designed to take the tedium out of distributed computing via message queues. Using regular Python classes and the mamba decorator, method calls can be executed asynchronously using pluggable backends such as SQS, memcached, local execution, etc.

Getting Started

Mamba uses a decorator to intercept method calls and serialize the function, arguments and class context to a specified receiver for execution.

Here's an example, using SQS as a dispatcher (a properly configured boto is assumed):

from mamba import mamba
from mamba.dispatchers import SQSDispatcher

class Addition(object):
    def __init__(self, a, b):
        self.a = a
        self.b = b

    @mamba(SQSDispatcher, queue='arithmetic')
    def add(self):
        return self.a + self.b

Calling the add() method of an instance of the Addition class places a message in an SQS queue named 'arithmetic' (if no queue is specified, a default queue name of 'mamba' is used).

In order to execute the queued method, the mamba daemon must be running on the machine(s) on which the action is to take place:

/usr/local/bin/mamba -c /path/to/mamba.yml -d

This effectively polls SQS for messages, and deserializes and executes when there is work to be done. The module/class of the calling method must be on the load path in order for execution to be successful!

A working mamba config for the above example would be:

arithmetic:
  receiver: SQSReceiver
  returner: SQSReturner
  queues:
    - arithmetic
  sleep: 5

Accessing Return Values

If included in the mamba config, return values will be stored using the specified returner class. When mamba-decorated methods are initially called, the return value is a UUID that can be used to later retrieve the value returned by the called method.

>>>> from mamba.returners import SQSReturner

>>>> a = Addition(1,1)
>>>> uuid = a.add()
675a4669-1d61-4297-af45-1e738f6098cc

>>>> store = SQSReturner()
>>>> store.get(uuid)
2

Bypassing Mamba

If you want to just run your mamba-decorated methods locally, pass in the secret keyword argument as follows:

>>>> a = Addition(1,1)
>>>> result = a.add(__nomamba__=True)
2

Meta

• Code: git clone git://github.com/neilmock/mamba.git