Skip to content

HTTPS clone URL

Subversion checkout URL

You can clone with HTTPS or Subversion.

Download ZIP
Newer
Older
100644 312 lines (212 sloc) 10.155 kb
54912c2 Set version to 0.1.0
Ask Solem authored
1 #############################################
65edd80 @ask Change tagline from AMQP messaging framework to just 'Messaging Frame…
authored
2 kombu - Messaging Framework for Python
54912c2 Set version to 0.1.0
Ask Solem authored
3 #############################################
da3e754 Initial import
Ask Solem authored
4
bc4813b @ask Bumps version to 2.1.7
authored
5 :Version: 2.1.7
da3e754 Initial import
Ask Solem authored
6
35f8d57 Some marketing in README
Ask Solem authored
7 Synopsis
8 ========
d83d22f Remove example using nonexisting Consumer.get in README
Ask Solem authored
9
6b06d13 Messaging -queue framework
Ask Solem authored
10 `Kombu` is an `AMQP`_ messaging framework for Python.
35f8d57 Some marketing in README
Ask Solem authored
11
12 AMQP is the Advanced Message Queuing Protocol, an open standard protocol
13 for message orientation, queuing, routing, reliability and security.
d83d22f Remove example using nonexisting Consumer.get in README
Ask Solem authored
14
8f03c8f README.rst now passes rst2html --strict
Ask Solem authored
15 One of the most popular implementations of AMQP is `RabbitMQ`_.
16
2ea7beb Moved serialization section from README to it's own section in the us…
Ask Solem authored
17 The aim of `Kombu` is to make messaging in Python as easy as possible by
76300d0 Improved the description some more
Ask Solem authored
18 providing an idiomatic high-level interface for the AMQP protocol, and also
19 provide proven and tested solutions to common messaging problems.
35f8d57 Some marketing in README
Ask Solem authored
20
21 Features
22 ========
23
24 * Allows application authors to support several message server
25 solutions by using pluggable transports.
26
180e055 Fixed rst links in README.rst
Ask Solem authored
27 * AMQP transports for both the `amqplib`_ (sync) and
28 `pika`_ (sync + async) clients.
35f8d57 Some marketing in README
Ask Solem authored
29
30 * Virtual transports makes it really easy to add support for non-AMQP
180e055 Fixed rst links in README.rst
Ask Solem authored
31 transports. There is already built-in support for `Redis`_,
2528581 @ask Fixes rst issues in README
authored
32 `Beanstalk`_, `Amazon SQS`_, `CouchDB`_, and `MongoDB`_.
35f8d57 Some marketing in README
Ask Solem authored
33
34 * SQLAlchemy and Django ORM transports exists as plug-ins (
35 `kombu-sqlalchemy`_ and `django-kombu`_).
36
37 * In-memory transport for unit testing.
38
39 * Supports automatic encoding, serialization and compression of message
40 payloads.
41
42 * Consistent exception handling across transports.
43
44 * The ability to ensure that an operation is performed by gracefully
e9bfde7 @ask Spell-check
authored
45 handling connection and channel errors.
d83d22f Remove example using nonexisting Consumer.get in README
Ask Solem authored
46
35f8d57 Some marketing in README
Ask Solem authored
47 * Several annoyances with `amqplib`_ has been fixed, like supporting
48 timeouts and the ability to wait for events on more than one channel.
d83d22f Remove example using nonexisting Consumer.get in README
Ask Solem authored
49
35f8d57 Some marketing in README
Ask Solem authored
50 * Projects already using `carrot`_ can easily be ported by using
51 a compatibility layer.
d83d22f Remove example using nonexisting Consumer.get in README
Ask Solem authored
52
53
8f03c8f README.rst now passes rst2html --strict
Ask Solem authored
54 For an introduction to AMQP you should read the article `Rabbits and warrens`_,
55 and the `Wikipedia article about AMQP`_.
56
57
2cb7c11 @ask Added transport comparison table
authored
58
d83d22f Remove example using nonexisting Consumer.get in README
Ask Solem authored
59 .. _`RabbitMQ`: http://www.rabbitmq.com/
60 .. _`AMQP`: http://amqp.org
bb3d96c Some README changes
Ask Solem authored
61 .. _`Redis`: http://code.google.com/p/redis/
a5816c4 @ask Updates README with SQS transport
authored
62 .. _`Amazon SQS`: http://aws.amazon.com/sqs/
180e055 Fixed rst links in README.rst
Ask Solem authored
63 .. _`MongoDB`: http://www.mongodb.org/
64 .. _`CouchDB`: http://couchdb.apache.org/
65 .. _`Beanstalk`: http://kr.github.com/beanstalkd/
d83d22f Remove example using nonexisting Consumer.get in README
Ask Solem authored
66 .. _`Rabbits and warrens`: http://blogs.digitar.com/jjww/2009/01/rabbits-and-warrens/
ce302f2 Fixed amqplib link in README.rst
Ask Solem authored
67 .. _`amqplib`: http://barryp.org/software/py-amqplib/
9748a46 @jplock pika site URL changed from tonyg/pika to pika/pika
jplock authored
68 .. _`pika`: http://github.com/pika/pika
d83d22f Remove example using nonexisting Consumer.get in README
Ask Solem authored
69 .. _`Wikipedia article about AMQP`: http://en.wikipedia.org/wiki/AMQP
bb3d96c Some README changes
Ask Solem authored
70 .. _`kombu-sqlalchemy`: http://github.com/ask/kombu-sqlalchemy/
71 .. _`django-kombu`: http://github.com/ask/django-kombu/
180e055 Fixed rst links in README.rst
Ask Solem authored
72 .. _`carrot`: http://pypi.python.org/pypi/carrot/
fe50742 Added API to readme
Ask Solem authored
73
2cb7c11 @ask Added transport comparison table
authored
74
75 Transport Comparison
76 ====================
77
c3003e3 @priteau Also update the transport comparison table in the README
priteau authored
78 +---------------+----------+------------+------------+---------------+
79 | **Client** | **Type** | **Direct** | **Topic** | **Fanout** |
80 +---------------+----------+------------+------------+---------------+
81 | *amqplib* | Native | Yes | Yes | Yes |
82 +---------------+----------+------------+------------+---------------+
83 | *pika* | Native | Yes | Yes | Yes |
84 +---------------+----------+------------+------------+---------------+
85 | *redis* | Virtual | Yes | Yes [#f1]_ | Yes (PUB/SUB) |
86 +---------------+----------+------------+------------+---------------+
87 | *beanstalk* | Virtual | Yes | Yes [#f1]_ | No |
88 +---------------+----------+------------+------------+---------------+
89 | *SQS* | Virtual | Yes | Yes [#f1]_ | Yes [#f2]_ |
90 +---------------+----------+------------+------------+---------------+
627caf5 @ask Update transport comparison table with fanout support for MongoDB
authored
91 | *mongodb* | Virtual | Yes | Yes [#f1]_ | Yes |
c3003e3 @priteau Also update the transport comparison table in the README
priteau authored
92 +---------------+----------+------------+------------+---------------+
93 | *couchdb* | Virtual | Yes | Yes [#f1]_ | No |
94 +---------------+----------+------------+------------+---------------+
95 | *in-memory* | Virtual | Yes | Yes [#f1]_ | No |
96 +---------------+----------+------------+------------+---------------+
97
98
99 .. [#f1] Declarations only kept in memory, so exchanges/queues
100 must be declared by all clients that needs them.
101
102 .. [#f2] Fanout supported via storing routing tables in SimpleDB.
75d781e @ask Transport comparison was wrong, fanout is not enabled by default for SQS
authored
103 Disabled by default, but can be enabled by using the
104 ``supports_fanout`` transport option.
2528581 @ask Fixes rst issues in README
authored
105
2cb7c11 @ask Added transport comparison table
authored
106
d83d22f Remove example using nonexisting Consumer.get in README
Ask Solem authored
107 Documentation
108 -------------
109
110 Kombu is using Sphinx, and the latest documentation is available at GitHub:
111
112 http://ask.github.com/kombu
113
114 Quick overview
115 --------------
116
d5dacea Remove Sphinx directives from README.rst
Ask Solem authored
117 ::
4f0cc4b Ported CouchDB transport from ghettoq
Ask Solem authored
118
204b245 @ask Adds Connection.Producer + Connection.Consumer
authored
119 from kombu import BrokerConnection, Exchange, Queue
fe50742 Added API to readme
Ask Solem authored
120
121 media_exchange = Exchange("media", "direct", durable=True)
60e44cc @heckj fixing docs re: key vs routing_key
heckj authored
122 video_queue = Queue("video", exchange=media_exchange, routing_key="video")
fe50742 Added API to readme
Ask Solem authored
123
fc9c760 @ZooKeeper Edited README.rst via GitHub
ZooKeeper authored
124 def process_media(body, message):
125 print body
126 message.ack()
127
204b245 @ask Adds Connection.Producer + Connection.Consumer
authored
128 # connections
129 with BrokerConnection("amqp://guest:guest@localhost//") as conn:
fe50742 Added API to readme
Ask Solem authored
130
3f18e9c @ask Quick overview should declare the queue before publishing to it
authored
131 # Declare the video queue so that the messages can be delivered.
132 # It is a best practice in Kombu to have both publishers and
bb308d1 @priteau Fix typo in the Quick overview example
priteau authored
133 # consumers declare the queue.
49f3be2 @priteau Fix undefined variable in Quick overview example
priteau authored
134 video_queue(conn.channel()).declare()
3f18e9c @ask Quick overview should declare the queue before publishing to it
authored
135
204b245 @ask Adds Connection.Producer + Connection.Consumer
authored
136 # produce
137 with conn.Producer(exchange=media_exchange,
fc9c760 @ZooKeeper Edited README.rst via GitHub
ZooKeeper authored
138 serializer="json", routing_key="video") as producer:
204b245 @ask Adds Connection.Producer + Connection.Consumer
authored
139 producer.publish({"name": "/tmp/lolcat1.avi", "size": 1301013})
fe50742 Added API to readme
Ask Solem authored
140
204b245 @ask Adds Connection.Producer + Connection.Consumer
authored
141 # consume
142 with conn.Consumer(video_queue, callbacks=[process_media]) as consumer:
143 # Process messages and handle events on all channels
144 while True:
fc9c760 @ZooKeeper Edited README.rst via GitHub
ZooKeeper authored
145 conn.drain_events()
fe50742 Added API to readme
Ask Solem authored
146
d83d22f Remove example using nonexisting Consumer.get in README
Ask Solem authored
147 # Consume from several queues on the same channel:
3c96d0a kombu.Binding has been renamed to kombu.Queue
Ask Solem authored
148 video_queue = Queue("video", exchange=media_exchange, key="video")
149 image_queue = Queue("image", exchange=media_exchange, key="image")
fe50742 Added API to readme
Ask Solem authored
150
204b245 @ask Adds Connection.Producer + Connection.Consumer
authored
151 with connection.Consumer([video_queue, image_queue],
152 callbacks=[process_media]) as consumer:
153 while True:
154 connection.drain_events()
155
156
e9bfde7 @ask Spell-check
authored
157 Or handle channels manually::
204b245 @ask Adds Connection.Producer + Connection.Consumer
authored
158
159 with connection.channel() as channel:
160 producer = Producer(channel, ...)
161 consumer = Producer(channel)
162
163
164 All objects can be used outside of with statements too,
165 just remember to close the objects after use::
166
167 from kombu import BrokerConnection, Consumer, Producer
168
169 connection = BrokerConnection()
170 # ...
171 connection.close()
172
173 consumer = Consumer(channel_or_connection, ...)
174 consumer.register_callback(my_callback)
d380f7c Instantiating a Consumer does not declare consumer. Use Consumer.cons…
Ask Solem authored
175 consumer.consume()
204b245 @ask Adds Connection.Producer + Connection.Consumer
authored
176 # ....
177 consumer.cancel()
178
d380f7c Instantiating a Consumer does not declare consumer. Use Consumer.cons…
Ask Solem authored
179
204b245 @ask Adds Connection.Producer + Connection.Consumer
authored
180 producer = Producer(channel_or_connection, ...)
181 # ....
182 producer.close()
fe50742 Added API to readme
Ask Solem authored
183
184
d83d22f Remove example using nonexisting Consumer.get in README
Ask Solem authored
185 `Exchange` and `Queue` are simply declarations that can be pickled
e9bfde7 @ask Spell-check
authored
186 and used in configuration files etc.
ddb17c7 Some simple refactoring
Ask Solem authored
187
2ea7beb Moved serialization section from README to it's own section in the us…
Ask Solem authored
188 They also support operations, but to do so they need to be bound
d83d22f Remove example using nonexisting Consumer.get in README
Ask Solem authored
189 to a channel:
190
d5dacea Remove Sphinx directives from README.rst
Ask Solem authored
191 ::
ddb17c7 Some simple refactoring
Ask Solem authored
192
193 >>> exchange = Exchange("tasks", "direct")
194
195 >>> connection = BrokerConnection()
196 >>> channel = connection.channel()
f33d57d Small typo
sebleier authored
197 >>> bound_exchange = exchange(channel)
ddb17c7 Some simple refactoring
Ask Solem authored
198 >>> bound_exchange.delete()
199
200 # the original exchange is not affected, and stays unbound.
201 >>> exchange.delete()
202 raise NotBoundError: Can't call delete on Exchange not bound to
203 a channel.
204
da3e754 Initial import
Ask Solem authored
205 Installation
206 ============
207
2ea7beb Moved serialization section from README to it's own section in the us…
Ask Solem authored
208 You can install `Kombu` either via the Python Package Index (PyPI)
da3e754 Initial import
Ask Solem authored
209 or from source.
210
fb1e7e0 Improved reference documentation
Ask Solem authored
211 To install using `pip`,::
da3e754 Initial import
Ask Solem authored
212
470a269 Update README
Ask Solem authored
213 $ pip install kombu
da3e754 Initial import
Ask Solem authored
214
fb1e7e0 Improved reference documentation
Ask Solem authored
215 To install using `easy_install`,::
da3e754 Initial import
Ask Solem authored
216
470a269 Update README
Ask Solem authored
217 $ easy_install kombu
da3e754 Initial import
Ask Solem authored
218
219 If you have downloaded a source tarball you can install it
220 by doing the following,::
221
222 $ python setup.py build
223 # python setup.py install # as root
224
225
226 Terminology
227 ===========
228
229 There are some concepts you should be familiar with before starting:
230
470a269 Update README
Ask Solem authored
231 * Producers
da3e754 Initial import
Ask Solem authored
232
470a269 Update README
Ask Solem authored
233 Producers sends messages to an exchange.
da3e754 Initial import
Ask Solem authored
234
235 * Exchanges
236
237 Messages are sent to exchanges. Exchanges are named and can be
238 configured to use one of several routing algorithms. The exchange
239 routes the messages to consumers by matching the routing key in the
240 message with the routing key the consumer provides when binding to
241 the exchange.
242
243 * Consumers
244
245 Consumers declares a queue, binds it to a exchange and receives
246 messages from it.
247
248 * Queues
249
250 Queues receive messages sent to exchanges. The queues are declared
251 by consumers.
252
253 * Routing keys
254
255 Every message has a routing key. The interpretation of the routing
256 key depends on the exchange type. There are four default exchange
257 types defined by the AMQP standard, and vendors can define custom
258 types (so see your vendors manual for details).
259
260 These are the default exchange types defined by AMQP/0.8:
261
262 * Direct exchange
263
264 Matches if the routing key property of the message and
fb1e7e0 Improved reference documentation
Ask Solem authored
265 the `routing_key` attribute of the consumer are identical.
da3e754 Initial import
Ask Solem authored
266
267 * Fan-out exchange
268
269 Always matches, even if the binding does not have a routing
270 key.
271
272 * Topic exchange
273
274 Matches the routing key property of the message by a primitive
275 pattern matching scheme. The message routing key then consists
fb1e7e0 Improved reference documentation
Ask Solem authored
276 of words separated by dots (`"."`, like domain names), and
277 two special characters are available; star (`"*"`) and hash
278 (`"#"`). The star matches any word, and the hash matches
279 zero or more words. For example `"*.stock.#"` matches the
280 routing keys `"usd.stock"` and `"eur.stock.db"` but not
281 `"stock.nasdaq"`.
da3e754 Initial import
Ask Solem authored
282
283 Getting Help
284 ============
285
286 Mailing list
287 ------------
288
289 Join the `carrot-users`_ mailing list.
290
291 .. _`carrot-users`: http://groups.google.com/group/carrot-users/
292
293 Bug tracker
294 ===========
295
296 If you have any suggestions, bug reports or annoyances please report them
470a269 Update README
Ask Solem authored
297 to our issue tracker at http://github.com/ask/kombu/issues/
da3e754 Initial import
Ask Solem authored
298
299 Contributing
300 ============
301
2ea7beb Moved serialization section from README to it's own section in the us…
Ask Solem authored
302 Development of `Kombu` happens at Github: http://github.com/ask/kombu
da3e754 Initial import
Ask Solem authored
303
304 You are highly encouraged to participate in the development. If you don't
305 like Github (for some reason) you're welcome to send regular patches.
306
307 License
308 =======
309
d5dacea Remove Sphinx directives from README.rst
Ask Solem authored
310 This software is licensed under the `New BSD License`. See the `LICENSE`
da3e754 Initial import
Ask Solem authored
311 file in the top distribution directory for the full license text.
Something went wrong with that request. Please try again.