Permalink
Browse files

Code blocks ftmfw

  • Loading branch information...
1 parent 0d7615f commit a9147f1aacf3b06ce76efbda1389e984acd6ca6e @dcramer dcramer committed Nov 1, 2012
Showing with 96 additions and 78 deletions.
  1. +96 −78 README.rst
View
@@ -3,23 +3,25 @@ Nydus
Generic database utilities, including connection clustering and routing so you can scale like a pro.
-The following example creates a Redis connection cluster which will distribute reads and writes based on a simple modulus lookup of the hashed key::
+The following example creates a Redis connection cluster which will distribute reads and writes based on a simple modulus lookup of the hashed key:
- >>> from nydus.db import create_cluster
- >>>
- >>> redis = create_cluster({
- >>> 'backend': 'nydus.db.backends.redis.Redis',
- >>> 'router': 'nydus.db.routers.keyvalue.PartitionRouter',
- >>> 'hosts': {
- >>> 0: {'db': 0},
- >>> 1: {'db': 1},
- >>> 2: {'db': 2},
- >>> }
- >>> })
- >>>
- >>> res = redis.incr('foo')
- >>>
- >>> assert res == 1
+.. code:: python
+
+ from nydus.db import create_cluster
+
+ redis = create_cluster({
+ 'backend': 'nydus.db.backends.redis.Redis',
+ 'router': 'nydus.db.routers.keyvalue.PartitionRouter',
+ 'hosts': {
+ 0: {'db': 0},
+ 1: {'db': 1},
+ 2: {'db': 2},
+ }
+ })
+
+ res = redis.incr('foo')
+
+ assert res == 1
nydus.db.create_cluster
@@ -44,30 +46,34 @@ Distributed Queries
-------------------
In some cases you may want to execute a query on many nodes (in parallel). Nydus has built-in support for this when any routing function
-returns a cluster of nodes::
+returns a cluster of nodes:
+
+.. code:: python
- >>> from nydus.db import create_cluster
+ from nydus.db import create_cluster
>>>
- >>> # by not specifying a router all queries are executed on all hosts
- >>> redis = create_cluster({
- >>> 'backend': 'nydus.db.backends.redis.Redis',
- >>> 'hosts': {
- >>> 0: {'db': 0},
- >>> 1: {'db': 1},
- >>> 2: {'db': 2},
- >>> }
- >>> })
+ # by not specifying a router all queries are executed on all hosts
+ redis = create_cluster({
+ 'backend': 'nydus.db.backends.redis.Redis',
+ 'hosts': {
+ 0: {'db': 0},
+ 1: {'db': 1},
+ 2: {'db': 2},
+ }
+ })
>>>
- >>> # map the query over all connections returned by the default router
- >>> res = redis.incr('foo')
+ # map the query over all connections returned by the default router
+ res = redis.incr('foo')
>>>
- >>> assert type(res) == list
- >>> assert len(res) == 3
+ assert type(res) == list
+ assert len(res) == 3
-You can also map many queries (utilizing an internal queue) over connections (again, returned by the router)::
+You can also map many queries (utilizing an internal queue) over connections (again, returned by the router):
- >>> with redis.map() as conn:
- >>> results = [conn.incr(k) for k in keys]
+.. code:: python
+
+ with redis.map() as conn:
+ results = [conn.incr(k) for k in keys]
As of release 0.5.0, the map() function now supports pipelines, and the included Redis backend will pipeline commands
wherever possible.
@@ -79,14 +85,16 @@ Nydus was originally designed as a toolkit to expand on the usage of Redis at DI
a framework for building clusters that are not Redis, much of the support has gone into providing utilities
for routing and querying on Redis clusters.
-You can configure the Redis client for a connection by specifying it's full path::
+You can configure the Redis client for a connection by specifying it's full path:
+
+.. code:: python
- >>> redis = create_cluster({
- >>> 'backend': 'nydus.db.backends.redis.Redis',
- >>> 'hosts': {
- >>> 0: {'db': 0},
- >>> },
- >>> )}
+ redis = create_cluster({
+ 'backend': 'nydus.db.backends.redis.Redis',
+ 'hosts': {
+ 0: {'db': 0},
+ },
+ )}
The available host options are:
@@ -97,65 +105,75 @@ The available host options are:
* password
The Redis client also supports pipelines via the map command. This means that all commands will hit servers at most
-as of once::
+as of once:
- >>> with redis.map() as conn:
- >>> conn.set('a', 1)
- >>> conn.incr('a')
- >>> results = [conn.get('a'), conn.get('b')]
- >>> results['a'] == 2
- >>> results['b'] == None
+.. code:: python
+
+ with redis.map() as conn:
+ conn.set('a', 1)
+ conn.incr('a')
+ results = [conn.get('a'), conn.get('b')]
+ results['a'] == 2
+ results['b'] == None
Simple Partition Router
~~~~~~~~~~~~~~~~~~~~~~~
There are also several options for builtin routing. The easiest is a simple partition router, which is just a simple
-hash on the key::
+hash on the key:
+
+.. code:: python
- >>> redis = create_cluster({
- >>> 'backend': 'nydus.db.backends.redis.Redis',
- >>> 'router': 'nydus.db.routers.keyvalue.PartitionRouter',
- >>> 'hosts': {
- >>> 0: {'db': 0},
- >>> },
- >>> )}
+ redis = create_cluster({
+ 'backend': 'nydus.db.backends.redis.Redis',
+ 'router': 'nydus.db.routers.keyvalue.PartitionRouter',
+ 'hosts': {
+ 0: {'db': 0},
+ },
+ )}
Consistent Hashing Router
~~~~~~~~~~~~~~~~~~~~~~~~~
-An improvement upon hashing, Nydus provides a Ketama-based consistent hashing router::
+An improvement upon hashing, Nydus provides a Ketama-based consistent hashing router:
- >>> redis = create_cluster({
- >>> 'backend': 'nydus.db.backends.redis.Redis',
- >>> 'router': 'nydus.db.routers.keyvalue.ConsistentHashingRouter',
- >>> 'hosts': {
- >>> 0: {'db': 0},
- >>> },
- >>> )}
+.. code:: python
+
+ redis = create_cluster({
+ 'backend': 'nydus.db.backends.redis.Redis',
+ 'router': 'nydus.db.routers.keyvalue.ConsistentHashingRouter',
+ 'hosts': {
+ 0: {'db': 0},
+ },
+ )}
Round Robin Router
~~~~~~~~~~~~~~~~~~
-An additional option for distributing queries is the round robin router::
+An additional option for distributing queries is the round robin router:
+
+.. code:: python
- >>> redis = create_cluster({
- >>> 'backend': 'nydus.db.backends.redis.Redis',
- >>> 'router': 'nydus.db.routers.RoundRobinRouter',
- >>> 'hosts': {
- >>> 0: {'db': 0},
- >>> },
- >>> )}
+ redis = create_cluster({
+ 'backend': 'nydus.db.backends.redis.Redis',
+ 'router': 'nydus.db.routers.RoundRobinRouter',
+ 'hosts': {
+ 0: {'db': 0},
+ },
+ )}
Pycassa
-------
Basic connection management for pycassa (Cassandra) clusters is supported, but we use a non-standard syntax
-for creating clusters as routing behavior and per-connection options are not useful in this context::
+for creating clusters as routing behavior and per-connection options are not useful in this context:
+
+.. code:: python
- >>> pycassa = create_cluster({
- >>> 'backend': 'nydus.db.backends.pycassa.Pycassa',
- >>> 'hosts': ['localhost'],
- >>> 'keyspace': 'test',
- >>> })
+ pycassa = create_cluster({
+ 'backend': 'nydus.db.backends.pycassa.Pycassa',
+ 'hosts': ['localhost'],
+ 'keyspace': 'test',
+ })
.. note:: Pycassa handles routing of hosts internally, which means things like ``map`` have no affect.

0 comments on commit a9147f1

Please sign in to comment.