docs user/configuration/Client #13

bloop/client.py

@@ -18,15 +18,15 @@
 ]
 
 
-def default_backoff_func(attempts):
+def default_backoff_func(failed_attempts):
     """
     Exponential backoff helper.
 
     attempts is the number of calls so far that have failed
     """
-    if attempts == DEFAULT_MAX_ATTEMPTS:
-        raise RuntimeError("Failed after {} attempts".format(attempts))
-    return (DEFAULT_BACKOFF_COEFF * (2 ** attempts)) / 1000.0
+    if failed_attempts == DEFAULT_MAX_ATTEMPTS:
+        raise RuntimeError("Failed after {} attempts".format(failed_attempts))
+    return (DEFAULT_BACKOFF_COEFF * (2 ** failed_attempts)) / 1000.0
 
 
 def partition_batch_get_input(items):
@@ -78,9 +78,13 @@ class Client(object):
         http://docs.aws.amazon.com/amazondynamodb/latest/APIReference/API_Operations.html
     """
     def __init__(self, boto_client=None, backoff_func=None):
-        """
+        """Create a new bloop Client that wraps the boto3 clients.
+
+        boto_client is an optional instance of a boto3 client, either
+            created with `boto3.client("dynamodb")` or through a
+            boto3.session.Session.
         backoff_func is an optional function that takes an int
-            (attempts so far) that should either:
+            (failed attempts so far) that should either:
             - return the number of seconds to sleep
             - raise to stop
         """

docs/user/configuration.rst

@@ -4,11 +4,12 @@ Configuration
 Bloop tries to minimize the configuration required for the most common uses without restricting your ability to
 swap out large portions of the engine, type system, and low-level clients.
 
-Most of the knobs are tucked away so at first, they won't bother you.  There are only three configuration
-options for the Engine, and the built-in Types cover common uses.
+Most of the knobs are tucked away so they won't bother you at first.  There are only three configuration
+options for the Engine, and the built-in Types cover common uses.  As your models and types grow more complex, these
+features will be easy to integrate with your existing code.
 
-However, you can entirely drop the Engine and simply use the modeling structures.  Or replace the client and talk to
-a different backend.  You can skip the base Type and implement view-specific serialization functions.  Subclass or
+If necessary you can entirely drop the Engine and simply use the modeling structures.  Or replace the client and talk
+to a different backend.  You can skip the base Type and implement view-specific serialization functions.  Subclass or
 replace the Column class to add init kwargs like ``nullable=``.
 
 Engine Configuration
@@ -51,7 +52,7 @@ custom types:
 
 *Defaults to False*
 
-Used as the default value for the ``atomic=`` kwarg of ``Engine.save`` and ``Engine.delete``.
+Used as the default value for the ``atomic`` kwarg of ``Engine.save`` and ``Engine.delete``.
 
 When enabled, modifying operations on objects will construct a minimum condition (that is combined with any other
 condition provided) which expects the current value in DynamoDB to match the last values loaded from or saved to
@@ -64,7 +65,7 @@ For more information see the Atomic Operations section.
 
 *Defaults to False*
 
-Used as the default value for the ``consistent=`` kwarg of ``Engine.load`` and the default consistent setting for
+Used as the default value for the ``consistent`` kwarg of ``Engine.load`` and the default consistent setting for
 queries and scans.  Note that a GSI cannot be queried or scanned consistently.
 
 When enabled, retrieving objects through load, query, and scan will be performed with `Strongly Consistent Reads`_.
@@ -79,12 +80,11 @@ Strongly consistent reads will generally be slower than the default eventually c
 
 Used as the default value for the ``strict`` setting of queries and scans when selecting which columns to load.
 
-This is bloop's only deviation from DynamoDB's default behavior.
+This is bloop's only deviation from DynamoDB's default behavior.  When enabled, if a query or scan on an LSI tries to
+select a superset of the projected columns for that index, bloop will raise a ``ValueError``.
 
-When enabled, if a query or scan on an LSI tries to select a superset of the projected columns for that index, bloop
-will raise a ``ValueError``.
+When disabled, the same attempt to query a superset of projected columns **on an LSI only** will be allowed.
 
-When strict is False, the same attempt to query a superset of projected columns **on an LSI only** will be allowed.
 This is because DynamoDB will `automatically fetch`_ those attributes with an additional read against the table:
 
     If you query a local secondary index and request only attributes that are projected into that index, the operation
@@ -95,7 +95,7 @@ This is because DynamoDB will `automatically fetch`_ those attributes with an ad
     If you query a global secondary index, you can only request attributes that are projected into the index. Global
     secondary index queries cannot fetch attributes from the parent table.
 
-It is **strongly encouraged** to keep ``strict=True`` so that your consumed throughput is predictable, and won't change
+It is **strongly encouraged** to keep strict enabled so that your consumed throughput is predictable, and won't change
 when a dynamic query adds a non-projected column.  It also makes the behavior consistent with queries against GSIs.
 
 For more information see the Indexes and Projections section.
@@ -105,11 +105,48 @@ For more information see the Indexes and Projections section.
 Client
 ======
 
+.. code-block:: python
+
+    Client(boto_client=None, backoff_func=None)
+
+The bloop client ``bloop.Client`` is a convenience layer between the Engine and the boto3 client which handles
+batching, some pagination, and retries with backoffs.  Methods with the same name as their boto3 counterpart will often
+have the same request format but may unpack the outer wrappers of the response.  For instance,
+``Client.batch_get_items`` will return the value of the boto3 client's ``"Response"`` key, automatically chunks
+requests with more than 25 items, follows ``UnprocessedKeys`` from responses, and combines paginated results into a
+single dict.
+
 Boto Client
 -----------
 
-Backoffs
---------
+Sometimes you will need to configure a ``boto3.session.Session`` before creating the client, instead of the default
+path of using ``boto3.client("dynamodb")``.  For example, the :ref:`patterns-local` pattern demonstrates how to set up
+a connection to a DynamoDB Local instance.
+
+Retries
+-------
+
+By default, the bloop client will use an exponential backoff when a call raises a ``botocore.exceptions.ClientError``
+with an error code of either ``"InternalServerError"`` or ``"ProvisionedThroughputExceededException"``.  The default
+settings will try four times, with three backoffs: 100ms, 200ms, and 400ms.  If the call after 400ms of waiting fails,
+then the client will raise the last error it encountered.
+
+You may provide a custom backoff function that takes an integer representing the number of attempts made so far, and
+returns the number of milliseconds to wait before trying the call again.  When you want to stop retrying the call,
+perhaps hitting an upper limit of calls or time just raise a ``RuntimeError``.
+
+Here's a backoff function that waits 2 seconds between calls and allows 10 total attempts (9 retries):
+
+.. code-block:: python
+
+    def constant_backoff(failed_attempts):
+        if failed_attempts == 10:
+            raise RuntimeError("Failed after 10 attempts")
+        # Wait 2 seconds before retrying
+        return 2000
+
+Note that the ``failed_attempts`` parameter is the number of attempts so far: the first time it is called,
+``failed_attempts`` will be 1.
 
 Type Engine
 ===========

docs/user/patterns.rst

@@ -1,6 +1,8 @@
 Common Patterns
 ^^^^^^^^^^^^^^^
 
+.. _patterns-local:
+
 DynamoDB Local
 ==============