Skip to content

Commit

Permalink
Move the attempts doctest out of the docs and added run doctest.
Browse files Browse the repository at this point in the history 
It's nice if docs are tested, but doctests shouldn't pollute docs.
  • Loading branch information
Jim Fulton committed Nov 7, 2016
1 parent 3681a34 commit c4cdfc9
Show file tree
Hide file tree
Showing 2 changed files with 117 additions and 112 deletions.
152 changes: 47 additions & 105 deletions docs/convenience.rst
View file
Original file line number Diff line number Diff line change
Expand Up @@ -50,9 +50,7 @@ Retries

Commits can fail for transient reasons, especially conflicts.
Applications will often retry transactions some number of times to
overcome transient failures. This typically looks something like:

.. doctest::
overcome transient failures. This typically looks something like::

for i in range(3):
try:
Expand All @@ -63,119 +61,63 @@ overcome transient failures. This typically looks something like:
else:
break

This is rather ugly.
This is rather ugly and easy to get wrong.

Transaction managers provide a helper for this case. To show this,
we'll use a contrived example:
Transaction managers provide two helpers for this case.

.. doctest::
Running and retrying functions as transactions
______________________________________________

>>> ntry = 0
>>> with transaction.manager:
... dm['ntry'] = 0

>>> import transaction.interfaces
>>> class Retry(transaction.interfaces.TransientError):
... pass

>>> for attempt in transaction.manager.attempts():
... with attempt as t:
... t.note('test')
... print("%s %s" % (dm['ntry'], ntry))
... ntry += 1
... dm['ntry'] = ntry
... if ntry % 3:
... raise Retry(ntry)
0 0
0 1
0 2

The raising of a subclass of TransientError is critical here. It's
what signals that the transaction should be retried. It is generally
up to the data manager to signal that a transaction should try again
by raising a subclass of TransientError (or TransientError itself, of
course).

You shouldn't make any assumptions about the object returned by the
iterator. (It isn't a transaction or transaction manager, as far as
you know. :) If you use the ``as`` keyword in the ``with`` statement,
a transaction object will be assigned to the variable named.

By default, it tries 3 times. We can tell it how many times to try:
The first helper runs a function as a transaction::

.. doctest::
def do_somthing():
"Do something"
... some something ...

>>> for attempt in transaction.manager.attempts(2):
... with attempt:
... ntry += 1
... if ntry % 3:
... raise Retry(ntry)
Traceback (most recent call last):
...
Retry: 5
transaction.manager.run(do_somthing)

It it doesn't succeed in that many times, the exception will be
propagated.
Of course you can run this as a decorator::

Of course, other errors are propagated directly:
@transaction.manager.run
def do_somthing():
"Do something"
... some something ...

.. doctest::
Some people find this easier to read, even though the result isn't a
decorated function, but rather the result of calling it in a
transaction.

>>> ntry = 0
>>> for attempt in transaction.manager.attempts():
... with attempt:
... ntry += 1
... if ntry % 3:
... raise ValueError(ntry)
Traceback (most recent call last):
...
ValueError: 3
The run method returns the successful result of calling the function.

We can use the default transaction manager:
The function name and docstring, if any, are added to the transaction
description.

.. doctest::
You can pass an integer number of times to try to the ``run`` method::

>>> ntry = 0
>>> for attempt in transaction.attempts():
... with attempt as t:
... t.note('test')
... print("%s %s" % (dm['ntry'], ntry))
... ntry += 1
... dm['ntry'] = ntry
... if ntry % 3:
... raise Retry(ntry)
3 0
3 1
3 2

Sometimes, a data manager doesn't raise exceptions directly, but
wraps other other systems that raise exceptions outside of it's
control. Data managers can provide a should_retry method that takes
an exception instance and returns True if the transaction should be
attempted again.
transaction.manager.run(do_somthing, 9)

.. doctest::
@transaction.manager.run(9)
def do_somthing():
"Do something"
... some something ...

>>> class DM(transaction.tests.savepointsample.SampleSavepointDataManager):
... def should_retry(self, e):
... if 'should retry' in str(e):
... return True

>>> ntry = 0
>>> dm2 = DM()
>>> with transaction.manager:
... dm2['ntry'] = 0
>>> for attempt in transaction.manager.attempts():
... with attempt:
... print("%s %s" % (dm['ntry'], ntry))
... ntry += 1
... dm['ntry'] = ntry
... dm2['ntry'] = ntry
... if ntry % 3:
... raise ValueError('we really should retry this')
3 0
3 1
3 2

>>> dm2['ntry']
3
The default number of times to try is 3.


Retrying code blocks using a attempt iterator
_____________________________________________

An older helper for running transactions uses an iterator of attempts::

for attempt in transaction.manager.attempts():
with attempt as t:
... some something ...


This runs the code block until it runs without a transient error or
until the number of attempts is exceeded. By default, it tries 3
times, but you can pass a number of attempts::

for attempt in transaction.manager.attempts(9):
with attempt as t:
... some something ...
77 changes: 70 additions & 7 deletions transaction/tests/test__manager.py
View file
Original file line number Diff line number Diff line change
Expand Up @@ -199,7 +199,7 @@ def abort(self):
with tm:
tm._txn = txn = _Test()
1/0
except ZeroDivisionError:
except ZeroDivisionError:
pass
self.assertFalse(txn._committed)
self.assertTrue(txn._aborted)
Expand Down Expand Up @@ -246,6 +246,69 @@ def test_attempts_stop_on_success(self):

self.assertEqual(i, 1)

def test_attempts_retries(self):
import transaction.interfaces
class Retry(transaction.interfaces.TransientError):
pass

tm = self._makeOne()
i = 0
for attempt in tm.attempts(4):
with attempt:
i += 1
if i < 4:
raise Retry

self.assertEqual(i, 4)

def test_attempts_retries_but_gives_up(self):
import transaction.interfaces
class Retry(transaction.interfaces.TransientError):
pass

tm = self._makeOne()
i = 0

with self.assertRaises(Retry):
for attempt in tm.attempts(4):
with attempt:
i += 1
raise Retry

self.assertEqual(i, 4)

def test_attempts_propigates_errors(self):
tm = self._makeOne()
with self.assertRaises(ValueError):
for attempt in tm.attempts(4):
with attempt:
raise ValueError

def test_attempts_defer_to_dm(self):
import transaction.tests.savepointsample

class DM(transaction.tests.savepointsample.SampleSavepointDataManager):
def should_retry(self, e):
if 'should retry' in str(e):
return True

ntry = 0
dm = transaction.tests.savepointsample.SampleSavepointDataManager()
dm2 = DM()
with transaction.manager:
dm2['ntry'] = 0

for attempt in transaction.manager.attempts():
with attempt:
ntry += 1
dm['ntry'] = ntry
dm2['ntry'] = ntry
if ntry % 3:
raise ValueError('we really should retry this')

self.assertEqual(ntry, 3)


def test_attempts_w_default_count(self):
from transaction._manager import Attempt
tm = self._makeOne()
Expand Down Expand Up @@ -501,13 +564,13 @@ def test___exit__no_exc_nonretryable_commit_exception(self):
self.assertTrue(manager.aborted)

def test___exit__no_exc_abort_exception_after_nonretryable_commit_exc(self):
manager = DummyManager(raise_on_abort=ValueError,
manager = DummyManager(raise_on_abort=ValueError,
raise_on_commit=KeyError)
inst = self._makeOne(manager)
self.assertRaises(ValueError, inst.__exit__, None, None, None)
self.assertTrue(manager.committed)
self.assertTrue(manager.aborted)

def test___exit__no_exc_retryable_commit_exception(self):
from transaction.interfaces import TransientError
manager = DummyManager(raise_on_commit=TransientError)
Expand All @@ -532,29 +595,29 @@ def test___exit__with_exception_value_nonretryable(self):
self.assertRaises(KeyError, inst.__exit__, KeyError, KeyError(), None)
self.assertFalse(manager.committed)
self.assertTrue(manager.aborted)


class DummyManager(object):
entered = False
committed = False
aborted = False

def __init__(self, raise_on_commit=None, raise_on_abort=None):
self.raise_on_commit = raise_on_commit
self.raise_on_abort = raise_on_abort

def _retryable(self, t, v):
from transaction._manager import TransientError
return issubclass(t, TransientError)

def __enter__(self):
self.entered = True

def abort(self):
self.aborted = True
if self.raise_on_abort:
raise self.raise_on_abort

def commit(self):
self.committed = True
if self.raise_on_commit:
Expand Down

0 comments on commit c4cdfc9

Please sign in to comment.