Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
38 changes: 38 additions & 0 deletions .github/workflows/docs.yml
Original file line number Diff line number Diff line change
@@ -0,0 +1,38 @@
name: Deploy Documentation

on:
push:
branches:
- main
pull_request:
branches:
- main

env:
FORCE_COLOR: 1
UV_NO_SYNC: 1

permissions:
contents: write

jobs:
deploy:
runs-on: ubuntu-latest
steps:
- uses: actions/checkout@08eba0b27e820071cde6df949e0beb9ba4906955 # v4.3.0

- uses: actions/setup-python@a26af69be951a213d495a4c3e4e4022e16d87065 # v5.6.0

- name: Install uv
uses: astral-sh/setup-uv@557e51de59eb14aaaba2ed9621916900a91d50c6 # v6.6.1

- name: Install dependencies
run: |
uv sync --group docs

- name: Build documentation
run: uv run mkdocs build

- name: Deploy to GitHub Pages
if: github.event_name == 'push' && github.ref == 'refs/heads/main'
run: uv run mkdocs gh-deploy --force
3 changes: 3 additions & 0 deletions .gitignore
Original file line number Diff line number Diff line change
Expand Up @@ -10,3 +10,6 @@ dist/
.vscode
.tox/
coverage.xml

# mkdocs
site/
1 change: 1 addition & 0 deletions CHANGELOG.md
Original file line number Diff line number Diff line change
Expand Up @@ -9,6 +9,7 @@
- Include tests in source distribution https://github.com/python-backoff/backoff/pull/13
- Added `logging.LoggerAdapter` to `_MaybeLogger` union https://github.com/python-backoff/backoff/pull/34 from @jcbertin
- Add `exception` to the typing-only `Details` dictionary for cases when `on_exception` is used https://github.com/python-backoff/backoff/pull/35
- Add GitHub Actions for CI, documentation, and publishing https://github.com/python-backoff/backoff/pull/39 from @tysoncung

### Packaging

Expand Down
120 changes: 120 additions & 0 deletions docs/advanced/combining-decorators.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,120 @@
# Combining Decorators

Stack multiple backoff decorators for complex retry logic.

## Basics

Decorators are applied from bottom to top (inside out):

```python
@backoff.on_predicate(backoff.fibo, lambda x: x is None) # Applied last
@backoff.on_exception(backoff.expo, HTTPError) # Applied second
@backoff.on_exception(backoff.expo, Timeout) # Applied first
def complex_operation():
pass
```

## Common Patterns

### Different Exceptions, Different Strategies

```python
@backoff.on_exception(
backoff.expo,
requests.exceptions.Timeout,
max_time=300 # Generous timeout for network issues
)
@backoff.on_exception(
backoff.expo,
requests.exceptions.HTTPError,
max_time=60, # Shorter timeout for HTTP errors
giveup=lambda e: 400 <= e.response.status_code < 500
)
def api_call(url):
response = requests.get(url)
response.raise_for_status()
return response.json()
```

### Exception + Predicate

```python
@backoff.on_predicate(
backoff.constant,
lambda result: result.get("status") == "pending",
interval=5,
max_time=600
)
@backoff.on_exception(
backoff.expo,
requests.exceptions.RequestException,
max_time=60
)
def poll_until_ready(job_id):
response = requests.get(f"/api/jobs/{job_id}")
response.raise_for_status()
return response.json()
```

## Execution Order

Inner decorators execute first:

```python
calls = []

def track_call(func_name):
def handler(details):
calls.append(func_name)
return handler

@backoff.on_exception(
backoff.constant,
ValueError,
on_backoff=track_call('outer'),
max_tries=2,
interval=0.01
)
@backoff.on_exception(
backoff.constant,
TypeError,
on_backoff=track_call('inner'),
max_tries=2,
interval=0.01
)
def failing_function(error_type):
raise error_type("Test")
```

- If `TypeError` raised: inner decorator retries
- If `ValueError` raised: outer decorator retries
- Both errors: inner handles TypeError, then outer handles ValueError

## Best Practices

### Specific Before General

```python
@backoff.on_exception(backoff.expo, Exception) # Catch-all
@backoff.on_exception(backoff.fibo, ConnectionError) # Specific
def network_operation():
pass
```

### Short Timeouts Inside, Long Outside

```python
@backoff.on_exception(
backoff.expo,
Exception,
max_time=600 # Overall 10-minute limit
)
@backoff.on_exception(
backoff.constant,
Timeout,
interval=1,
max_tries=3 # Quick retries for timeouts
)
def layered_retry():
pass
```
117 changes: 117 additions & 0 deletions docs/advanced/custom-strategies.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,117 @@
# Custom Wait Strategies

Create custom wait generators for specialized retry patterns.

## Wait Generator Interface

A wait generator is a function that yields wait times in seconds:

```python
def my_wait_gen():
"""Yields: 1, 2, 3, 4, 5, 5, 5, ..."""
for i in range(1, 6):
yield i
while True:
yield 5

@backoff.on_exception(my_wait_gen, Exception)
def my_function():
pass
```

## Parameters

Accept parameters to customize behavior:

```python
def linear_backoff(start=1, increment=1, max_value=None):
"""Linear backoff: start, start+increment, start+2*increment, ..."""
value = start
while True:
if max_value and value > max_value:
yield max_value
else:
yield value
value += increment

@backoff.on_exception(
linear_backoff,
Exception,
start=2,
increment=3,
max_value=30
)
def my_function():
pass
```

## Examples

### Polynomial Backoff

```python
def polynomial_backoff(base=2, exponent=2, max_value=None):
"""Polynomial: base^(tries^exponent)"""
n = 1
while True:
value = base ** (n ** exponent)
if max_value and value > max_value:
yield max_value
else:
yield value
n += 1

@backoff.on_exception(polynomial_backoff, Exception, base=2, exponent=1.5)
```

### Stepped Backoff

```python
def stepped_backoff(steps):
"""Different wait times for different ranges
steps = [(3, 1), (5, 5), (None, 10)] # 3 tries at 1s, next 5 at 5s, rest at 10s
"""
for max_tries, wait_time in steps:
if max_tries is None:
while True:
yield wait_time
else:
for _ in range(max_tries):
yield wait_time

@backoff.on_exception(
stepped_backoff,
Exception,
steps=[(3, 1), (3, 5), (None, 30)]
)
```

### Random Backoff

```python
import random

def random_backoff(min_wait=1, max_wait=60):
"""Random wait between min and max"""
while True:
yield random.uniform(min_wait, max_wait)

@backoff.on_exception(random_backoff, Exception, min_wait=1, max_wait=10)
```

### Time-of-Day Aware

```python
from datetime import datetime

def business_hours_backoff():
"""Shorter waits during business hours"""
while True:
hour = datetime.now().hour
if 9 <= hour < 17:
yield 5 # 5 seconds during business hours
else:
yield 60 # 1 minute otherwise

@backoff.on_exception(business_hours_backoff, Exception)
```
97 changes: 97 additions & 0 deletions docs/advanced/runtime-config.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,97 @@
# Runtime Configuration

Configure backoff behavior dynamically at runtime.

## Overview

Decorator parameters can accept callables that are evaluated at runtime, allowing dynamic configuration based on application state, environment variables, or configuration files.

## Basic Pattern

```python
class Config:
MAX_RETRIES = 5
MAX_TIME = 60

@backoff.on_exception(
backoff.expo,
Exception,
max_tries=lambda: Config.MAX_RETRIES,
max_time=lambda: Config.MAX_TIME
)
def configurable_function():
pass

# Change configuration at runtime
Config.MAX_RETRIES = 10
```

## Environment Variables

```python
import os

@backoff.on_exception(
backoff.expo,
Exception,
max_tries=lambda: int(os.getenv('RETRY_MAX_TRIES', '5')),
max_time=lambda: int(os.getenv('RETRY_MAX_TIME', '60'))
)
def env_configured():
pass
```

## Configuration Files

```python
import json

def load_config():
with open('config.json') as f:
return json.load(f)

@backoff.on_exception(
backoff.expo,
Exception,
max_tries=lambda: load_config()['retry']['max_tries'],
max_time=lambda: load_config()['retry']['max_time']
)
def file_configured():
pass
```

## Dynamic Wait Strategies

```python
def get_wait_gen():
if app.config.get('fast_retry'):
return backoff.constant
return backoff.expo

@backoff.on_exception(
lambda: get_wait_gen(),
Exception
)
def dynamic_wait():
pass
```

## Application State

```python
class RateLimiter:
def __init__(self):
self.rate_limited = False

def get_interval(self):
return 10 if self.rate_limited else 1

rate_limiter = RateLimiter()

@backoff.on_predicate(
backoff.constant,
interval=lambda: rate_limiter.get_interval()
)
def adaptive_poll():
return check_resource()
```
Loading
Loading