Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Feature Request: support direct invocation with functools.partial #210

Open
mikepii opened this issue Nov 30, 2023 · 0 comments
Open

Feature Request: support direct invocation with functools.partial #210

mikepii opened this issue Nov 30, 2023 · 0 comments

Comments

@mikepii
Copy link

mikepii commented Nov 30, 2023

I would like to be be able to use the backoff library to decorate a Callable created using functools.partial so that I can:

  • specify some of backoff's parameters dynamically, such as max_time, and
  • use backoff in my own wrapper function (which accepts a Callable, for which functools.partial is preferred over a lambda in general for its repr() functionality).

Example of proposed backoff usage with functools.partial:

from functools import partial
from typing import Any, Callable

import backoff


def main() -> None:
    run_http_client_operation_with_retries(partial(foobar, baz=123))


def run_http_client_operation_with_retries(operation: Callable[[], Any], max_time: Float) -> Any:
    create_callable_with_retries = backoff.on_exception(
        lambda: backoff.constant(20), ApiRetryableError, max_time=max_time
    )
    operation_with_retries = create_callable_with_retries(operation)
    return operation_with_retries()


def foobar(baz):
    ...


class ApiRetryableError(Exception):
    pass

Currently, this is close to working, but it fails to generate the "backing off" log message (with backoff 2.2.1 and Python 3.10.6):

  File "/.../lib/python3.10/site-packages/backoff/_common.py", line 97, in _log_backoff
    log_args = [details['target'].__name__, details['wait']]
AttributeError: 'functools.partial' object has no attribute '__name__'

A Python lambda can be used as a workaround (run_http_client_operation_with_retries(lambda: foobar(baz=123))), but then the log message is unclear (Backing off <lambda>(...) ...). One advantage of functools.partial is that it provides a clearer function name when you do repr(functools.partial(...)).

Using backoff as a context manager was proposed in #198 and #18. That would solve the same problem of being able to specify backoff parameters dynamically. However, it's impossible to implement a context manager that re-runs the code block it manages (see #18 (comment)).

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

No branches or pull requests

1 participant