Add decorator_factory function to functools module

[uriyyo]

BPO	42455
Nosy	@rhettinger, @ncoghlan, @serhiy-storchaka, @isidentical, @uriyyo
PRs	bpo-42455: Add decorator_factory to functools module #23501

^{Note: these values reflect the state of the issue at the time it was migrated and might not reflect the current state.}

Show more details

GitHub fields:

assignee = None
closed_at = <Date 2021-01-05.11:24:00.539>
created_at = <Date 2020-11-24.20:34:37.829>
labels = ['type-feature', 'library', '3.10']
title = 'Add decorator_factory function to functools module'
updated_at = <Date 2021-01-05.11:34:43.026>
user = 'https://github.com/uriyyo'

bugs.python.org fields:

activity = <Date 2021-01-05.11:34:43.026>
actor = 'ncoghlan'
assignee = 'none'
closed = True
closed_date = <Date 2021-01-05.11:24:00.539>
closer = 'ncoghlan'
components = ['Library (Lib)']
creation = <Date 2020-11-24.20:34:37.829>
creator = 'uriyyo'
dependencies = []
files = []
hgrepos = []
issue_num = 42455
keywords = ['patch']
message_count = 12.0
messages = ['381773', '381793', '382065', '382068', '382101', '382120', '382124', '382188', '384263', '384269', '384382', '384385']
nosy_count = 5.0
nosy_names = ['rhettinger', 'ncoghlan', 'serhiy.storchaka', 'BTaskaya', 'uriyyo']
pr_nums = ['23501']
priority = 'normal'
resolution = 'rejected'
stage = 'resolved'
status = 'closed'
superseder = None
type = 'enhancement'
url = 'https://bugs.python.org/issue42455'
versions = ['Python 3.10']

[uriyyo]

In the current python codebases, there are a lot of decorators with parameters, and their code in most cases contains a lot of code boilerplates. The purpose of this issue is to add decorator_with_params function to functools module. This function will allow using a wrapped function as a simple decorator and decorator with parameter.

I believe the real example will bring more context, for instance, typing._tp_cache function can be rewritten from:

def _tp_cache(func=None, /, *, typed=False):
    """Internal wrapper caching __getitem__ of generic types with a fallback to
    original function for non-hashable arguments.
    """
    def decorator(func):
        cached = functools.lru_cache(typed=typed)(func)
        _cleanups.append(cached.cache_clear)

        @functools.wraps(func)
        def inner(*args, **kwds):
            try:
                return cached(*args, **kwds)
            except TypeError:
                pass
            return func(*args, **kwds)
        return inner

    if func is not None:
        return decorator(func)

    return decorator

To

@decorator_with_params
def _tp_cache(func=None, /, *, typed=False):
    """Internal wrapper caching __getitem__ of generic types with a fallback to
    original function for non-hashable arguments.
    """
    cached = functools.lru_cache(typed=typed)(func)
    _cleanups.append(cached.cache_clear)

    @functools.wraps(func)
    def inner(*args, **kwds):
        try:
            return cached(*args, **kwds)
        except TypeError:
            pass  # All real errors (not unhashable args) are raised below.
        return func(*args, **kwds)

    return inner

And _tp_cache can be used as:

    @_tp_cache(typed=True)
    def __getitem__(self, parameters):
        return self._getitem(self, parameters)
    ...
    @_tp_cache
    def __getitem__(self, parameters):
        return self._getitem(self, parameters)

Proposed implementation is quite simple (I used it in a lot of projects):

def decorator_with_params(deco):
    @wraps(deco)
    def decorator(func=None, /, *args, **kwargs):
        if func is not None:
            return deco(func, *args, **kwargs)

        def wrapper(f):
            return deco(f, *args, **kwargs)

        return wrapper

    return decorator

I believe it will be a great enhancement for the Python standard library and will save such important indentations.

[uriyyo]

Add Raymond Hettinger because he is at Expert Index for functools module

[serhiy-storchaka]

I would call it decorator_factory.

[uriyyo]

I agree, decorator factory definitely a better name

[rhettinger]

I've add Nick so that he can opine. For me, I'm -0 on this one. It does factor-out a common pattern, but I find it less obvious what it does and would not enjoy debugging code that used this decorator.

The idea is definitely interesting and worth discussing. If Nick doesn't come along shortly, please bring this up on python-ideas.

[uriyyo]

Let's wait for Nick.

I have found great examples to show how decorator_factory can simplify things.

dataclasses.dataclass

def dataclass(cls=None, /, *, init=True, repr=True, eq=True, order=False,
              unsafe_hash=False, frozen=False):
    def wrap(cls):
        return _process_class(cls, init, repr, eq, order, unsafe_hash, frozen)

    # See if we're being called as @dataclass or @dataclass().
    if cls is None:
        # We're called with parens.
        return wrap

    # We're called as @dataclass without parens.
    return wrap(cls)

@functools.decorator_factory
def dataclass(cls, /, *, init=True, repr=True, eq=True, order=False,
              unsafe_hash=False, frozen=False):
    return _process_class(cls, init, repr, eq, order, unsafe_hash, frozen)

functools.lru_cache

def lru_cache(maxsize=128, typed=False):
    if isinstance(maxsize, int):
        # Negative maxsize is treated as 0
        if maxsize < 0:
            maxsize = 0
    elif callable(maxsize) and isinstance(typed, bool):
        # The user_function was passed in directly via the maxsize argument
        user_function, maxsize = maxsize, 128
        wrapper = _lru_cache_wrapper(user_function, maxsize, typed, _CacheInfo)
        wrapper.cache_parameters = lambda : {'maxsize': maxsize, 'typed': typed}
        return update_wrapper(wrapper, user_function)
    elif maxsize is not None:
        raise TypeError(
            'Expected first argument to be an integer, a callable, or None')

    def decorating_function(user_function):
        wrapper = _lru_cache_wrapper(user_function, maxsize, typed, _CacheInfo)
        wrapper.cache_parameters = lambda : {'maxsize': maxsize, 'typed': typed}
        return update_wrapper(wrapper, user_function)

    return decorating_function

@decorator_factory
def lru_cache(user_function, /, maxsize=128, typed=False):
    if isinstance(maxsize, int):
        # Negative maxsize is treated as 0
        if maxsize < 0:
            maxsize = 0
    elif maxsize is not None:
        raise TypeError(
            'Expected first argument to be an integer, a callable, or None')

    wrapper = _lru_cache_wrapper(user_function, maxsize, typed, _CacheInfo)
    wrapper.cache_parameters = lambda : {'maxsize': maxsize, 'typed': typed}
    return update_wrapper(wrapper, user_function)

[serhiy-storchaka]

Would be nice to include these examples (and more if they exist) in the PR. It will help to see the benefit from the new feature and to test that it does not break anything.

I have no opinion yet, but multiple examples can convince me. Or not.