Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

operator.call/operator.__call__ #88185

Closed
anntzer mannequin opened this issue May 3, 2021 · 18 comments
Closed

operator.call/operator.__call__ #88185

anntzer mannequin opened this issue May 3, 2021 · 18 comments
Labels
3.11 only security fixes stdlib Python modules in the Lib dir type-feature A feature request or enhancement

Comments

@anntzer
Copy link
Mannequin

anntzer mannequin commented May 3, 2021

BPO 44019
Nosy @brettcannon, @rhettinger, @terryjreedy, @mdickinson, @vstinner, @corona10, @hrik2001, @Kreusada
PRs
  • bpo-44019: Implement operator.call(). #27888
  • bpo-44019: Add missing comma to operator.call doc #28551
  • bpo-44019: Add operator.call() to __all__ for the operator module #29110
  • bpo-44019: Add test_all_exported_names for operator module #29124
  • Revert "bpo-44019: Add test_all_exported_names for operator module" #29142
  • Note: these values reflect the state of the issue at the time it was migrated and might not reflect the current state.

    Show more details

    GitHub fields:

    assignee = None
    closed_at = <Date 2021-09-24.15:26:49.947>
    created_at = <Date 2021-05-03.14:24:00.272>
    labels = ['type-feature', 'library', '3.11']
    title = 'operator.call/operator.__call__'
    updated_at = <Date 2021-10-21.22:58:23.491>
    user = 'https://github.com/anntzer'

    bugs.python.org fields:

    activity = <Date 2021-10-21.22:58:23.491>
    actor = 'corona10'
    assignee = 'none'
    closed = True
    closed_date = <Date 2021-09-24.15:26:49.947>
    closer = 'mark.dickinson'
    components = ['Library (Lib)']
    creation = <Date 2021-05-03.14:24:00.272>
    creator = 'Antony.Lee'
    dependencies = []
    files = []
    hgrepos = []
    issue_num = 44019
    keywords = ['patch']
    message_count = 15.0
    messages = ['392809', '398859', '400125', '400627', '400628', '400640', '400786', '400821', '402570', '402572', '402580', '404579', '404585', '404605', '404702']
    nosy_count = 8.0
    nosy_names = ['brett.cannon', 'rhettinger', 'terry.reedy', 'mark.dickinson', 'vstinner', 'corona10', 'hrik2001', 'Kreusada']
    pr_nums = ['27888', '28551', '29110', '29124', '29142']
    priority = 'normal'
    resolution = None
    stage = 'resolved'
    status = 'closed'
    superseder = None
    type = 'enhancement'
    url = 'https://bugs.python.org/issue44019'
    versions = ['Python 3.11']

    @anntzer
    Copy link
    Mannequin Author

    anntzer mannequin commented May 3, 2021

    Adding a call/call function to the operator module (where operator.call(*args, **kwargs)(func) == func(*args, **kwargs), similarly to operator.methodcaller) seems consistent with the design with the rest of the operator module.

    An actual use case I had for such an operator was collecting a bunch of callables in a list and wanting to dispatch them to concurrent.futures.Executor.map, i.e. something like executor.map(operator.call, funcs) (to get the parallelized version of [func() for func in funcs]).

    @anntzer anntzer mannequin added 3.11 only security fixes stdlib Python modules in the Lib dir labels May 3, 2021
    @anntzer
    Copy link
    Mannequin Author

    anntzer mannequin commented Aug 4, 2021

    Actually, upon further thought, the semantics I suggested above should go into operator.caller (cf. operator.methodcaller), and operator.call/operator.__call__ should instead be defined as operator.call(f, *args, **kwargs) == f(*args, **kwargs), so that the general rule operator.opname(a, b) == a.__opname__(b) (modulo dunder lookup rules) remains applicable.

    @mdickinson
    Copy link
    Member

    This seems like a reasonable addition to me. Victor: any thoughts?

    @vstinner
    Copy link
    Member

    Python 2.7 had apply(func, args, kwargs) which called func(*args, **kwargs).
    https://docs.python.org/2.7/library/functions.html#apply

    There is also functools.partial(func, *args, **kwargs)(*args2, **kwargs2) which calls func(*args, *args2, **kwargs, **kwargs2).
    https://docs.python.org/dev/library/functools.html#functools.partial

    operator.methodcaller(name, /, *args, **kwargs)(obj) calls getattr(obj, name)(*args, **kwargs).
    https://docs.python.org/dev/library/operator.html#operator.methodcaller

    I'm not convinced that operator.caller() would be useful to me. Why do you consider that it belongs to the stdlib? It is a common pattern? Did you see in this pattern in the current stdlib?

    Can't you easily implement such helper function in a few lines of Python?

    operator documentation says: "The operator module exports a set of efficient functions corresponding to the intrinsic operators of Python". I don't see how operator.caller() implements an existing "intrinsic operators of Python".

    methodcaller() can be implemented in 4 lines of Python, as shown in its documentation:
    ---

    def methodcaller(name, /, *args, **kwargs):
        def caller(obj):
            return getattr(obj, name)(*args, **kwargs)
        return caller

    @vstinner
    Copy link
    Member

    An actual use case I had for such an operator was collecting a bunch of callables in a list and wanting to dispatch them to concurrent.futures.Executor.map, i.e. something like executor.map(operator.call, funcs) (to get the parallelized version of [func() for func in funcs]).

    Can't you use functools.partial() for that?

    @anntzer
    Copy link
    Mannequin Author

    anntzer mannequin commented Aug 30, 2021

    I'm not convinced that operator.caller() would be useful to me.

    To be clear, as noted above, I have realized that the semantics I initially proposed (now known as "caller") are not particularly useful; the semantics I am proposing (and implementing in the linked PR) are call(f, *args, **kwargs) == f(*args, **kwargs).

    I don't see how operator.caller() implements an existing "intrinsic operators of Python".

    Agreed; on the other hand function calling is much more intrinsic(?!)

    Can't you use functools.partial() for that?

    How do you propose to do that? Perhaps I am missing an easy solution...

    @vstinner
    Copy link
    Member

    call(f, *args, **kwargs) == f(*args, **kwargs)

    So you can want to reintroduce the Python 2 apply() function which was removed in Python 3.

    You can reimplement it in 2 lines, no?

    def call(func, *args, **kwargs):
      return func(*args, **kwargs)

    @anntzer
    Copy link
    Mannequin Author

    anntzer mannequin commented Sep 1, 2021

    Python2's apply has different semantics: it takes non-unpacked arguments, i.e.

        def apply(f, args, kwargs={}): return f(*args, **kwargs)

    rather than

        def call(f, *args, **kwargs): return f(*args, **kwargs)

    I agree that both functions can be written in two (or one) line, but the same can be said of most functions in the operator module (def add(x, y): return x + y); from the module's doc ("efficient functions corresponding to the intrinsic operators"), I would argue that the criteria for inclusion are efficiency (operator.call is indeed fast, see the linked PR) and intrinsicness (I don't know if there's a hard definition, but function calling certainly seems intrinsic).

    @mdickinson
    Copy link
    Member

    New changeset 6587fc6 by Antony Lee in branch 'main':
    bpo-44019: Implement operator.call(). (GH-27888)
    6587fc6

    @mdickinson
    Copy link
    Member

    Thanks for the contribution!

    @mdickinson mdickinson added the type-feature A feature request or enhancement label Sep 24, 2021
    @mdickinson mdickinson added the type-feature A feature request or enhancement label Sep 24, 2021
    @mdickinson
    Copy link
    Member

    New changeset bfe26bb by Terry Jan Reedy in branch 'main':
    bpo-44019: Add missing comma to operator.call doc (GH-28551)
    bfe26bb

    @corona10
    Copy link
    Member

    New changeset a53456e by Kreus Amredes in branch 'main':
    bpo-44019: Add operator.call() to __all__ for the operator module (GH-29110)
    a53456e

    @vstinner
    Copy link
    Member

    test___all__ was not supposed to fail with the missing "call" in operator.__all__?

    @corona10
    Copy link
    Member

    test___all__ was not supposed to fail with the missing "call" in operator.__all__?

    AFAIK, it doesn't check.

    I add the test for the operator module.

    @corona10
    Copy link
    Member

    New changeset 37fad7d by Dong-hee Na in branch 'main':
    bpo-44019: Add test_all_exported_names for operator module (GH-29124)
    37fad7d

    @ezio-melotti ezio-melotti transferred this issue from another repository Apr 10, 2022
    @mrucker
    Copy link

    mrucker commented Dec 29, 2022

    I'd really like to use this in python 3.7. Any chance there is a python package available which backports this?

    @terryjreedy
    Copy link
    Member

    terryjreedy commented Dec 29, 2022

    operator is written in python, so you can copy anything you want into your code, or even monkeypatch operator.

    def call(obj, /, *args, **kwargs):
        """Same as obj(*args, **kwargs)."""
        return obj(*args, **kwargs)
    
    __call__ = call

    @mrucker
    Copy link

    mrucker commented Dec 29, 2022

    @terryjreedy Thanks! I have a python implementation already but it is a little slower than I'd like so I was hoping the c implementation in the operator module might be a little faster.

    Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
    Labels
    3.11 only security fixes stdlib Python modules in the Lib dir type-feature A feature request or enhancement
    Projects
    None yet
    Development

    No branches or pull requests

    5 participants