New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

cannot pass class-bound method to context call #313

Closed
jesteria opened this Issue Jul 20, 2018 · 3 comments

Comments

Projects
None yet
2 participants
@jesteria
Contributor

jesteria commented Jul 20, 2018

As described by the documentation, I expect to be able to specify as a call target of Context.call_asyncfn – either a:

  • module-level function
  • static method
  • class method

However, at least in naively passing the Python object reference of either version of a method, mitogen is either unable to find or unable to process the call target.

For example, using staticmethod

#!/usr/bin/env python
import os
import pathlib
import socket

import mitogen.utils

class MyClass:

    @staticmethod
    def my_first_function():
        print('hostname:', socket.gethostname())
        print('remote PID:', os.getpid())
        print('file:', pathlib.Path(__file__))
        return 123

    @classmethod
    def main(cls, router):
        local = router.local()
        result = local.call(cls.my_first_function)
        print(result, socket.gethostname(), os.getpid())

if __name__ == '__main__':
    mitogen.utils.log_to_file()
    mitogen.utils.run_with_router(MyClass.main)

– results in the following error –

Traceback (most recent call last):
  File "./fake_manage_method.py", line 27, in <module>
    mitogen.utils.run_with_router(MyClass.main)
  File "/home/USER/dev/repo/opt/mitogen/mitogen/utils.py", line 100, in run_with_router
    return func(router, *args, **kwargs)
  File "./fake_manage_method.py", line 21, in main
    result = local.call(cls.my_first_function)
  File "/home/USER/dev/repo/opt/mitogen/mitogen/parent.py", line 1016, in call
    return receiver.get().unpickle(throw_dead=False)
  File "/home/USER/dev/repo/opt/mitogen/mitogen/core.py", line 487, in unpickle
    raise obj
mitogen.core.CallError: builtins.AttributeError: module '__main__' has no attribute 'my_first_function'
  File "<stdin>", line 2049, in _dispatch_calls
  File "<stdin>", line 2036, in _dispatch_one

Or, replacing the staticmethod with classmethod:

Traceback (most recent call last):
  File "./fake_manage_method.py", line 27, in <module>
    mitogen.utils.run_with_router(MyClass.main)
  File "/home/USER/dev/repo/opt/mitogen/mitogen/utils.py", line 100, in run_with_router
    return func(router, *args, **kwargs)
  File "./fake_manage_method.py", line 21, in main
    result = local.call(cls.my_first_function)
  File "/home/USER/dev/repo/opt/mitogen/mitogen/parent.py", line 1015, in call
    receiver = self.call_async(fn, *args, **kwargs)
  File "/home/USER/dev/repo/opt/mitogen/mitogen/parent.py", line 1012, in call_async
    return self.send_async(make_call_msg(fn, *args, **kwargs))
  File "/home/USER/dev/repo/opt/mitogen/mitogen/parent.py", line 401, in make_call_msg
    isinstance(fn.im_self, (type, types.ClassType)):
AttributeError: 'function' object has no attribute 'im_self'

In the case of the staticmethod, (unpickling), this may be worked around, for example –

def my_first_function():
    print('hostname:', socket.gethostname())
    print('remote PID:', os.getpid())
    print('file:', pathlib.Path(__file__))
    return 123

class MyClass:

    my_first_function = staticmethod(my_first_function)

    @classmethod
    def main(cls, router):
        local = router.local()
        result = local.call(cls.my_first_function)
        print(result, socket.gethostname(), os.getpid())    

– such that mitogen can now find the call target at the module level; (however, if mitogen intends to support static methods, this is less than ideal).

@dw

This comment has been minimized.

Owner

dw commented Jul 21, 2018

Yeah this is far from ideal -- actually, I'm not sure how static methods ended up in the docs, they're very hard to disambiguate on Python 2. Let me think about this one -- a proper solution to this is actually quite hard (AST parsing wasn't available in 2.4), but a 3.x-only solution is easily doable at least

dw added a commit that referenced this issue Jul 28, 2018

Remove staticmethod from docs.
Can re-add this later for 3.x, but it's pretty impossible in general for
2.x.

Closes #313.
@dw

This comment has been minimized.

Owner

dw commented Jul 28, 2018

Hi there, I've just removed staticmethod from the docs. It's not worth the effort to support -- so sorry for the original confusion! Those docs were written about ~8 years after the function call code ;)

@dw dw closed this Jul 29, 2018

@jesteria

This comment has been minimized.

Contributor

jesteria commented Sep 4, 2018

No problem, that makes sense to me.

Does this resolve the issue with classmethod however?

Sorry for the delayed response, but the original issue referred to class methods as well:

Or, replacing the staticmethod with classmethod:

Traceback (most recent call last):
  File "./fake_manage_method.py", line 27, in <module>
    mitogen.utils.run_with_router(MyClass.main)
  File "/home/USER/dev/repo/opt/mitogen/mitogen/utils.py", line 100, in run_with_router
    return func(router, *args, **kwargs)
  File "./fake_manage_method.py", line 21, in main
    result = local.call(cls.my_first_function)
  File "/home/USER/dev/repo/opt/mitogen/mitogen/parent.py", line 1015, in call
    receiver = self.call_async(fn, *args, **kwargs)
  File "/home/USER/dev/repo/opt/mitogen/mitogen/parent.py", line 1012, in call_async
    return self.send_async(make_call_msg(fn, *args, **kwargs))
  File "/home/USER/dev/repo/opt/mitogen/mitogen/parent.py", line 401, in make_call_msg
    isinstance(fn.im_self, (type, types.ClassType)):
AttributeError: 'function' object has no attribute 'im_self'

The issue with class methods is a bit more awkward to work around, since the target callable has an implicit expectation that it will receive the class, after all. That said, it looked more like something that just needed some tweaking, since class-bound methods are more straight-forward to detect, (though I could certainly be missing something).

Thanks again.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment