Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

tests fail on Python 3.8 (macOS) #30

Closed
belm0 opened this issue Jun 2, 2021 · 1 comment
Closed

tests fail on Python 3.8 (macOS) #30

belm0 opened this issue Jun 2, 2021 · 1 comment

Comments

@belm0
Copy link
Member

belm0 commented Jun 2, 2021

e.g.

_____ ERROR at setup of test_grpc_client_large_payload[purerpc_echo_port] ______

cm_gen = <function run_purerpc_service_in_process.<locals>.target_fn at 0x10c175310>

    @contextlib.contextmanager
    def _run_context_manager_generator_in_process(cm_gen):
        parent_conn, child_conn = multiprocessing.Pipe(duplex=False)
        target_fn = _wrap_gen_in_process(child_conn)(cm_gen)

        process = multiprocessing.Process(target=target_fn)
>       process.start()

src/purerpc/test_utils.py:97:
_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _
../../.pyenv/versions/3.8.5/lib/python3.8/multiprocessing/process.py:121: in start
    self._popen = self._Popen(self)
../../.pyenv/versions/3.8.5/lib/python3.8/multiprocessing/context.py:224: in _Popen
    return _default_context.get_context().Process._Popen(process_obj)
../../.pyenv/versions/3.8.5/lib/python3.8/multiprocessing/context.py:284: in _Popen
    return Popen(process_obj)
../../.pyenv/versions/3.8.5/lib/python3.8/multiprocessing/popen_spawn_posix.py:32: in __init__
    super().__init__(process_obj)
../../.pyenv/versions/3.8.5/lib/python3.8/multiprocessing/popen_fork.py:19: in __init__
    self._launch(process_obj)
../../.pyenv/versions/3.8.5/lib/python3.8/multiprocessing/popen_spawn_posix.py:47: in _launch
    reduction.dump(process_obj, fp)
_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _

obj = <Process name='Process-1' parent=73643 initial>
file = <_io.BytesIO object at 0x10c1ddf40>, protocol = None

    def dump(obj, file, protocol=None):
        '''Replacement for pickle.dump() using ForkingPickler.'''
>       ForkingPickler(file, protocol).dump(obj)
E       AttributeError: Can't pickle local object 'run_purerpc_service_in_process.<locals>.target_fn'

../../.pyenv/versions/3.8.5/lib/python3.8/multiprocessing/reduction.py:60: AttributeError
@belm0 belm0 changed the title tests fail on Python 3.8 tests fail on Python 3.8 (macOS) Jun 2, 2021
@belm0
Copy link
Member Author

belm0 commented Jun 2, 2021

It may be specific to macOS.

Changed in version 3.8: On macOS, the spawn start method is now the default. The fork start method should be considered unsafe as it can lead to crashes of the subprocess. See bpo-33725.

https://docs.python.org/3/library/multiprocessing.html#contexts-and-start-methods

Using spawn seems to have various implications for existing code.

https://chrissardegna.com/blog/multiprocessing-changes-python-3-8/#picklingerror-crash

Possible workaround (given that it would only be utilized for tests):

multiprocessing.set_start_method("fork")

@belm0 belm0 closed this as completed in 9b59b66 Apr 15, 2022
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

No branches or pull requests

1 participant