Can't pickle trio as a package

[uduse]

import trio
import pickle

pickle.dumps(trio)

gives

TypeError: cannot pickle '_ModuleWithDeprecations' object
---------------------------------------------------------------------------
TypeError                                 Traceback (most recent call last)
/tmp/ipykernel_9132/1773362268.py in <module>
      2 import pickle
      3 
----> 4 pickle.dumps(trio)

TypeError: cannot pickle '_ModuleWithDeprecations' object

This is problematic for serializing objects that have methods that use trio somehow.

[uduse]

This patch seems to work, do you want me to PR as a method?

import trio
import pickle

def __reduce__(self):
    return (self.__class__, tuple())


trio._deprecate._ModuleWithDeprecations.__reduce__ = __reduce__

pickle.dumps(trio._deprecate._ModuleWithDeprecations("hello"))

[njsmith]

It's an internal implementation detail whether any given module is a _ModuleWithDeprecations or a regular module. So I think we want to pickle the same way as a regular module -- which for pickle is not supported, but for cloudpickle (which IIRC ray uses) it basically pickles to import <module name>.

[njsmith]

...Actually, I'm getting a bit confused about cloudpickle here -- I get cannot pickle 'module' object for some modules and not others.

Can you show your real code that's failing, or at least the real traceback?

[uduse]

For example,

import ray
import trio

ray.init()

@ray.remote
def task():
    async def sleeper():
        await trio.sleep(1)

    trio.run(sleeper)

task.remote()

This gives:

/root/.local/share/virtualenvs/moozi-g1CZ00E9/lib/python3.8/site-packages/trio/_core/_multierror.py:511: RuntimeWarning: You seem to already have a custom sys.excepthook handler installed. I'll skip installing Trio's custom handler, but this means MultiErrors will not show full tracebacks.
  warnings.warn(
2021-10-05 06:28:59,438 INFO services.py:1250 -- View the Ray dashboard at http://127.0.0.1:8266
Traceback (most recent call last):
  File "scripts/trio_pickle.py", line 12, in <module>
    task.remote()
  File "/root/.local/share/virtualenvs/moozi-g1CZ00E9/lib/python3.8/site-packages/ray/remote_function.py", line 123, in _remote_proxy
    return self._remote(args=args, kwargs=kwargs)
  File "/root/.local/share/virtualenvs/moozi-g1CZ00E9/lib/python3.8/site-packages/ray/util/tracing/tracing_helper.py", line 293, in _invocation_remote_span
    return method(self, args, kwargs, *_args, **_kwargs)
  File "/root/.local/share/virtualenvs/moozi-g1CZ00E9/lib/python3.8/site-packages/ray/remote_function.py", line 252, in _remote
    self._pickled_function = pickle.dumps(self._function)
  File "/root/.local/share/virtualenvs/moozi-g1CZ00E9/lib/python3.8/site-packages/ray/cloudpickle/cloudpickle_fast.py", line 73, in dumps
    cp.dump(obj)
  File "/root/.local/share/virtualenvs/moozi-g1CZ00E9/lib/python3.8/site-packages/ray/cloudpickle/cloudpickle_fast.py", line 580, in dump
    return Pickler.dump(self, obj)
TypeError: cannot pickle '_ModuleWithDeprecations' object

Moving the import statement into the remote actor seems to solve the problem, but seems too hacky?

import ray

ray.init()

@ray.remote
def task():
    import trio
    async def sleeper():
        await trio.sleep(1)

    trio.run(sleeper)

task.remote()

This works.