Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

AttributeError: Can't pickle local object 'make_env.<locals>._make_env' #51

Open
lucifer2859 opened this issue Sep 23, 2020 · 4 comments

Comments

@lucifer2859
Copy link

Ubuntu 18.04.5 LTS
Meet this problem after adding 'mp.set_start_method('spawn')'
#40 (comment)

Traceback (most recent call last):
File "train.py", line 122, in
main(args)
File "train.py", line 54, in main
num_workers=args.num_workers)
File "/home/dchen/pytorch-maml-rl/maml_rl/samplers/multi_task_sampler.py", line 107, in init
worker.start()
File "/home/dchen/anaconda3/envs/torch/lib/python3.6/multiprocessing/process.py", line 105, in start
self._popen = self._Popen(self)
File "/home/dchen/anaconda3/envs/torch/lib/python3.6/multiprocessing/context.py", line 223, in _Popen
return _default_context.get_context().Process._Popen(process_obj)
File "/home/dchen/anaconda3/envs/torch/lib/python3.6/multiprocessing/context.py", line 284, in _Popen
return Popen(process_obj)
File "/home/dchen/anaconda3/envs/torch/lib/python3.6/multiprocessing/popen_spawn_posix.py", line 32, in init
super().init(process_obj)
File "/home/dchen/anaconda3/envs/torch/lib/python3.6/multiprocessing/popen_fork.py", line 19, in init
self._launch(process_obj)
File "/home/dchen/anaconda3/envs/torch/lib/python3.6/multiprocessing/popen_spawn_posix.py", line 47, in _launch
reduction.dump(process_obj, fp)
File "/home/dchen/anaconda3/envs/torch/lib/python3.6/multiprocessing/reduction.py", line 60, in dump
ForkingPickler(file, protocol).dump(obj)
AttributeError: Can't pickle local object 'make_env.._make_env'

@lucifer2859
Copy link
Author

I see the similar problem in #41

@tristandeleu
Copy link
Owner

I am able to replicate this issue using mp.set_start_method('spawn'). Can you try the fix/mp-spawn branch, see if that fixes your issue?
I'm having trouble understanding why this doesn't work though. I know that objects that are passed as arguments to a process are pickled, and local functions generally don't work well with pickle, but here that function is not passed as an argument, but is instead called inside the process (and therefore shouldn't need to be pickled?).

@lucifer2859
Copy link
Author

Thanks a lot. After adding 'mp.set_start_method('spawn')', I fix this problem using this branch https://github.com/tristandeleu/pytorch-maml-rl/tree/fix/mp-spawn

@TriumphLearning
Copy link

Thanks a lot. After adding 'mp.set_start_method('spawn')', I fix this problem using this branch https://github.com/tristandeleu/pytorch-maml-rl/tree/fix/mp-spawn

Hello, I have same problem with you,and I just wonder where 'mp.set_start_method('spawn')' should be placed in that position in the branch https://github.com/tristandeleu/pytorch-maml-rl/tree/fix/mp-spawn ?
Thank you very much

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

No branches or pull requests

3 participants