Can't pickle local object

[connorvr]

Hi,

I am getting the following error when I run it:

WARNING:root:Setting up a new session...
create web directory ./checkpoints\clevr_monet\web...
Traceback (most recent call last):
  File "train.py", line 43, in <module>
    for i, data in enumerate(dataset):  # inner loop within one epoch
  File "F:\Documents\WinPython\MONet-pytorch\data\__init__.py", line 90, in __iter__
    for i, data in enumerate(self.dataloader):
  File "C:\ProgramData\Anaconda3\envs\tf-gpu\lib\site-packages\torch\utils\data\dataloader.py", line 819, in __iter__
    return _DataLoaderIter(self)
  File "C:\ProgramData\Anaconda3\envs\tf-gpu\lib\site-packages\torch\utils\data\dataloader.py", line 560, in __init__
    w.start()
  File "C:\ProgramData\Anaconda3\envs\tf-gpu\lib\multiprocessing\process.py", line 105, in start
    self._popen = self._Popen(self)
  File "C:\ProgramData\Anaconda3\envs\tf-gpu\lib\multiprocessing\context.py", line 223, in _Popen
    return _default_context.get_context().Process._Popen(process_obj)
  File "C:\ProgramData\Anaconda3\envs\tf-gpu\lib\multiprocessing\context.py", line 322, in _Popen
    return Popen(process_obj)
  File "C:\ProgramData\Anaconda3\envs\tf-gpu\lib\multiprocessing\popen_spawn_win32.py", line 65, in __init__
    reduction.dump(process_obj, to_child)
  File "C:\ProgramData\Anaconda3\envs\tf-gpu\lib\multiprocessing\reduction.py", line 60, in dump
    ForkingPickler(file, protocol).dump(obj)
AttributeError: Can't pickle local object '_get_transforms.<locals>.<lambda>'

(tf-gpu) F:\Documents\WinPython\MONet-pytorch>Traceback (most recent call last):
  File "<string>", line 1, in <module>
  File "C:\ProgramData\Anaconda3\envs\tf-gpu\lib\multiprocessing\spawn.py", line 105, in spawn_main
    exitcode = _main(fd)
  File "C:\ProgramData\Anaconda3\envs\tf-gpu\lib\multiprocessing\spawn.py", line 115, in _main
    self = reduction.pickle.load(from_parent)
EOFError: Ran out of input

It seems like the pickling within the multiprocessing library is causing the crash. I would appreciate any ideas on how to fix it.

[baudm]

Hi @connorvr could you try again? It's a pickle error due to the lambda in _get_transforms(). If it still doesn't work, try creating a new environment with Python 3.7, PyTorch 1.1, and CUDA 10. These are the versions I'm using.

[connorvr]

Hmm, no luck. I updated it and created a new conda environment with the versions you listed but get a similar error:

WARNING:root:Setting up a new session...
create web directory ./checkpoints\clevr_monet\web...
Traceback (most recent call last):
  File "train.py", line 43, in <module>
    for i, data in enumerate(dataset):  # inner loop within one epoch
  File "F:\Documents\WinPython\MONet-pytorch\data\__init__.py", line 90, in __iter__
    for i, data in enumerate(self.dataloader):
  File "C:\Users\Connor\.conda\envs\torch-env\lib\site-packages\torch\utils\data\dataloader.py", line 193, in __iter__
    return _DataLoaderIter(self)
  File "C:\Users\Connor\.conda\envs\torch-env\lib\site-packages\torch\utils\data\dataloader.py", line 469, in __init__
    w.start()
  File "C:\Users\Connor\.conda\envs\torch-env\lib\multiprocessing\process.py", line 112, in start
    self._popen = self._Popen(self)
  File "C:\Users\Connor\.conda\envs\torch-env\lib\multiprocessing\context.py", line 223, in _Popen
    return _default_context.get_context().Process._Popen(process_obj)
  File "C:\Users\Connor\.conda\envs\torch-env\lib\multiprocessing\context.py", line 322, in _Popen
    return Popen(process_obj)
  File "C:\Users\Connor\.conda\envs\torch-env\lib\multiprocessing\popen_spawn_win32.py", line 89, in __init__
    reduction.dump(process_obj, to_child)
  File "C:\Users\Connor\.conda\envs\torch-env\lib\multiprocessing\reduction.py", line 60, in dump
    ForkingPickler(file, protocol).dump(obj)
_pickle.PicklingError: Can't pickle <function <lambda> at 0x000001D62A2EA488>: attribute lookup <lambda> on data.clevr_dataset failed