PicklingError when using cotyledon on win64

[jdandrea]

Unsure how to proceed/mitigate this one. I normally run on Ubuntu or macOS. First time we're trying on win64 (developer workstation).

It seems to originate with a call from cotyledon, but I don't see any reference to identity in the cotyledon source, nor in my own project. Thoughts welcomed!

File "C:\...path-to-venv...\lib\site-packages\cotyledon\_utils.py", line 69, in spawn_process
  p.start()
File "C:\Python27\Lib\multiprocessing\process.py", line 130, in start
  self._popen = Popen(self)
File "C:\Python27\Lib\multiprocessing\forking.py", line 277, in __init__
  dump(process_obj, to_child, HIGHEST_PROTOCOL)
File "C:\Python27\Lib\multiprocessing\forking.py", line 199, in dump
  ForkingPickler(file, protocol).dump(obj)
File "C:\Python27\Lib\pickle.py", line 224, in dump
  self.save(obj)
File "C:\Python27\Lib\pickle.py", line 331, in save
  self.save_reduce(obj=obj, *rv)
File "C:\Python27\Lib\pickle.py", line 425, in save_reduce
  save(state)
File "C:\Python27\Lib\pickle.py", line 286, in save
  f(self, obj) # Call unbound method with explicit self
File "C:\Python27\Lib\pickle.py", line 655, in save_dict
  self._batch_setitems(obj.iteritems())
File "C:\Python27\Lib\pickle.py", line 687, in _batch_setitems
  save(v)
File "C:\Python27\Lib\pickle.py", line 286, in save
  f(self, obj) # Call unbound method with explicit self
File "C:\Python27\Lib\pickle.py", line 568, in save_tuple
  save(element)
File "C:\Python27\Lib\pickle.py", line 331, in save
  self.save_reduce(obj=obj, *rv)
File "C:\Python27\Lib\pickle.py", line 425, in save_reduce
  save(state)
File "C:\Python27\Lib\pickle.py", line 286, in save
  f(self, obj) # Call unbound method with explicit self
File "C:\Python27\Lib\pickle.py", line 655, in save_dict
  self._batch_setitems(obj.iteritems())
File "C:\Python27\Lib\pickle.py", line 687, in _batch_setitems
  save(v)
File "C:\Python27\Lib\pickle.py", line 286, in save
  f(self, obj) # Call unbound method with explicit self
File "C:\Python27\Lib\pickle.py", line 554, in save_tuple
  save(element)
File "C:\Python27\Lib\pickle.py", line 331, in save
  self.save_reduce(obj=obj, *rv)
File "C:\Python27\Lib\pickle.py", line 425, in save_reduce
  save(state)
File "C:\Python27\Lib\pickle.py", line 286, in save
  f(self, obj) # Call unbound method with explicit self
File "C:\Python27\Lib\pickle.py", line 655, in save_dict
  self._batch_setitems(obj.iteritems())
File "C:\Python27\Lib\pickle.py", line 687, in _batch_setitems
  save(v)
File "C:\Python27\Lib\pickle.py", line 331, in save
  self.save_reduce(obj=obj, *rv)
File "C:\Python27\Lib\pickle.py", line 425, in save_reduce
  save(state)
File "C:\Python27\Lib\pickle.py", line 286, in save
  f(self, obj) # Call unbound method with explicit self
File "C:\Python27\Lib\pickle.py", line 655, in save_dict
  self._batch_setitems(obj.iteritems())
File "C:\Python27\Lib\pickle.py", line 687, in _batch_setitems
  save(v)
File "C:\Python27\Lib\pickle.py", line 286, in save
  f(self, obj) # Call unbound method with explicit self
File "C:\Python27\Lib\pickle.py", line 655, in save_dict
  self._batch_setitems(obj.iteritems())
File "C:\Python27\Lib\pickle.py", line 687, in _batch_setitems
  save(v)
File "C:\Python27\Lib\pickle.py", line 331, in save
  self.save_reduce(obj=obj, *rv)
File "C:\Python27\Lib\pickle.py", line 425, in save_reduce
  save(state)
File "C:\Python27\Lib\pickle.py", line 286, in save
  f(self, obj) # Call unbound method with explicit self
File "C:\Python27\Lib\pickle.py", line 655, in save_dict
  self._batch_setitems(obj.iteritems())
File "C:\Python27\Lib\pickle.py", line 687, in _batch_setitems
  save(v)
File "C:\Python27\Lib\pickle.py", line 331, in save
  self.save_reduce(obj=obj, *rv)
File "C:\Python27\Lib\pickle.py", line 425, in save_reduce
  save(state)
File "C:\Python27\Lib\pickle.py", line 286, in save
  f(self, obj) # Call unbound method with explicit self
File "C:\Python27\Lib\pickle.py", line 655, in save_dict
  self._batch_setitems(obj.iteritems())
File "C:\Python27\Lib\pickle.py", line 687, in _batch_setitems
  save(v)
File "C:\Python27\Lib\pickle.py", line 286, in save
  f(self, obj) # Call unbound method with explicit self
File "C:\Python27\Lib\pickle.py", line 655, in save_dict
  self._batch_setitems(obj.iteritems())
File "C:\Python27\Lib\pickle.py", line 687, in _batch_setitems
  save(v)
File "C:\Python27\Lib\pickle.py", line 286, in save
  f(self, obj) # Call unbound method with explicit self
File "C:\Python27\Lib\pickle.py", line 655, in save_dict
  self._batch_setitems(obj.iteritems())
File "C:\Python27\Lib\pickle.py", line 692, in _batch_setitems
  save(v)
File "C:\Python27\Lib\pickle.py", line 286, in save
  f(self, obj) # Call unbound method with explicit self
File "C:\Python27\Lib\pickle.py", line 754, in save_global
  (obj, module, name))

PicklingError: Can't pickle <function identity at 0x03F9B230>: it's not found as argparse.identity

[sileht]

To support windows all objects passed to children process must support Pickle. Your application seems the pass some argparse objects to children. More generally master process should do a very minimal works.

[sileht]

I 'm guessing identity is this one: https://github.com/python/cpython/blob/master/Lib/argparse.py#L1638

[jdandrea]

Thank you! I did a search for argparse in my project and only one component uses it: pbr. I think I will ask them about it or see if anyone has filed an issue.

[jacobluo]

@jdandrea how to solve this problem on win64?

[jdandrea]

Whoops, I'm now at a loss as to how the case was cracked (and why on earth nothing was mentioned herein). I will go ISO answers and, hopefully, report back.

[jdandrea]

@jacobluo A patch has merged for pbr that I hope resolves this issue.