Joblib can't pickle torch.dtype objects (but both pickle and cloudpickle can)

[anna-hope]

Minimal steps to reproduce:

import torch
from joblib import Parallel, delayed
import pickle, cloudpickle


def identity(x):
    return x

dtypes = [torch.int32 for _ in range(100)]
Parallel(n_jobs=-1)(delayed(identity)(x) for x in dtypes)

Results in the following error:

Traceback (most recent call last):
  File "/opt/anaconda3/lib/python3.7/site-packages/joblib/externals/loky/backend/queues.py", line 150, in _feed
    obj_ = dumps(obj, reducers=reducers)
  File "/opt/anaconda3/lib/python3.7/site-packages/joblib/externals/loky/backend/reduction.py", line 243, in dumps
    dump(obj, buf, reducers=reducers, protocol=protocol)
  File "/opt/anaconda3/lib/python3.7/site-packages/joblib/externals/loky/backend/reduction.py", line 236, in dump
    _LokyPickler(file, reducers=reducers, protocol=protocol).dump(obj)
  File "/opt/anaconda3/lib/python3.7/site-packages/joblib/externals/cloudpickle/cloudpickle.py", line 267, in dump
    return Pickler.dump(self, obj)
  File "/opt/anaconda3/lib/python3.7/pickle.py", line 437, in dump
    self.save(obj)
  File "/opt/anaconda3/lib/python3.7/pickle.py", line 549, in save
    self.save_reduce(obj=obj, *rv)
  File "/opt/anaconda3/lib/python3.7/pickle.py", line 662, in save_reduce
    save(state)
  File "/opt/anaconda3/lib/python3.7/pickle.py", line 504, in save
    f(self, obj) # Call unbound method with explicit self
  File "/opt/anaconda3/lib/python3.7/site-packages/dill/_dill.py", line 902, in save_module_dict
    StockPickler.save_dict(pickler, obj)
  File "/opt/anaconda3/lib/python3.7/pickle.py", line 856, in save_dict
    self._batch_setitems(obj.items())
  File "/opt/anaconda3/lib/python3.7/pickle.py", line 882, in _batch_setitems
    save(v)
  File "/opt/anaconda3/lib/python3.7/pickle.py", line 549, in save
    self.save_reduce(obj=obj, *rv)
  File "/opt/anaconda3/lib/python3.7/pickle.py", line 662, in save_reduce
    save(state)
  File "/opt/anaconda3/lib/python3.7/pickle.py", line 504, in save
    f(self, obj) # Call unbound method with explicit self
  File "/opt/anaconda3/lib/python3.7/site-packages/dill/_dill.py", line 902, in save_module_dict
    StockPickler.save_dict(pickler, obj)
  File "/opt/anaconda3/lib/python3.7/pickle.py", line 856, in save_dict
    self._batch_setitems(obj.items())
  File "/opt/anaconda3/lib/python3.7/pickle.py", line 887, in _batch_setitems
    save(v)
  File "/opt/anaconda3/lib/python3.7/pickle.py", line 549, in save
    self.save_reduce(obj=obj, *rv)
  File "/opt/anaconda3/lib/python3.7/pickle.py", line 662, in save_reduce
    save(state)
  File "/opt/anaconda3/lib/python3.7/pickle.py", line 504, in save
    f(self, obj) # Call unbound method with explicit self
  File "/opt/anaconda3/lib/python3.7/site-packages/dill/_dill.py", line 902, in save_module_dict
    StockPickler.save_dict(pickler, obj)
  File "/opt/anaconda3/lib/python3.7/pickle.py", line 856, in save_dict
    self._batch_setitems(obj.items())
  File "/opt/anaconda3/lib/python3.7/pickle.py", line 882, in _batch_setitems
    save(v)
  File "/opt/anaconda3/lib/python3.7/pickle.py", line 504, in save
    f(self, obj) # Call unbound method with explicit self
  File "/opt/anaconda3/lib/python3.7/pickle.py", line 816, in save_list
    self._batch_appends(obj)
  File "/opt/anaconda3/lib/python3.7/pickle.py", line 843, in _batch_appends
    save(tmp[0])
  File "/opt/anaconda3/lib/python3.7/pickle.py", line 504, in save
    f(self, obj) # Call unbound method with explicit self
  File "/opt/anaconda3/lib/python3.7/pickle.py", line 771, in save_tuple
    save(element)
  File "/opt/anaconda3/lib/python3.7/pickle.py", line 504, in save
    f(self, obj) # Call unbound method with explicit self
  File "/opt/anaconda3/lib/python3.7/pickle.py", line 771, in save_tuple
    save(element)
  File "/opt/anaconda3/lib/python3.7/pickle.py", line 535, in save
    self.save_global(obj, rv)
  File "/opt/anaconda3/lib/python3.7/site-packages/joblib/externals/cloudpickle/cloudpickle.py", line 692, in save_global
    if obj.__module__ == "__main__":
AttributeError: 'torch.dtype' object has no attribute '__module__'
"""

However, the following examples work correctly:

pickle.dumps(torch.int32)
cloudpickle.dumps(torch.int32)

Environment:

Python 3.7.3 (Anaconda)
joblib 0.13.2
cloudpickle 1.2.1
pytorch 1.1.0

[pierreglaser]

The last joblib release internally uses an old cloudpickle. joblib master uses a new cloudpickle version, and on master, this bug should not appear. We will make a new joblib release with a recent cloudpickle soon.