Skip to content

Commit

Permalink
Fix dill._dill submodule being saved as `GLOBAL "dill._shims" "_dil…
Browse files Browse the repository at this point in the history
…l"` (#490)

I'm tracing some problems with my "portable" mode prototype and hit a bug with references the `_dill` submodule saved as global:

```python
>>> import dill
>>> dill.dumps(dill._dill, 0)
b'cdill._shims\n_dill\np0\n.'
```
The `_dill` submodule is saved as an attribute of the `dill._shims` submodule, which is valid, but it should be just `dill._dill`.

The `_dill` submodule is special-cased to be saved as global:
https://github.com/uqfoundation/dill/blob/8e5e450b9ed8dff164fc259c468365e2235c6581/dill/_dill.py#L1807-L1810

But `pickle.whichmodule` misidentifies it as pertaining to `dill._shims`, because this entry is set earlier in `sys.modules` and it picks the first match.

---

The change fixes things for this especial case, but there is potential for new bugs related to other submodules:
```python
>>> import dill, glob, importlib, os, pickle, pprint
>>> os.chdir(dill.__path__[0])
>>> modules = [mod.rpartition('.')[0] for mod in glob.glob('*py') if not mod.startswith('__')]
>>> modules = {mod: importlib.import_module('dill.' + mod) for mod in modules}
>>> pprint.pprint({name: pickle.whichmodule(mod, name) for name, mod in modules.items()})
{'_dill': 'dill._shims',
 '_objects': 'dill',
 '_shims': 'dill._dill',
 'detect': 'dill',
 'objtypes': 'dill',
 'pointers': 'dill',
 'settings': '__main__',
 'source': 'dill',
 'temp': 'dill'}
```
Note how `_shims` is attributed to `dill._dill` and `settings` is attributed to `__main__`(???). But currently they are not saved as globals.
  • Loading branch information
leogama committed Jun 8, 2022
1 parent 260b372 commit 4462409
Showing 1 changed file with 2 additions and 0 deletions.
2 changes: 2 additions & 0 deletions dill/_dill.py
Original file line number Diff line number Diff line change
Expand Up @@ -21,6 +21,8 @@
'UnpicklingError','HANDLE_FMODE','CONTENTS_FMODE','FILE_FMODE',
'PickleError','PickleWarning','PicklingWarning','UnpicklingWarning']

__module__ = 'dill'

import logging
log = logging.getLogger("dill")
log.addHandler(logging.StreamHandler())
Expand Down

0 comments on commit 4462409

Please sign in to comment.