Skip to content

Commit

Permalink
Fix Python 3 compatibility bug in pickler.py
Browse files Browse the repository at this point in the history
sys.modules changes during iteration. It seems that the intention was to
make a copy using sys.modules.values(), which returns a copy in Python
2, but a view in Python 3.

Partial list of modules observed as added during iteration:
```python
{<module 'py._log' from '/home/ehudm/src/beam/sdks/python/target/.tox/py3-pytest/lib/python3.5/site-packages/py/_log/__init__.py'>, <module 'syslog' (built-in)>, <module 'py._log.log' from '/home/ehudm/src/beam/sdks/python/target/.tox/py3-pytest/lib/python3.5/site-packages/py/_log/log.py'>, <module 'py._log.warning' from '/home/ehudm/src/beam/sdks/python/target/.tox/py3-pytest/lib/python3.5/site-packages/py/_log/warning.py'>}
{<module 'py._process.forkedfunc' from '/home/ehudm/src/beam/sdks/python/target/.tox/py3-pytest/lib/python3.5/site-packages/py/_process/forkedfunc.py'>, <module 'py._process.killproc' from '/home/ehudm/src/beam/sdks/python/target/.tox/py3-pytest/lib/python3.5/site-packages/py/_process/killproc.py'>, <module 'py._process.cmdexec' from '/home/ehudm/src/beam/sdks/python/target/.tox/py3-pytest/lib/python3.5/site-packages/py/_process/cmdexec.py'>, <module 'py._process' from '/home/ehudm/src/beam/sdks/python/target/.tox/py3-pytest/lib/python3.5/site-packages/py/_process/__init__.py'>}
{<module 'py._path.cacheutil' from '/home/ehudm/src/beam/sdks/python/target/.tox/py3-pytest/lib/python3.5/site-packages/py/_path/cacheutil.py'>, <module 'py._path.svnurl' from '/home/ehudm/src/beam/sdks/python/target/.tox/py3-pytest/lib/python3.5/site-packages/py/_path/svnurl.py'>, <module 'py._path.svnwc' from '/home/ehudm/src/beam/sdks/python/target/.tox/py3-pytest/lib/python3.5/site-packages/py/_path/svnwc.py'>}
```
  • Loading branch information
udim committed Feb 27, 2019
1 parent ae243ad commit 56b5e38
Showing 1 changed file with 1 addition and 1 deletion.
2 changes: 1 addition & 1 deletion sdks/python/apache_beam/internal/pickler.py
Original file line number Diff line number Diff line change
Expand Up @@ -164,7 +164,7 @@ def new_save_module_dict(pickler, obj):
obj_id = id(obj)
if not known_module_dicts or '__file__' in obj or '__package__' in obj:
if obj_id not in known_module_dicts:
for m in sys.modules.values():
for m in list(sys.modules.values()):
try:
if (m
and m.__name__ != '__main__'
Expand Down

0 comments on commit 56b5e38

Please sign in to comment.