scan_module(): ValueError: bad marshal data (unknown type code)

[jwilk]

This happens with CPython from git master (e768c86ef442ef89004089a8a34ce5909ffb90f2):

Traceback (most recent call last):
  File "setup.py", line 16, in <module>
    install_requires=["appdirs"],
  File ".../python3.7/distutils/core.py", line 148, in setup
    dist.run_commands()
  File ".../python3.7/distutils/dist.py", line 966, in run_commands
    self.run_command(cmd)
  File ".../python3.7/distutils/dist.py", line 985, in run_command
    cmd_obj.run()
  File ".../python3.7/site-packages/setuptools/command/install.py", line 67, in run
    self.do_egg_install()
  File ".../python3.7/site-packages/setuptools/command/install.py", line 109, in do_egg_install
    self.run_command('bdist_egg')
  File ".../python3.7/distutils/cmd.py", line 313, in run_command
    self.distribution.run_command(command)
  File ".../python3.7/distutils/dist.py", line 985, in run_command
    cmd_obj.run()
  File ".../python3.7/site-packages/setuptools/command/bdist_egg.py", line 209, in run
    os.path.join(archive_root, 'EGG-INFO'), self.zip_safe()
  File ".../python3.7/site-packages/setuptools/command/bdist_egg.py", line 245, in zip_safe
    return analyze_egg(self.bdist_dir, self.stubs)
  File ".../python3.7/site-packages/setuptools/command/bdist_egg.py", line 355, in analyze_egg
    safe = scan_module(egg_dir, base, name, stubs) and safe
  File ".../python3.7/site-packages/setuptools/command/bdist_egg.py", line 392, in scan_module
    code = marshal.load(f)
ValueError: bad marshal data (unknown type code)

I believe this is because PEP 552 has been implemented, which adds extra 4 bytes to the .pyc header; but setuptools is not aware of that yet.

[benoit-pierre]

See #1263.

[jaraco]

Release on v38.4.1 pending...

[jenisys]

Same problem occurred w/ Python 3.7.0a4+ on Travis CI (reported as: travis-ci/travis-ci#9181), probably w/o the fix.

[smagafurov]

@jaraco Same problem
https://travis-ci.org/aio-libs/aiopg/jobs/341326946

[benoit-pierre]

@smagafurov : did you update setuptools?

[smagafurov]

setuptools (38.5.1)

[benoit-pierre]

@smagafurov: I'm sorry but I don't see the setuptools' update in your Travis log and the traceback line numbers don't match 38.5.1. Can you add a call to pip freeze --all before to make sure the right version is used? Just to be clear, you're not relying on setup_requires to update setuptools, right?

[smagafurov]

@benoit-pierre: I repeat this problem locally

$ python setup.py install
running install
running bdist_egg
running egg_info
writing aiopg.egg-info/PKG-INFO
writing dependency_links to aiopg.egg-info/dependency_links.txt
writing requirements to aiopg.egg-info/requires.txt
writing top-level names to aiopg.egg-info/top_level.txt
reading manifest file 'aiopg.egg-info/SOURCES.txt'
reading manifest template 'MANIFEST.in'
warning: no previously-included files found matching 'tests/**'
writing manifest file 'aiopg.egg-info/SOURCES.txt'
installing library code to build/bdist.macosx-10.13-x86_64/egg
running install_lib
running build_py
copying aiopg.egg-info/PKG-INFO -> build/bdist.macosx-10.13-x86_64/egg/EGG-INFO
copying aiopg.egg-info/SOURCES.txt -> build/bdist.macosx-10.13-x86_64/egg/EGG-INFO
copying aiopg.egg-info/dependency_links.txt -> build/bdist.macosx-10.13-x86_64/egg/EGG-INFO
copying aiopg.egg-info/requires.txt -> build/bdist.macosx-10.13-x86_64/egg/EGG-INFO
copying aiopg.egg-info/top_level.txt -> build/bdist.macosx-10.13-x86_64/egg/EGG-INFO
zip_safe flag not set; analyzing archive contents...
Traceback (most recent call last):
  File "setup.py", line 63, in <module>
    include_package_data=True)
  File "/Users/sergey/projects/aiopg/venv/lib/python3.7/site-packages/setuptools/__init__.py", line 129, in setup
    return distutils.core.setup(**attrs)
  File "/Users/sergey/.pyenv/versions/3.7.0a3/lib/python3.7/distutils/core.py", line 148, in setup
    dist.run_commands()
  File "/Users/sergey/.pyenv/versions/3.7.0a3/lib/python3.7/distutils/dist.py", line 966, in run_commands
    self.run_command(cmd)
  File "/Users/sergey/.pyenv/versions/3.7.0a3/lib/python3.7/distutils/dist.py", line 985, in run_command
    cmd_obj.run()
  File "/Users/sergey/projects/aiopg/venv/lib/python3.7/site-packages/setuptools/command/install.py", line 67, in run
    self.do_egg_install()
  File "/Users/sergey/projects/aiopg/venv/lib/python3.7/site-packages/setuptools/command/install.py", line 109, in do_egg_install
    self.run_command('bdist_egg')
  File "/Users/sergey/.pyenv/versions/3.7.0a3/lib/python3.7/distutils/cmd.py", line 313, in run_command
    self.distribution.run_command(command)
  File "/Users/sergey/.pyenv/versions/3.7.0a3/lib/python3.7/distutils/dist.py", line 985, in run_command
    cmd_obj.run()
  File "/Users/sergey/projects/aiopg/venv/lib/python3.7/site-packages/setuptools/command/bdist_egg.py", line 220, in run
    os.path.join(archive_root, 'EGG-INFO'), self.zip_safe()
  File "/Users/sergey/projects/aiopg/venv/lib/python3.7/site-packages/setuptools/command/bdist_egg.py", line 273, in zip_safe
    return analyze_egg(self.bdist_dir, self.stubs)
  File "/Users/sergey/projects/aiopg/venv/lib/python3.7/site-packages/setuptools/command/bdist_egg.py", line 383, in analyze_egg
    safe = scan_module(egg_dir, base, name, stubs) and safe
  File "/Users/sergey/projects/aiopg/venv/lib/python3.7/site-packages/setuptools/command/bdist_egg.py", line 422, in scan_module
    code = marshal.load(f)
ValueError: bad marshal data (unknown type code)
(venv) sergey@magafurov:~/projects/aiopg [master]
$ pip list | grep tool
DEPRECATION: The default format will switch to columns in the future. You can use --format=(legacy|columns) (or define a format=(legacy|columns) in your pip.conf under the [list] section) to disable this warning.
prompt-toolkit (1.0.15)
setuptools (38.5.1)

[smagafurov]

@benoit-pierre: How to repeat:

git clone git@github.com:aio-libs/aiopg.git
cd aiopg
python3.7 -m venv ./venv
source ./venv/bin/activate
sudo apt-get install libpq-dev
pip install -U setuptools
python setup.py install