Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

new version of setuptools_scm just broke my build #164

Closed
tardyp opened this issue Apr 7, 2017 · 18 comments
Closed

new version of setuptools_scm just broke my build #164

tardyp opened this issue Apr 7, 2017 · 18 comments
Assignees

Comments

@tardyp
Copy link

tardyp commented Apr 7, 2017

Please have a look at this build:
https://nine.buildbot.net/#/builders/2/builds/125578

looks like you just release setuptools_scm, so I would expect this is a problem with the new release..

Collecting lz4 (from buildbot===0.9.5-83-gad0d939)
  Downloading lz4-0.9.0.tar.gz (86kB)
    Complete output from command python setup.py egg_info:
    Traceback (most recent call last):
      File "<string>", line 1, in <module>
      File "/tmp/pip-build-ukoFIU/lz4/setup.py", line 160, in <module>
        'Programming Language :: Python :: 3.6',
      File "/usr/lib/python2.7/distutils/core.py", line 111, in setup
        _setup_distribution = dist = klass(attrs)
      File "/buildbot/buildbot-job/build/sandbox/local/lib/python2.7/site-packages/setuptools/dist.py", line 321, in __init__
        _Distribution.__init__(self, attrs)
      File "/usr/lib/python2.7/distutils/dist.py", line 287, in __init__
        self.finalize_options()
      File "/buildbot/buildbot-job/build/sandbox/local/lib/python2.7/site-packages/setuptools/dist.py", line 388, in finalize_options
        ep.load()(self, ep.name, value)
      File "build/bdist.linux-x86_64/egg/setuptools_scm/integration.py", line 21, in version_keyword
      File "build/bdist.linux-x86_64/egg/setuptools_scm/__init__.py", line 118, in get_version
      File "build/bdist.linux-x86_64/egg/setuptools_scm/__init__.py", line 96, in _do_parse
    LookupError: setuptools-scm was unable to detect version for '/tmp/pip-build-ukoFIU/lz4'.
    
    Make sure you're either building from a fully intact git repository or PyPI tarballs. Most other sources (such as GitHub's tarballs, a git checkout without the .git folder) don't contain the necessary metadata and will not work.
    
    For example, if you're using pip, instead of https://github.com/user/proj/archive/master.zip use git+https://github.com/user/proj.git#egg=proj

@RonnyPfannschmidt
Copy link
Contributor

this seems to be an artifact of uploading the sdist first and the bdist-egg second
setuptools and its sandboxing deliberately break the world there,

please rerun the build to see if its still happens, i strongly suggest including setuptools_scm into the build

im considering warning if setuptools_scm was egg/easy_installed

@joevgear
Copy link

joevgear commented Apr 7, 2017

We are seeing the same thing by just running: pip install -r requirements.txt. What is the workaround?

15:36:20 Complete output from command python setup.py egg_info:
15:36:20 Traceback (most recent call last):
15:36:20 File "", line 1, in
15:36:20 File "/tmp/pip-build-1c650j/pytest-xdist/setup.py", line 37, in
15:36:20 'Programming Language :: Python :: 3',
15:36:20 File "/opt/anaconda2/lib/python2.7/distutils/core.py", line 111, in setup
15:36:20 _setup_distribution = dist = klass(attrs)
15:36:20 File "/opt/anaconda2/lib/python2.7/site-packages/setuptools-23.0.0-py2.7.egg/setuptools/dist.py", line 272, in init
15:36:20 File "/opt/anaconda2/lib/python2.7/distutils/dist.py", line 287, in init
15:36:20 self.finalize_options()
15:36:20 File "/opt/anaconda2/lib/python2.7/site-packages/setuptools-23.0.0-py2.7.egg/setuptools/dist.py", line 327, in finalize_options
15:36:20 File "build/bdist.linux-x86_64/egg/setuptools_scm/integration.py", line 21, in version_keyword
15:36:20 File "build/bdist.linux-x86_64/egg/setuptools_scm/init.py", line 118, in get_version
15:36:20 File "build/bdist.linux-x86_64/egg/setuptools_scm/init.py", line 96, in _do_parse
15:36:20 LookupError: setuptools-scm was unable to detect version for '/tmp/pip-build-1c650j/pytest-xdist'.
15:36:20

@mkolodny
Copy link

mkolodny commented Apr 8, 2017

We're seeing the same thing:

Collecting pytest-xdist>=1.15.0
  Downloading https://company.com/pypi/packages/dc/b2/a59db3ef996b93c3ef35dbbc33557a71ef67a6839d94a52c88eeb1086002/pytest-xdist-1.15.0.tar.gz (87kB)
    100% |████████████████████████████████| 92kB 70.2MB/s
    Complete output from command python setup.py egg_info:
    Traceback (most recent call last):
      File "<string>", line 1, in <module>
      File "/tmp/pip-build-vyEgO5/pytest-xdist/setup.py", line 37, in <module>
        'Programming Language :: Python :: 3',
      File "/usr/lib/python2.7/distutils/core.py", line 111, in setup
        _setup_distribution = dist = klass(attrs)
      File "/home/company/package_name/env/local/lib/python2.7/site-packages/setuptools/dist.py", line 318, in __init__
        _Distribution.__init__(self, attrs)
      File "/usr/lib/python2.7/distutils/dist.py", line 287, in __init__
        self.finalize_options()
      File "/home/company/package_name/env/local/lib/python2.7/site-packages/setuptools/dist.py", line 376, in finalize_options
        ep.load()(self, ep.name, value)
      File "build/bdist.linux-x86_64/egg/setuptools_scm/integration.py", line 21, in version_keyword
      File "build/bdist.linux-x86_64/egg/setuptools_scm/__init__.py", line 118, in get_version
      File "build/bdist.linux-x86_64/egg/setuptools_scm/__init__.py", line 96, in _do_parse
    LookupError: setuptools-scm was unable to detect version for '/tmp/pip-build-vyEgO5/pytest-xdist'.

    Make sure you're either building from a fully intact git repository or PyPI tarballs. Most other sources (such as GitHub's tarballs, a git checkout without the .git folder) don't contain the necessary metadata and will not work.

    For example, if you're using pip, instead of https://github.com/user/proj/archive/master.zip use git+https://github.com/user/proj.git#egg=proj

    ----------------------------------------
Command "python setup.py egg_info" failed with error code 1 in /tmp/pip-build-vyEgO5/pytest-xdist/

@dstufft
Copy link
Member

dstufft commented Apr 8, 2017

I'm hitting this too, package only ever had a sdist created for it:

Collecting pypi-theme==1.37 (from -r requirements/theme.txt (line 1))
  Downloading https://.../pypi-theme-1.37.0.tar.gz (131kB)
  Requested pypi-theme==1.37 from https://.../pypi-theme-1.37.0.tar.gz (from -r requirements/theme.txt (line 1)), but installing version 1.None.0
Installing collected packages: pypi-theme
  Found existing installation: pypi-theme 1.None.0
    Uninstalling pypi-theme-1.None.0:
      Successfully uninstalled pypi-theme-1.None.0
  Running setup.py install for pypi-theme: started
    Running setup.py install for pypi-theme: finished with status 'error'
    Complete output from command /app/.heroku/python/bin/python -u -c "import setuptools, tokenize;__file__='/tmp/pip-build-zg8aipv0/pypi-theme/setup.py';f=getattr(tokenize, 'open', open)(__file__);code=f.read().replace('\r\n', '\n');f.close();exec(compile(code, __file__, 'exec'))" install --record /tmp/pip-h4pe6ocr-record/install-record.txt --single-version-externally-managed --compile:
    Traceback (most recent call last):
      File "<string>", line 1, in <module>
      File "/tmp/pip-build-zg8aipv0/pypi-theme/setup.py", line 24, in <module>
        setup_requires=["setuptools_scm"],
      File "/app/.heroku/python/lib/python3.5/distutils/core.py", line 108, in setup
        _setup_distribution = dist = klass(attrs)
      File "/app/.heroku/python/lib/python3.5/site-packages/setuptools/dist.py", line 321, in __init__
        _Distribution.__init__(self, attrs)
      File "/app/.heroku/python/lib/python3.5/distutils/dist.py", line 281, in __init__
        self.finalize_options()
      File "/app/.heroku/python/lib/python3.5/site-packages/setuptools/dist.py", line 390, in finalize_options
        ep.load()(self, ep.name, value)
      File "/tmp/pip-build-zg8aipv0/pypi-theme/.eggs/setuptools_scm-1.15.3-py3.5.egg/setuptools_scm/integration.py", line 21, in version_keyword
      File "/tmp/pip-build-zg8aipv0/pypi-theme/.eggs/setuptools_scm-1.15.3-py3.5.egg/setuptools_scm/__init__.py", line 118, in get_version
      File "/tmp/pip-build-zg8aipv0/pypi-theme/.eggs/setuptools_scm-1.15.3-py3.5.egg/setuptools_scm/__init__.py", line 83, in _do_parse
      File "/tmp/pip-build-zg8aipv0/pypi-theme/.eggs/setuptools_scm-1.15.3-py3.5.egg/setuptools_scm/__init__.py", line 37, in _version_from_entrypoint
      File "/tmp/pip-build-zg8aipv0/pypi-theme/.eggs/setuptools_scm-1.15.3-py3.5.egg/setuptools_scm/hacks.py", line 24, in parse_pip_egg_info
      File "/tmp/pip-build-zg8aipv0/pypi-theme/.eggs/setuptools_scm-1.15.3-py3.5.egg/setuptools_scm/hacks.py", line 13, in parse_pkginfo
      File "/tmp/pip-build-zg8aipv0/pypi-theme/.eggs/setuptools_scm-1.15.3-py3.5.egg/setuptools_scm/version.py", line 92, in meta
    AssertionError: cant parse version None
    
    ----------------------------------------
  Rolling back uninstall of pypi-theme
Command "/app/.heroku/python/bin/python -u -c "import setuptools, tokenize;__file__='/tmp/pip-build-zg8aipv0/pypi-theme/setup.py';f=getattr(tokenize, 'open', open)(__file__);code=f.read().replace('\r\n', '\n');f.close();exec(compile(code, __file__, 'exec'))" install --record /tmp/pip-h4pe6ocr-record/install-record.txt --single-version-externally-managed --compile" failed with error code 1 in /tmp/pip-build-zg8aipv0/pypi-theme/

@tardyp
Copy link
Author

tardyp commented Apr 8, 2017

@RonnyPfannschmidt this happen for all the 15 builds of my build matrix: https://nine.buildbot.net/#/builders/1/builds/1707
So this is reproducible easily. I can install setuptools_scm before the build like I did for pbr, but maybe there is a fix to include in setuptools_scm or something.

The thing is:

  • my build worked yesterday
  • new version of setuptools_scm was released
  • my build does not work anymore

Buildbot does not use setuptools_scm, but it does use lz4, which in turn uses setuptools_scm. I'd expect a couple of other projects to be in the same position.

Could you please re-release a -p1 package bdist-egg first?
If I understand your answer correctly, this should fix the issue.

tardyp added a commit to tardyp/buildbot that referenced this issue Apr 8, 2017
@RonnyPfannschmidt
Copy link
Contributor

@tardyp it seems i misinterpreted the build log, i need to investigate why the egg in your case seems to have an issue (because the setuptools_scm eggs) are build precisely to avoid such issues

since your issue happens i have to presume that setuptools_scm no longer correctly works for sdists
i will investigate and try to release a fix within the next 2 hours else i'll pull the full release as broken by a regression

@RonnyPfannschmidt
Copy link
Contributor

i can replicate the problems with

$ nix-shell -p 'python.withPackages (ps: with ps; [setuptools pip])'
$ pip download lz4
Collecting lz4
  Downloading lz4-0.9.0.tar.gz (86kB)
    100% |████████████████████████████████| 92kB 1.2MB/s 
  Saved ./lz4-0.9.0.tar.gz
    Complete output from command python setup.py egg_info:
    Traceback (most recent call last):
      File "<string>", line 1, in <module>
      File "/run/user/1000/pip-build-12l0ex/lz4/setup.py", line 160, in <module>
        'Programming Language :: Python :: 3.6',
      File "/nix/store/sc7zay1kxpjcpiw7ykgivq3ca68vjqqf-python-2.7.13-env/lib/python2.7/distutils/core.py", line 111, in setup
        _setup_distribution = dist = klass(attrs)
      File "/nix/store/sc7zay1kxpjcpiw7ykgivq3ca68vjqqf-python-2.7.13-env/lib/python2.7/site-packages/setuptools-30.2.0-py2.7.egg/setuptools/dist.py", line 318, in __init__
      File "/nix/store/sc7zay1kxpjcpiw7ykgivq3ca68vjqqf-python-2.7.13-env/lib/python2.7/distutils/dist.py", line 287, in __init__
        self.finalize_options()
      File "/nix/store/sc7zay1kxpjcpiw7ykgivq3ca68vjqqf-python-2.7.13-env/lib/python2.7/site-packages/setuptools-30.2.0-py2.7.egg/setuptools/dist.py", line 376, in finalize_options
      File "/run/user/1000/pip-build-12l0ex/lz4/.eggs/setuptools_scm-1.15.3-py2.7.egg/setuptools_scm/integration.py", line 21, in version_keyword
      File "/run/user/1000/pip-build-12l0ex/lz4/.eggs/setuptools_scm-1.15.3-py2.7.egg/setuptools_scm/__init__.py", line 118, in get_version
      File "/run/user/1000/pip-build-12l0ex/lz4/.eggs/setuptools_scm-1.15.3-py2.7.egg/setuptools_scm/__init__.py", line 96, in _do_parse
    LookupError: setuptools-scm was unable to detect version for '/run/user/1000/pip-build-12l0ex/lz4'.
                                                                                                                                                                                                                                                                                                                             
    Make sure you're either building from a fully intact git repository or PyPI tarballs. Most other sources (such as GitHub's tarballs, a git checkout without the .git folder) don't contain the necessary metadata and will not work.                                                                                     
                                                                                                                                                                                                                                                                                                                             
    For example, if you're using pip, instead of https://github.com/user/proj/archive/master.zip use git+https://github.com/user/proj.git#egg=proj                                                                                                                                                                           
                                                                                                                                                                                                                                                                                                                             
    ----------------------------------------                                                                                                                                                                                                                                                                                 
Command "python setup.py egg_info" failed with error code 1 in /run/user/1000/pip-build-12l0ex/lz4/  

@RonnyPfannschmidt
Copy link
Contributor

the issue was introduced in the fix for #138 - add support for the pip-egg-info folder #148

RonnyPfannschmidt added a commit to RonnyPfannschmidt/setuptools_scm that referenced this issue Apr 8, 2017
@tardyp
Copy link
Author

tardyp commented Apr 8, 2017

@RonnyPfannschmidt thanks for working on it.

Reverting to 1.15.0 did fix the issue:
https://github.com/buildbot/buildbot/pull/3105/files
I'd rather not merge this PR, and wait for a proper fix from you.

@tardyp
Copy link
Author

tardyp commented Apr 8, 2017

updating my PR to pull git+https://github.com/pypa/setuptools_scm to see if your fix works for me.

@RonnyPfannschmidt
Copy link
Contributor

@tardyp a release is upcoming after #165 passes, master isn't fixed until its merged

@RonnyPfannschmidt
Copy link
Contributor

@tardyp master is fixed now - please tell me if you prefer i release, or if you would run against it first to verify

@tardyp
Copy link
Author

tardyp commented Apr 8, 2017

looks like installing from git+https will not work

Collecting git+https://github.com/pypa/setuptools_scm
  Cloning https://github.com/pypa/setuptools_scm to /tmp/pip-Qh8VgW-build
Collecting pbr
  Downloading pbr-2.0.0-py2.py3-none-any.whl (98kB)
Installing collected packages: pbr, setuptools-scm
  Running setup.py install for setuptools-scm: started
    Running setup.py install for setuptools-scm: finished with status 'error'
    Complete output from command /buildbot/buildbot-job/build/sandbox/bin/python2.7 -u -c "import setuptools, tokenize;__file__='/tmp/pip-Qh8VgW-build/setup.py';f=getattr(tokenize, 'open', open)(__file__);code=f.read().replace('\r\n', '\n');f.close();exec(compile(code, __file__, 'exec'))" install --record /tmp/pip-vFTFZP-record/install-record.txt --single-version-externally-managed --compile --install-headers /buildbot/buildbot-job/build/sandbox/include/site/python2.7/setuptools-scm:
    important note:
    
    the setup of setuptools_scm is self-using,
    the first execution of `python setup.py egg_info`
    will generate partial data
    its critical to run `python setup.py egg_info`
    once before running sdist or easy_install on a fresh checkouts
    
    pip usage is recommended
    
    please run `python setup.py egg_info` first

Do you have another suggestion for testing it before release?
you may release an rc1 so that I can try with pip install --pre ?

@RonnyPfannschmidt
Copy link
Contributor

im preparing something

RonnyPfannschmidt added a commit to RonnyPfannschmidt/setuptools_scm that referenced this issue Apr 8, 2017
fixes an interaction issue discovered in pypa#164
@RonnyPfannschmidt
Copy link
Contributor

@tardyp please rerun your build, i update setuptools_scm to work from pip install via git

@RonnyPfannschmidt
Copy link
Contributor

ps, i removed the 1.15.3 release from pypi

ikalnytskyi added a commit to sphinx-contrib/redoc that referenced this issue Apr 8, 2017
There were a temporary issue in setuptools_scm 1.15.3 [1] that broke the
world. Fortunately, it has been fixed in 1.15.4 so we are allowed to use
latest version once again.

[1] pypa/setuptools-scm#164
matthew-brett added a commit to MacPython/matplotlib-wheels that referenced this issue Apr 8, 2017
openstack-gerrit pushed a commit to openstack/rally that referenced this issue Apr 9, 2017
Due to pypa/setuptools-scm#164 we are unable
to install pytest-xdist

Change-Id: I52da4b071a1c57270e7dfbbe981a8eeeb839aba1
openstack-gerrit pushed a commit to openstack/rally that referenced this issue Apr 10, 2017
Due to pypa/setuptools-scm#164 we are unable
to install pytest-xdist

Change-Id: I52da4b071a1c57270e7dfbbe981a8eeeb839aba1
(cherry picked from commit 2d594c2)
@twall
Copy link

twall commented May 21, 2018

I see a similar error using a version like 3.3.1-rc25 (fails) versus 3.3.1.rc25. Reverting to 1.15.0 fixes the error.

Note that when 3.3.1.rc25 is used, setuptools_scm transforms the version into 3.3.3rc25, which is unexpected.

looking for ep setuptools_scm.parse_scm_fallback .
root '/Users/twall/plex/rd'
looking for ep setuptools_scm.parse_scm /Users/twall/plex/rd
found ep .git = setuptools_scm.git:parse
cmd 'git rev-parse --show-toplevel'
out b'/Users/twall/plex/rd\n'
real root /Users/twall/plex/rd
cmd 'git describe --dirty --tags --long --match *.*'
out b'3.3.1-rc25-0-g0e5349e\n'
cmd 'git rev-parse --abbrev-ref HEAD'
out b'pyplex\n'
tag 3.3.1-rc25
version <LegacyVersion('rc25')>
version None
Traceback (most recent call last):
  File "setup.py", line 43, in <module>
    entry_points = {
  File "/Users/twall/plex/rd/search/.venv-3.6/lib/python3.6/site-packages/setuptools/__init__.py", line 129, in setup
    return distutils.core.setup(**attrs)
  File "/Users/twall/plex/rd/search/.venv-3.6/lib/python3.6/distutils/core.py", line 108, in setup
    _setup_distribution = dist = klass(attrs)
  File "/Users/twall/plex/rd/search/.venv-3.6/lib/python3.6/site-packages/setuptools/dist.py", line 370, in __init__
    k: v for k, v in attrs.items()
  File "/Users/twall/plex/rd/search/.venv-3.6/lib/python3.6/distutils/dist.py", line 281, in __init__
    self.finalize_options()
  File "/Users/twall/plex/rd/search/.venv-3.6/lib/python3.6/site-packages/setuptools/dist.py", line 529, in finalize_options
    ep.load()(self, ep.name, value)
  File "/Users/twall/plex/rd/search/.eggs/setuptools_scm-2.1.0-py3.6.egg/setuptools_scm/integration.py", line 22, in version_keyword
    dist.metadata.version = get_version(**value)
  File "/Users/twall/plex/rd/search/.eggs/setuptools_scm-2.1.0-py3.6.egg/setuptools_scm/__init__.py", line 119, in get_version
    parsed_version = _do_parse(root, parse)
  File "/Users/twall/plex/rd/search/.eggs/setuptools_scm-2.1.0-py3.6.egg/setuptools_scm/__init__.py", line 83, in _do_parse
    version = version_from_scm(root) or _version_from_entrypoint(
  File "/Users/twall/plex/rd/search/.eggs/setuptools_scm-2.1.0-py3.6.egg/setuptools_scm/__init__.py", line 31, in version_from_scm
    return _version_from_entrypoint(root, 'setuptools_scm.parse_scm')
  File "/Users/twall/plex/rd/search/.eggs/setuptools_scm-2.1.0-py3.6.egg/setuptools_scm/__init__.py", line 36, in _version_from_entrypoint
    version = ep.load()(root)
  File "/Users/twall/plex/rd/search/.eggs/setuptools_scm-2.1.0-py3.6.egg/setuptools_scm/git.py", line 128, in parse
    return meta(tag, node=node, dirty=dirty, branch=branch)
  File "/Users/twall/plex/rd/search/.eggs/setuptools_scm-2.1.0-py3.6.egg/setuptools_scm/version.py", line 135, in meta
    assert tag is not None, 'cant parse version %s' % tag
AssertionError: cant parse version None

@RonnyPfannschmidt
Copy link
Contributor

@twall thats a new issue please open a new one

tcuthbert pushed a commit to tcuthbert/jenkins-charm that referenced this issue Mar 22, 2023
- pin setuptools to an earlier version for < py36 (xenial)
  ```
  File Finder Failed for .git = setuptools_scm.git:list_files_in_archive
  ```
- preinstalling basic layer dependencies to avoid chicken and egg issues
  deploying on xenial

This [related](pypa/setuptools-scm#164)
issue provides more context.
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
Development

No branches or pull requests

6 participants