# 20.5.0+ breaks existing setup.py files - UndefinedEnvironmentName #523

Closed
opened this Issue Mar 29, 2016 · 50 comments

Projects
None yet

### lnielsen commented Mar 29, 2016

 The latest 20.6.4 PyPI is breaking our setup.py files with the following exception: Traceback (most recent call last): File "setup.py", line 163, in cmdclass={'test': PyTest}, File "/opt/python/2.7.9/lib/python2.7/distutils/core.py", line 151, in setup dist.run_commands() File "/opt/python/2.7.9/lib/python2.7/distutils/dist.py", line 953, in run_commands self.run_command(cmd) File "/opt/python/2.7.9/lib/python2.7/distutils/dist.py", line 972, in run_command cmd_obj.run() File "/home/travis/virtualenv/python2.7.9/lib/python2.7/site-packages/setuptools/command/test.py", line 152, in run self.distribution.fetch_build_eggs(self.distribution.tests_require) File "/home/travis/virtualenv/python2.7.9/lib/python2.7/site-packages/setuptools/dist.py", line 313, in fetch_build_eggs replace_conflicting=True, File "/home/travis/virtualenv/python2.7.9/lib/python2.7/site-packages/pkg_resources/__init__.py", line 814, in resolve if req.marker and not req.marker.evaluate(): File "/home/travis/virtualenv/python2.7.9/lib/python2.7/site-packages/pkg_resources/_vendor/packaging/markers.py", line 273, in evaluate return _evaluate_markers(self._markers, current_environment) File "/home/travis/virtualenv/python2.7.9/lib/python2.7/site-packages/pkg_resources/_vendor/packaging/markers.py", line 198, in _evaluate_markers lhs_value = _get_env(environment, lhs.value) File "/home/travis/virtualenv/python2.7.9/lib/python2.7/site-packages/pkg_resources/_vendor/packaging/markers.py", line 180, in _get_env "{0!r} does not exist in evaluation environment.".format(name) pkg_resources._vendor.packaging.markers.UndefinedEnvironmentName: 'extra' does not exist in evaluation environment. 

### lnielsen commented Mar 29, 2016

 Seems to be related to this commit d25e2a5
Member

### jaraco commented Mar 29, 2016

 @s-t-e-v-e-n-k Can you look into this one? @lnielsen How severe is the failure? Should I pull 20.5+ from PyPI?

### lnielsen commented Mar 29, 2016

 Looks pretty serve. Just upgrading setuptools in a virtualenv breaks the code.

Member

### jaraco commented Mar 29, 2016

 I've removed 20.6.4 and 20.5.0 until the issue can be resolved.
Contributor

### msabramo commented Mar 29, 2016

 Also seeing this.
Contributor

### msabramo commented Mar 29, 2016

 My colleague has been looking into this and he thinks that it has to do with the fact that our tests are using a variable/parameter name called extra_environ. This is a parameter that is used by webtest (e.g.: see http://docs.pylonsproject.org/projects/webtest/en/latest/testapp.html#modifying-the-environment-simulating-authentication)

### bszonye commented Mar 29, 2016

 I am seeing this in code that has variables and parameters named extra_environ that are ultimately passed to the webtest API. Renaming to extra_env_vars and extra_evars still hit the problem, but renaming to other_environ fixed it, so it looks like something is incorrectly identifying the extra_ pattern as an environment marker.
Contributor

### nakato commented Mar 30, 2016

 @lnielsen I can't reproduce this. How are you getting that traceback? What commands are you executing? What is the logging leading up to it? Is there anything that follows it?
Member

### jaraco commented Mar 31, 2016

 I can't replicate this either. I inferred from the link that the command being executed is python setup.py test in the invenio-records-rest project, so I checked out that repo and ran setup.py test under Python 2.7 and 3.5 using setuptools 20.5+, and while some tests failed, the reported error did not occur. I sense now that the failures are fairly isolated and only affecting tests, so I'm going to re-release the code in the latest version, but I am eager to see a fix for this issue, so do please provide more detail to describe how one can recreate the issue, preferably with a minimal use case.

### bszonye commented Mar 31, 2016

 Our teams are encountering this when using the webtest API in a py.test fixture, so it affects all of their tests. I do not yet have a workaround that successfully builds against the webtest classes without triggering the setuptools bug, so those teams cannot even run commit tests to integrate new code.
Member

### jaraco commented Mar 31, 2016

 @bszonye Sorry. I don't mean to cause you trouble. Can you pin your environment to Setuptools 20.4 until the issue can be resolved?

### bszonye commented Mar 31, 2016

 That’s what I recommended to the team for now, to pin setuptools<20.5 – tomorrow I will take a closer look to see if I can find a minimal test case.

### bszonye commented Mar 31, 2016

 @jaraco Just confirmed with the team that version 20.6.6 does break their tests again, but they are able to work around by pinning to the 20.4 version. I suspect that something in packaging is misidentifying extra markers from code that happens to contain the string in certain identifiers. Our traceback ends at the same place but starts with py.test instead of setup.py test: [tests.python.functional.test_XXX.TestXXX.test_XXX] tests/python/functional/utils.py:44: in setUp self.testapp = get_webtest_testapp() tests/python/functional/utils.py:98: in get_webtest_testapp app = get_app(join(conf_dir, 'test.ini')) .tox/py27/local/lib/python2.7/site-packages/pyramid/paster.py:31: in get_app global_conf=options) .tox/py27/local/lib/python2.7/site-packages/paste/deploy/loadwsgi.py:247: in loadapp return loadobj(APP, uri, name=name, **kw) .tox/py27/local/lib/python2.7/site-packages/paste/deploy/loadwsgi.py:271: in loadobj global_conf=global_conf) .tox/py27/local/lib/python2.7/site-packages/paste/deploy/loadwsgi.py:296: in loadcontext global_conf=global_conf) .tox/py27/local/lib/python2.7/site-packages/paste/deploy/loadwsgi.py:320: in _loadconfig return loader.get_context(object_type, name, global_conf) .tox/py27/local/lib/python2.7/site-packages/paste/deploy/loadwsgi.py:450: in get_context global_additions=global_additions) .tox/py27/local/lib/python2.7/site-packages/paste/deploy/loadwsgi.py:559: in _pipeline_app_context APP, pipeline[-1], global_conf) .tox/py27/local/lib/python2.7/site-packages/paste/deploy/loadwsgi.py:454: in get_context section) .tox/py27/local/lib/python2.7/site-packages/paste/deploy/loadwsgi.py:476: in _context_from_use object_type, name=use, global_conf=global_conf) .tox/py27/local/lib/python2.7/site-packages/paste/deploy/loadwsgi.py:406: in get_context global_conf=global_conf) .tox/py27/local/lib/python2.7/site-packages/paste/deploy/loadwsgi.py:296: in loadcontext global_conf=global_conf) .tox/py27/local/lib/python2.7/site-packages/paste/deploy/loadwsgi.py:328: in _loadegg return loader.get_context(object_type, name, global_conf) .tox/py27/local/lib/python2.7/site-packages/paste/deploy/loadwsgi.py:620: in get_context object_type, name=name) .tox/py27/local/lib/python2.7/site-packages/paste/deploy/loadwsgi.py:640: in find_egg_entry_point pkg_resources.require(self.spec) .tox/py27/local/lib/python2.7/site-packages/pkg_resources/__init__.py:947: in require needed = self.resolve(parse_requirements(requirements)) .tox/py27/local/lib/python2.7/site-packages/pkg_resources/__init__.py:814: in resolve if req.marker and not req.marker.evaluate(): .tox/py27/local/lib/python2.7/site-packages/pkg_resources/_vendor/packaging/markers.py:278: in evaluate return _evaluate_markers(self._markers, current_environment) .tox/py27/local/lib/python2.7/site-packages/pkg_resources/_vendor/packaging/markers.py:203: in _evaluate_markers lhs_value = _get_env(environment, lhs.value) .tox/py27/local/lib/python2.7/site-packages/pkg_resources/_vendor/packaging/markers.py:185: in _get_env "{0!r} does not exist in evaluation environment.".format(name) E UndefinedEnvironmentName: 'extra' does not exist in evaluation environment. 
Contributor

### s-t-e-v-e-n-k commented Mar 31, 2016

 @bszonye I'd like to see which requirement is causing this issue, are you able to edit the virtual env's copy of local/lib/python2.7/site-packages/pkg_resources/init.py to log what 'req' is set to just before the call to req.marker.evaluate()? I'd personally use the q module.

### warner commented Mar 31, 2016

 This is biting us too, on today's (brand-new) release of tahoe-lafs. To reproduce: pip install tahoe-lafs, then tahoe --version. If I add a print req just before the req.marker.evaluate() call at pkg_resources/__init__.py line 814, then when I run our application, I see: pyOpenSSL>=0.14 Nevow>=0.11.1 Twisted[tls]>=15.1.0 pyasn1-modules>=0.0.5 pyasn1>=0.1.8 characteristic>=14.0.0 service-identity pycryptopp>=0.6.0 pycrypto!=2.2,!=2.4,>=2.1.0 foolscap>=0.10.1 zope.interface!=3.6.3,!=3.6.4,>=3.6.0 simplejson>=1.4 zfec>=1.1.0 setuptools>=11.3 six>=1.5.2 cryptography>=1.3 twisted>=13.0 pyopenssl>=0.13; extra == "tls"  followed by the same traceback as above. Tahoe depends, among most of those other things, on twisted[tls] >= 15.1.0 and pyopenssl >= 0.14. (See https://tahoe-lafs.org/~warner/deps.png for a graph of the dependencies, created by https://github.com/tahoe-lafs/tahoe-lafs/blob/master/misc/coding_tools/graph-deps.py). Nevow wants twisted >= 0.13, so I'm guessing that's where the next-to-last req comes from. Twisted[tls] wants pyopenssl >= 0.13. Is it normal that the req should have "extra == tls" on the pyopenssl line, when the extra is associated with Twisted? Grepping through our code, we do have functions and variables with extra_ in them, in case it matters. Please let me know if there's anything I can do to help track this one down.

### warner commented Mar 31, 2016

 I don't know this code very well, but I think the requirement that it's tripping on is this one from Twisted's dist-info/METADATA file: Requires-Dist: pyopenssl (>=0.13); extra == 'tls'  and I think the req.marker.evaluate() call in the traceback is treating this extra=='tls' as a marker (e.g. python_version>=2.7), rather than as an extra. I know the line-based METADATA format necessarily merges extras and markers into the same syntax. Is this causing confusion? I see calls to e.g. req.marker.evaluate({'extra': extra}) elsewhere, and it seems like adding that kind of argument to our evaluate() call would help. I see req.extras is available, but I'm not sure how to deal with the fact that it's a list. Call it once with everything in req.extras and continue if any of them say True? Or if none of them say False?
Contributor

### s-t-e-v-e-n-k commented Mar 31, 2016

 extra == 'tls' is a valid environment marker. But yes, you're totally right, the evaluate() call needs to respect extras, which I'll be looking at tomorrow.
Contributor

### rbtcollins commented Mar 31, 2016

 For clarity, extra=='tls' is not a valid marker for setup.py scripts. Its valid in the syntax for distribution metadata only. Callers of the Python API must supply a extra mapping when evaluating requirements specifications if they are processing distribution metadata entries (like the one above) . tl;dr: pkg_resources, when processing METADATA files can supply a value for extra. It may be wired up wrongly at the moment. extra can't be used from setup_requires or other such places though - its a gnarly thing in the distirbution metadata spec, not a user interface element. I'll look more closely at the actual calling code tomorrow - this is mainly meant to hint to folk that have already been looking at it.
Contributor

### rbtcollins commented Mar 31, 2016

 Oh, and - 23:19 < warner> we've got a requirement on "twisted[tls]", and twisted"s got an extra that says [tls] needs pyopenssl 23:19 < warner> and the exception happens when it tries to do req.marker.evaluate() on a req for pyopenssl and a marker for "extra = 'tls'" 23:21 < lifeless> yeah 23:22 < lifeless> I think the problem here is that something has started reflecting markers back into requiremnets 23:22 < lifeless> but we can only do that for a subset of markers 23:22 < lifeless> the extra mapping needs to be translated when its mapped back from IRC. HTH

Merged

### autopulated commented Mar 31, 2016

 Also just tripped over this, to reproduce: pip install requests[security] python -c "import pkg_resources; pkg_resources.require('requests[security]')" Reverting back to an older setuptools with pip install setuptools==20.4 works.
Member

### jaraco commented Mar 31, 2016

 Thanks @autopulated - I was able to reproduce the issue with those commands. This definitely is a widespread issue. Rather than pull the latest releases, I'm going to disable this new functionality until a solution can be implemented.

### robdennis commented Mar 31, 2016

 I'm seeing this also when running a console entry point: leaving this stack trace in case it helps anyone. I got it on both versions 20.5 and 20.6.6 (we don't pin setuptools versions) I happened when executing an entry point, here's the setup.py definition:  entry_points={'console_scripts': ['redacted_name = redacted_name.main:main'], },  and the stack trace: File "/opt/redacted_name/bin/redacted_name", line 8, in from pkg_resources import load_entry_point File "/opt/redacted_name/lib/python2.7/site-packages/pkg_resources/__init__.py", line 2905, in @_call_aside File "/opt/redacted_name/lib/python2.7/site-packages/pkg_resources/__init__.py", line 2891, in _call_aside f(*args, **kwargs) File "/opt/redacted_name/lib/python2.7/site-packages/pkg_resources/__init__.py", line 2918, in _initialize_master_working_set working_set = WorkingSet._build_master() File "/opt/redacted_name/lib/python2.7/site-packages/pkg_resources/__init__.py", line 642, in _build_master ws.require(__requires__) File "/opt/redacted_name/lib/python2.7/site-packages/pkg_resources/__init__.py", line 947, in require needed = self.resolve(parse_requirements(requirements)) File "/opt/redacted_name/lib/python2.7/site-packages/pkg_resources/__init__.py", line 814, in resolve if req.marker and not req.marker.evaluate(): File "/opt/redacted_name/lib/python2.7/site-packages/pkg_resources/_vendor/packaging/markers.py", line 273, in evaluate return _evaluate_markers(self._markers, current_environment) File "/opt/yamato/lib/python2.7/site-packages/pkg_resources/_vendor/packaging/markers.py", line 198, in _evaluate_markers lhs_value = _get_env(environment, lhs.value) File "/opt/redacted_name/lib/python2.7/site-packages/pkg_resources/_vendor/packaging/markers.py", line 180, in _get_env "{0!r} does not exist in evaluation environment.".format(name) pkg_resources._vendor.packaging.markers.UndefinedEnvironmentName: 'extra' does not exist in evaluation environment. 

### jaraco added a commit that referenced this issue Mar 31, 2016

 Bypass environment marker evaluation in requirements resolution. Ref #… 
…523.
 04d10ff 
Member

### jaraco commented Mar 31, 2016

 v20.6.7 is released bypassing the issue.

Closed

### bszonye commented Mar 31, 2016

 Thanks for the troubleshooting tips, @s-t-e-v-e-n-k – I’ll see which requirement is causing the problem for us. I now suspect that it’s something dynamically loaded in Pyramid, and the extra_environ identifiers are a red herring.

Open

Closed

Merged

### gitrookie commented May 17, 2016

 I am getting this same issue. pip install scikit-image Collecting scikit-image Using cached scikit_image-0.12.3-cp35-cp35m-manylinux1_x86_64.whl Requirement already satisfied (use --upgrade to upgrade): pillow>=2.1.0 in /usr/lib/python3/dist-packages (from scikit-image) Requirement already satisfied (use --upgrade to upgrade): six>=1.7.3 in /usr/lib/python3/dist-packages (from scikit-image) Collecting dask[array]>=0.5.0 (from scikit-image) Using cached dask-0.9.0-py2.py3-none-any.whl Collecting networkx>=1.8 (from scikit-image) Using cached networkx-1.11-py2.py3-none-any.whl Requirement already satisfied (use --upgrade to upgrade): numpy; extra == "array" in /usr/lib/python3/dist-packages (from dask[array]>=0.5.0->scikit-image) Exception: Traceback (most recent call last): File "/usr/lib/python3/dist-packages/pip/basecommand.py", line 209, in main status = self.run(options, args) File "/usr/lib/python3/dist-packages/pip/commands/install.py", line 328, in run wb.build(autobuilding=True) File "/usr/lib/python3/dist-packages/pip/wheel.py", line 748, in build self.requirement_set.prepare_files(self.finder) File "/usr/lib/python3/dist-packages/pip/req/req_set.py", line 360, in prepare_files ignore_dependencies=self.ignore_dependencies)) File "/usr/lib/python3/dist-packages/pip/req/req_set.py", line 448, in _prepare_file req_to_install, finder) File "/usr/lib/python3/dist-packages/pip/req/req_set.py", line 387, in _check_skip_installed req_to_install.check_if_exists() File "/usr/lib/python3/dist-packages/pip/req/req_install.py", line 997, in check_if_exists self.satisfied_by = pkg_resources.get_distribution(self.req) File "/home/gaurav/.virtualEnv/py3.5/share/python-wheels/pkg_resources-0.0.0-py2.py3-none-any.whl/pkg_resources/__init__.py", line 535, in get_distribution dist = get_provider(dist) File "/home/gaurav/.virtualEnv/py3.5/share/python-wheels/pkg_resources-0.0.0-py2.py3-none-any.whl/pkg_resources/__init__.py", line 415, in get_provider return working_set.find(moduleOrReq) or require(str(moduleOrReq))[0] File "/home/gaurav/.virtualEnv/py3.5/share/python-wheels/pkg_resources-0.0.0-py2.py3-none-any.whl/pkg_resources/__init__.py", line 943, in require needed = self.resolve(parse_requirements(requirements)) File "/home/gaurav/.virtualEnv/py3.5/share/python-wheels/pkg_resources-0.0.0-py2.py3-none-any.whl/pkg_resources/__init__.py", line 808, in resolve if not req_extras.markers_pass(req): File "/home/gaurav/.virtualEnv/py3.5/share/python-wheels/pkg_resources-0.0.0-py2.py3-none-any.whl/pkg_resources/__init__.py", line 993, in markers_pass return not req.marker or any(extra_evals) or req.marker.evaluate() File "/home/gaurav/.virtualEnv/py3.5/share/python-wheels/pkg_resources-0.0.0-py2.py3-none-any.whl/pkg_resources/_vendor/packaging/markers.py", line 278, in evaluate return _evaluate_markers(self._markers, current_environment) File "/home/gaurav/.virtualEnv/py3.5/share/python-wheels/pkg_resources-0.0.0-py2.py3-none-any.whl/pkg_resources/_vendor/packaging/markers.py", line 203, in _evaluate_markers lhs_value = _get_env(environment, lhs.value) File "/home/gaurav/.virtualEnv/py3.5/share/python-wheels/pkg_resources-0.0.0-py2.py3-none-any.whl/pkg_resources/_vendor/packaging/markers.py", line 185, in _get_env "{0!r} does not exist in evaluation environment.".format(name) pkg_resources.extern.packaging.markers.UndefinedEnvironmentName: 'extra' does not exist in evaluation environment. How to circumvent it. Thanks 
Member

### jaraco commented Aug 2, 2016

 @gitrookie: What version of setuptools do you have? Looks like you have a wheel called pkg_resources-0.0.0, which I think is an unreleased and unsupported version of pkg_resources.

### vhakulin commented Sep 28, 2016

 This seems to be broken again in setuptools 28.0.0: I was generating message files with pybabel from jinja templates when I got this exception. Downgrading setuptools to 27.3.1 fixed the issue.

Closed

### rwillmer commented Dec 12, 2016 • edited

 And here's a workaround which I don't understand. Downgrading and then upgrading back to the same version and it works. If you add these two lines before the 'RUN bin/paster' line, it works fine. RUN bin/pip install setuptools==30.4.0 RUN bin/pip install setuptools==31.0.0 

Open

Member

### jaraco commented Dec 12, 2016

 @rwillmer: What do you get if you invoke bin/easy_install --version prior to the workaround?

### bmjjr commented May 8, 2018

 Hit this today trying to build and install from a github repo with pip install git+git://github.com/repo. Downgraded setuptools-38.5.1 to setuptools-34.00 and this resolved it so I could successfully install.

### aventurella commented Oct 18, 2018 • edited

 I am getting this also when installing gevent pip==18.1 setuptools==40.4.3 gevent==1.3.7 Exception: Traceback (most recent call last): File "/usr/local/lib/python3.6/site-packages/pip/_internal/cli/base_command.py", line 143, in main status = self.run(options, args) File "/usr/local/lib/python3.6/site-packages/pip/_internal/commands/install.py", line 349, in run self._warn_about_conflicts(to_install) File "/usr/local/lib/python3.6/site-packages/pip/_internal/commands/install.py", line 475, in _warn_about_conflicts package_set, _dep_info = check_install_conflicts(to_install) File "/usr/local/lib/python3.6/site-packages/pip/_internal/operations/check.py", line 108, in check_install_conflicts package_set, should_ignore=lambda name: name not in whitelist File "/usr/local/lib/python3.6/site-packages/pip/_internal/operations/check.py", line 74, in check_package_set missed = req.marker.evaluate() File "/usr/local/lib/python3.6/site-packages/pip/_vendor/packaging/markers.py", line 301, in evaluate return _evaluate_markers(self._markers, current_environment) File "/usr/local/lib/python3.6/site-packages/pip/_vendor/packaging/markers.py", line 226, in _evaluate_markers lhs_value = _get_env(environment, lhs.value) File "/usr/local/lib/python3.6/site-packages/pip/_vendor/packaging/markers.py", line 208, in _get_env "{0!r} does not exist in evaluation environment.".format(name) pip._vendor.packaging.markers.UndefinedEnvironmentName: 'extra' does not exist in evaluation environment.  However, if as @bmjjr stated, I downgrade to setuptools==34.0.0 it installs fine.
Member

### benoit-pierre commented Oct 18, 2018

 You'll have to give more information than that. I can't reproduce with a fresh virtualenv/venv with Python 2.7 / Python 3.7. I setuptools unvendored?
Member

### benoit-pierre commented Oct 18, 2018

 Additionally, this occurs in the "check for conflicts phase" of pip's install, which means any one of the other distributions currently installed could be the problem, so it would be worth checking with pypa/pip#5842 to find out which one.