'extras_require' requirements cannot include environment markers

[sashkab]

I just upgraded to the latest setuptools by accident, and now get following error:

 'extras_require' requirements cannot include environment markers, in 'notebook': 'enum34==1.1.6; python_version < "3.4"'

Portion of my extras_notebook_requirement file:

enum34==1.1.6; python_version < '3.4'

This is probably related to the #1081.

[benoit-pierre]

Can you give more information about your exact use-case? The thing is, while this was previously allowed, this would result in wheels that don't work as expected.

For example:

> cat setup.py
import sys

from setuptools import setup

setup(
    name='foo',
    extras_require={
        'test': 'barbazquux; "foobar" in sys_platform',
    },
)
> python ./setup.py -q bdist_wheel
> unzip -p dist/foo-0.0.0-py3-none-any.whl foo-0.0.0.dist-info/metadata.json | json_pp
{
   "extensions" : {
      "python.details" : {
         "document_names" : {
            "description" : "DESCRIPTION.rst"
         }
      }
   },
   "version" : "0.0.0",
   "name" : "foo",
   "summary" : "UNKNOWN",
   "metadata_version" : "2.0",
   "run_requires" : [
      {
         "extra" : "test",
         "requires" : [
            "barbazquux"
         ]
      }
   ],
   "extras" : [
      "test"
   ],
   "generator" : "bdist_wheel (0.29.0)"
}
> pip install --user dist/foo-0.0.0-py3-none-any.whl'[test]'
Processing ./dist/foo-0.0.0-py3-none-any.whl
Collecting barbazquux; extra == "test" (from foo==0.0.0)
  Could not find a version that satisfies the requirement barbazquux; extra == "test" (from foo==0.0.0) (from versions: )
No matching distribution found for barbazquux; extra == "test" (from foo==0.0.0)

As you can see, the environment marker was stripped, and barbarzquux will always be installed.

[erikwright]

We use extras_require to define a list of dev requirements, including mypy. mypy does not install on Python 2.

[benoit-pierre]

The valid syntax is:

> cat setup.py
import sys

from setuptools import setup

setup(
    name='foo',
    extras_require={
        'test:"foobar" in sys_platform': ['barbazquux'],
    },
)
> python ./setup.py -q bdist_wheel
> unzip -p dist/foo-0.0.0-py3-none-any.whl foo-0.0.0.dist-info/metadata.json | json_pp
{
   "name" : "foo",
   "metadata_version" : "2.0",
   "generator" : "bdist_wheel (0.29.0)",
   "summary" : "UNKNOWN",
   "extensions" : {
      "python.details" : {
         "document_names" : {
            "description" : "DESCRIPTION.rst"
         }
      }
   },
   "version" : "0.0.0",
   "run_requires" : [
      {
         "extra" : "test",
         "requires" : [
            "barbazquux"
         ],
         "environment" : "\"foobar\" in sys_platform"
      }
   ],
   "extras" : [
      "test"
   ]
}
> pip install --user dist/foo-0.0.0-py3-none-any.whl'[test]'
Processing ./dist/foo-0.0.0-py3-none-any.whl
Installing collected packages: foo
Successfully installed foo-0.0.0

[sashkab]

My problem wasn't with the change -- it is clearly bug fix, but with the fact that breaking change was introduced in the maintenance release.

I'd expect first deprecation warning, and then removed in follow up release.

[benoit-pierre]

But it does not break a valid case. It was already broken, you just did not know it because setuptools was silently allowing it.

[erikwright]

@benoit-pierre So if I understand correctly, we will define two separate extras_require sections, one for Python 2 and one for Python 3. They will both be named test and they will both be complete lists of packages, and one or the other will be installed. Is that right?

[benoit-pierre]

@erikwright: yes, using something like:

    extras_require={
        'dev:python_version <= "2.7"': ['py2dep1', 'py2dep2'],
        'dev:python_version > "2.7"': ['py3dep1', 'py3dep2'],
   }

should work as expected.

[benoit-pierre]

Just to clarify, this would work as expected:

    extras_require={
        'test': ['flake8'],
        'test:python_version >= "3.4"': ['mypy'],
    },

installing both flake8 and mypy when using Python 3.4+.

[benoit-pierre]

@sashkab: honestly, I did not even know a syntax like 'test:python_version >= "3.4"': ['mypy'], was valid when I make the PR (I never saw it documented anywhere). Maybe I can make a new PR to automatically convert something like 'test': ['mypy; python_version >= "3.4"], to 'test:python_version >= "3.4"': ['mypy'],.

[sashkab]

'test:python_version >= "3.4"': ['mypy'],

No, syntax was:

extras_require={ 
   'notebook': ["enum34==1.1.6; python_version < '3.4'"]
}

[benoit-pierre]

OK, but this would not work as expected, enum34 would be installed no matter the Python version. While this would have worked (and still does):

extras_require={ 
   'notebook: python_version < "3.4"': ['enum34==1.1.6']
}

[sashkab]

this would not work as expected

It was working. enum34 wasn't installing in the python >=3.4. Maybe, due to pinned setuptools to 33.1.1.

[benoit-pierre]

I just tested with Python 3.6 and setuptools==33.1.1, and when installing from wheel enum34 definitively get installed too as the environment marker was stripped in the wheel metadata.