36.2.1 appears to ignore install_requires with extras

[warner]

Travis, Appveyor, and local builds of magic-wormhole started failing about an hour ago, when 36.2.1 was released. Our setup.py has:

      install_requires=[
          "spake2==0.7", "pynacl",
          "six",
          "twisted[tls] >= 17.5.0", # 17.5.0 adds failAfterFailures=
          "autobahn[twisted] >= 0.14.1",
          "automat",
          "hkdf",
          "tqdm >= 4.13.0", # 4.13.0 fixes crash on NetBSD
          "click",
          "humanize",
          "ipaddress",
          "txtorcon >= 0.19.3",
      ],

When I run pip install -e . in a virtualenv with setuptools-36.2.0, I see Autobahn-17.7.1 installed as expected. If I have setuptools-36.2.1, then Autobahn does not get installed. (Twisted does get installed, but it's a dependency of txtorcon).

[benoit-pierre]

Should that be pip install -e '.[twisted]'?

[benoit-pierre]

Because of the extra specifiers:

setup(install_requires=["autobahn[twisted] >= 0.14.1"])

is equivalent to

setup(extras_require={'twisted': ["autobahn >= 0.14.1"]})

Previously those were ignored.

[warner]

Nope, in this case, the un-extra-ed magic-wormhole installation requires both the base autobahn and autobahn's twisted support. (magic-wormhole depends upon twisted directly, but when you ask autobahn to give you its twisted support, it could conceivably depend upon something beyond just twisted, maybe some tx*foo package).

Depending upon autobahn instead of autobahn[twisted] gets you a subset of the functionality that we need.

The only extras that magic-wormhole supports are pip install magic-wormhole[dev], and a :sys_platform=="win32" that gets us some extra windows support.

[warner]

Oh, what? Have I been mis-expressing this all this time?

What's the syntax to say "I want dependency X and also X's extra Y"?

[benoit-pierre]

No, I think I'm the one being wrong, and I have introduced a regression...

[benoit-pierre]

https://www.python.org/dev/peps/pep-0508/#extras

[warner]

Whew, ok, good :). I looked at https://setuptools.readthedocs.io/en/latest/setuptools.html#declaring-extras-optional-features-with-their-own-dependencies , and it seems to match the way magic-wormhole's setup.py is doing things.

[AiOO]

I'm having the same problem.

$ cat setup.py
from setuptools import setup
setup(
    name="testtest",
    install_requires=['requests[security]']
)

$ pip install .
Processing /Users/bright/Playground/testtest
Installing collected packages: testtest
  Running setup.py install for testtest ... done
Successfully installed testtest-0.0.0

$ pip freeze
testtest==0.0.0

Expected result:

$ pip freeze
asn1crypto==0.22.0
certifi==2017.4.17
cffi==1.10.0
chardet==3.0.4
cryptography==2.0
enum34==1.1.6
idna==2.5
ipaddress==1.0.18
pycparser==2.18
pyOpenSSL==17.2.0
requests==2.18.1
six==1.10.0
testtest==0.0.0
urllib3==1.21.1

[jaraco]

setup(install_requires=["autobahn[twisted] >= 0.14.1"])
is equivalent to
setup(extras_require={'twisted': ["autobahn >= 0.14.1"]})

I'm pretty sure this statement is wrong.

The former syntax says "this package depends on autobahn with its twisted extra and autobahn should be version 0.14.1 or later". The latter syntax says "this package offers an optional feature called "twisted" and if you ask for thispackage[twisted], this package will additionally depend on autobahn 0.14.1 or later."

[jaraco]

Fix released with 36.2.2.