pip module's new list-of-names feature (v2.2) breaks pip's "extras"

[akhayyat]

ISSUE TYPE

• Bug Report

COMPONENT NAME

pip

ANSIBLE VERSION

ansible 2.2.0.0
  config file = /etc/ansible/ansible.cfg
  configured module search path = Default w/o overrides

CONFIGURATION

No changes.

OS / ENVIRONMENT

Debian GNU/Linux testing (stretch)

SUMMARY

With the new support for list of names in 2.2, listing pip-supported setuptools "extras" is broken. "extras" are optional package features. See example 6 in pip examples and Declaring “Extras” (optional features with their own dependencies) in setuptools documentation for more information on "extras".

Moved from ansible/ansible-modules-core#5843.

STEPS TO REPRODUCE

- hosts: localhost
  tasks:
    - pip: name=buildbot[bundle,tls] version=0.9.1

EXPECTED RESULTS

buildbot==0.9.1 is installed with the bundle and tls extras/options. This works with pre-2.2 versions of Ansible.

ACTUAL RESULTS

The comma separating the extras is instead interpreted as separating multiple python packages for pip to install: Invalid requirement: 'buildbot[bundle==0.9.1.

PLAY [localhost] ***************************************************************

TASK [setup] *******************************************************************
ok: [localhost]

TASK [pip] *********************************************************************
fatal: [localhost]: FAILED! => {"changed": false, "cmd": "/usr/bin/pip2 install buildbot[bundle==0.9.1 tls]==0.9.1", "failed": true, "msg": "\n:stderr: Invalid requirement: 'buildbot[bundle==0.9.1'\nTraceback (most recent call last):\n  File \"/usr/lib/python2.7/dist-packages/pip/req/req_install.py\", line 82, in __init__\n    req = Requirement(req)\n  File \"/usr/share/python-wheels/packaging-16.8-py2.py3-none-any.whl/packaging/requirements.py\", line 94, in __init__\n    requirement_string[e.loc:e.loc + 8]))\nInvalidRequirement: Invalid requirement, parse error at \"'[bundle='\"\n\n"}
        to retry, use: --limit @/home/ahmad/local/test/ansible/pip.retry

PLAY RECAP *********************************************************************
localhost                  : ok=1    changed=0    unreachable=0    failed=1

[ansibot]

cc @mattupstate
click here for bot help

[akhayyat]

Any plans for fixing this issue? This bug introduces a backwards incompatibility. A working configuration for Ansible 2.1 does not work with any >=2.2.0 version. At the same time, some third-party roles require >=2.2.0 versions!

I looked at the code, but could not easily see a straightforward way to fix this bug. Lists are parsed in a generic way that is not specific to this module.

Perhaps that generic list parsing should ignore comas enclosed in any type of parentheses? Would that break anything else?

[bcoca]

so this is a consequence of pip being able to take a 'list of packages' now, you should be able to work around it by avoiding the k=v syntax:

- hosts: localhost
  tasks:
    - pip: 
        name:
            - 'buildbot[bundle,tls]'
        version: 0.9.1

[akhayyat]

That worked! Thanks @bcoca

[Cryolite]

On Ansible 2.7, even the workaround described in #19131 (comment) does not seem to work.