pip: Add support for various pip install modes

[nightowlengineer]

SUMMARY

Introduces support for the various 'site' locations in pip - system (default), user, target, prefix, root. Fixes #56333, potential fix for #71781

ISSUE TYPE

• Feature Pull Request

COMPONENT NAME

pip

ADDITIONAL INFORMATION

Currently pip can only be used to install to the 'user site' location by including extra_args, which is not a great user experience. The module also (correctly) marks any invocation with extra_args as 'changed' when in check mode. This will allow at least some tasks to be migrated away and have more accurate idempotency.

Before:

{
  "virtualenv": null,
  "changed": true,
  "requirements": null,
  "name": [
    "pyyaml"
  ],
  "stdout": "Collecting pyyaml==5.3.0\nInstalling collected packages: pyyaml\nSuccessfully installed pyyaml-5.3\n",
  "cmd": [
    "/usr/bin/pip3",
    "install",
    "pyyaml==5.3.0"
  ],
  "state": "present",
  "version": "5.3.0",
  "stderr": "",
  "invocation": {
    "module_args": {
      "virtualenv": null,
      "virtualenv_site_packages": false,
      "executable": "pip3",
      "chdir": null,
      "requirements": null,
      "name": [
        "pyyaml"
      ],
      "virtualenv_python": null,
      "editable": false,
      "umask": null,
      "virtualenv_command": "virtualenv",
      "extra_args": null,
      "state": "present",
      "version": "5.3.0"
    }
  }
}

After (user):

{
  "virtualenv": null,
  "changed": true,
  "requirements": null,
  "name": [
    "pyyaml"
  ],
  "stdout": "Collecting pyyaml==5.3.0\nInstalling collected packages: pyyaml\nSuccessfully installed pyyaml-5.3\n",
  "cmd": [
    "/usr/bin/pip3",
    "install",
    "--user",
    "pyyaml==5.3.0"
  ],
  "state": "present",
  "version": "5.3.0",
  "stderr": "",
  "invocation": {
    "module_args": {
      "virtualenv": null,
      "virtualenv_site_packages": false,
      "executable": "pip3",
      "chdir": null,
      "requirements": null,
      "name": [
        "pyyaml"
      ],
      "virtualenv_python": null,
      "editable": false,
      "umask": null,
      "site": "user",
      "virtualenv_command": "virtualenv",
      "extra_args": null,
      "state": "present",
      "version": "5.3.0",
      "path": null
    }
  }
}

After (prefix):

{
  "virtualenv": null,
  "changed": true,
  "requirements": null,
  "name": [
    "pyyaml"
  ],
  "stdout": "Collecting pyyaml==5.3.0\nInstalling collected packages: pyyaml\nSuccessfully installed pyyaml-5.3\n",
  "cmd": [
    "/usr/bin/pip3",
    "install",
    "--root",
    "/tmp/prefixed-path",
    "pyyaml==5.3.0"
  ],
  "state": "present",
  "version": "5.3.0",
  "stderr": "",
  "invocation": {
    "module_args": {
      "virtualenv": null,
      "virtualenv_site_packages": false,
      "executable": "pip3",
      "chdir": null,
      "requirements": null,
      "name": [
        "pyyaml"
      ],
      "virtualenv_python": null,
      "editable": false,
      "umask": null,
      "site": "root",
      "virtualenv_command": "virtualenv",
      "extra_args": null,
      "state": "present",
      "version": "5.3.0",
      "path": "/tmp/prefixed-path"
    }
  }
}

[ansibot]

The test ansible-test sanity --test validate-modules [explain] failed with 1 error:

lib/ansible/modules/pip.py:0:0: doc-choices-do-not-match-spec: Argument 'site' in argument_spec defines choices as (['system', 'user', 'root', 'prefix', 'target']) but documentation defines choices as (['system', 'user', 'home', 'root', 'prefix', 'target'])

The test ansible-test sanity --test pylint [explain] failed with 12 errors:

lib/ansible/modules/pip.py:646:10: bad-whitespace: No space allowed after bracket           [ "site", "prefix", [ "path" ] ],           ^
lib/ansible/modules/pip.py:646:30: bad-whitespace: No space allowed after bracket           [ "site", "prefix", [ "path" ] ],                               ^
lib/ansible/modules/pip.py:646:39: bad-whitespace: No space allowed before bracket           [ "site", "prefix", [ "path" ] ],                                        ^
lib/ansible/modules/pip.py:646:41: bad-whitespace: No space allowed before bracket           [ "site", "prefix", [ "path" ] ],                                          ^
lib/ansible/modules/pip.py:647:10: bad-whitespace: No space allowed after bracket           [ "site", "root", [ "path" ] ],           ^
lib/ansible/modules/pip.py:647:28: bad-whitespace: No space allowed after bracket           [ "site", "root", [ "path" ] ],                             ^
lib/ansible/modules/pip.py:647:37: bad-whitespace: No space allowed before bracket           [ "site", "root", [ "path" ] ],                                      ^
lib/ansible/modules/pip.py:647:39: bad-whitespace: No space allowed before bracket           [ "site", "root", [ "path" ] ],                                        ^
lib/ansible/modules/pip.py:648:10: bad-whitespace: No space allowed after bracket           [ "site", "target", [ "path" ] ]           ^
lib/ansible/modules/pip.py:648:30: bad-whitespace: No space allowed after bracket           [ "site", "target", [ "path" ] ]                               ^
lib/ansible/modules/pip.py:648:39: bad-whitespace: No space allowed before bracket           [ "site", "target", [ "path" ] ]                                        ^
lib/ansible/modules/pip.py:648:41: bad-whitespace: No space allowed before bracket           [ "site", "target", [ "path" ] ]                                          ^

The test ansible-test sanity --test pep8 [explain] failed with 14 errors:

lib/ansible/modules/pip.py:646:11: E121: continuation line under-indented for hanging indent
lib/ansible/modules/pip.py:646:12: E201: whitespace after '['
lib/ansible/modules/pip.py:646:32: E201: whitespace after '['
lib/ansible/modules/pip.py:646:39: E202: whitespace before ']'
lib/ansible/modules/pip.py:646:41: E202: whitespace before ']'
lib/ansible/modules/pip.py:647:12: E201: whitespace after '['
lib/ansible/modules/pip.py:647:30: E201: whitespace after '['
lib/ansible/modules/pip.py:647:37: E202: whitespace before ']'
lib/ansible/modules/pip.py:647:39: E202: whitespace before ']'
lib/ansible/modules/pip.py:648:12: E201: whitespace after '['
lib/ansible/modules/pip.py:648:32: E201: whitespace after '['
lib/ansible/modules/pip.py:648:39: E202: whitespace before ']'
lib/ansible/modules/pip.py:648:41: E202: whitespace before ']'
lib/ansible/modules/pip.py:770:29: E128: continuation line under-indented for visual indent

click here for bot help

[nightowlengineer]

ready_for_review

[nitzmahone]

A number of the idempotence checks would be broken by this change, as well as potentially allowing "escape" from a virtualenv. Would also need to look into using module respawn to ensure that the idempotence checks run in the proper context. If you're still interested in moving forward with this, we can do a full review- just let us know. Thanks!

[nightowlengineer]

A number of the idempotence checks would be broken by this change, as well as potentially allowing "escape" from a virtualenv. Would also need to look into using module respawn to ensure that the idempotence checks run in the proper context. If you're still interested in moving forward with this, we can do a full review- just let us know. Thanks!

Hey @nitzmahone - certainly happy to consider more changes, though with it being 18 months since I worked on this it will probably require quite a few updates anyway, I see some merge conflicts are showing up. In the meantime, I'd be interested to have more information on the problems you describe 😃