pip.installed not available with pip 9.0.2

[autophagy]

Description of Issue/Question

Attempting to run highstate against a minion with multiple pip.installed definitions results in an exception:

State 'pip.installed' was not found in SLS 'k8s'
Reason: 'pip.installed' is not available.

However, this only occurs if the pip version is 9.0.2, and not if the pip version 9.0.1. When the pip version is 9.0.1, the highstate functions normally. This occurs regardless of whether I refresh the modules.

Setup

(Please provide relevant configs and/or SLS files (Be sure to remove sensitive info).)

kubernetes:
  pip.installed: 
    - name: kubernetes==3.0
    - reload_modules: True

Steps to Reproduce Issue

Define the pip.installed definition as above, install pip 9.0.2 on the minion. Results in:

          ID: kubernetes
    Function: pip.installed
        Name: kubernetes==3.0
      Result: False
     Comment: State 'pip.installed' was not found in SLS 'k8s'
              Reason: 'pip.installed' is not available.

If run on the minion:

[root@node ~]$ sudo salt-call state.highstate
[ERROR   ] Failed to import states pip_state, this is due most likely to a syntax error:
Traceback (most recent call last):
  File "/usr/lib/python2.7/site-packages/salt/loader.py", line 1412, in _load_module
    mod = imp.load_module(mod_namespace, fn_, fpath, desc)
  File "/usr/lib/python2.7/site-packages/salt/states/pip_state.py", line 37, in <module>
    import pip
  File "/usr/lib/python2.7/site-packages/pip-9.0.2-py2.7.egg/pip/__init__.py", line 45, in <module>
    from pip.vcs import git, mercurial, subversion, bazaar  # noqa
  File "/usr/lib/python2.7/site-packages/pip-9.0.2-py2.7.egg/pip/vcs/mercurial.py", line 9, in <module>
    from pip.download import path_to_url
  File "/usr/lib/python2.7/site-packages/pip-9.0.2-py2.7.egg/pip/download.py", line 40, in <module>
    from pip._vendor import requests, six
  File "/usr/lib/python2.7/site-packages/pip-9.0.2-py2.7.egg/pip/_vendor/requests/__init__.py", line 98, in <module>
    from . import packages
  File "/usr/lib/python2.7/site-packages/pip-9.0.2-py2.7.egg/pip/_vendor/requests/packages.py", line 12, in <module>
    sys.modules['pip._vendor.requests.packages.' + mod] = sys.modules["pip._vendor." + mod]
KeyError: 'pip._vendor.urllib3.urllib'

Downgrade pip to 9.0.1 (with pip install pip==9.0.1) and the highstate is correctly applied.

Versions Report

Salt Version:
           Salt: 2017.7.4
 
Dependency Versions:
           cffi: Not Installed
       cherrypy: Not Installed
       dateutil: 2.7.0
      docker-py: 1.10.6
          gitdb: Not Installed
      gitpython: Not Installed
          ioflo: Not Installed
         Jinja2: 2.7.2
        libgit2: Not Installed
        libnacl: Not Installed
       M2Crypto: Not Installed
           Mako: Not Installed
   msgpack-pure: Not Installed
 msgpack-python: 0.4.6
   mysql-python: Not Installed
      pycparser: Not Installed
       pycrypto: 2.6.1
   pycryptodome: Not Installed
         pygit2: Not Installed
         Python: 2.7.5 (default, Aug  4 2017, 00:39:18)
   python-gnupg: Not Installed
         PyYAML: 3.12
          PyZMQ: 15.3.0
           RAET: Not Installed
          smmap: Not Installed
        timelib: Not Installed
        Tornado: 4.2.1
            ZMQ: 4.1.4
 
System Versions:
           dist: centos 7.4.1708 Core
         locale: UTF-8
        machine: x86_64
        release: 3.10.0-693.21.1.el7.x86_64
         system: Linux
        version: CentOS Linux 7.4.1708 Core

No difference between master and minion versions.

[autophagy]

I believe this might be the same issue mentioned here: #44980 (comment)

[diginc]

Attempted the locations.py patch from #44980 without luck. Downgrading to pip 9.0.1 did work though per this comment.

[ERROR   ] Failed to import states pip_state, this is due most likely to a syntax error:
Traceback (most recent call last):
  File "/usr/lib/python2.7/dist-packages/salt/loader.py", line 1411, in _load_module
    mod = imp.load_module(mod_namespace, fn_, fpath, desc)
  File "/usr/lib/python2.7/dist-packages/salt/states/pip_state.py", line 36, in <module>
    import pip
  File "/usr/local/lib/python2.7/dist-packages/pip/__init__.py", line 45, in <module>
    from pip.vcs import git, mercurial, subversion, bazaar  # noqa
  File "/usr/local/lib/python2.7/dist-packages/pip/vcs/mercurial.py", line 9, in <module>
    from pip.download import path_to_url
  File "/usr/local/lib/python2.7/dist-packages/pip/download.py", line 40, in <module>
    from pip._vendor import requests, six
  File "/usr/local/lib/python2.7/dist-packages/pip/_vendor/requests/__init__.py", line 98, in <module>
    from . import packages
  File "/usr/local/lib/python2.7/dist-packages/pip/_vendor/requests/packages.py", line 12, in <module>
    sys.modules['pip._vendor.requests.packages.' + mod] = sys.modules["pip._vendor." + mod]
KeyError: 'pip._vendor.urllib3.urllib'

pip 9.0.2 from /usr/local/lib/python2.7/dist-packages (python 2.7), Ubuntu 14.04

[garethgreenaway]

Seems similar to #44980. The pip module isn't available because something isn't loading right, which definitely seems like an upstream issue.

[vdog]

pypa/pip#5101 actually should be the right fix.

[vdog]

This looks like it may not get fixed, or it will surely get broken in pip 10, which is going to be released soon @garethgreenaway

pypa/pip#5081

The pip maintainers are stressing that import pip should not be used by other python programs.

https://pip.pypa.io/en/latest/user_guide/#using-pip-from-your-program

[diginc]

Thankfully the fixed it with 9.0.3 - which is the right thing considering this wasn't an advertised breaking change or semantic versioned correctly.

I'm surprised by the best practice/recommendation to never use import pip...I'd expect the top python package provider to provide a stable python native (not subprocess shell calls) API interface.

[bryceml]

I seem to have this issue running on version 9.0.1 on Ubuntu 18.04. It also still happens if I do pip install --upgrade pip and specify the bin_env to /usr/local/bin/pip and therefore attempt to use 18.0.

Relevant state:

# makes sure python-pip and python3-pip are installed
python-pip:
  pkg.installed:
    - pkgs:
      - python-pip
      - python3-pip
# installs python2 version of tensorflow
tensorflow:
  pip.installed:
    - name: tensorflow
    - require:
      - pkg: python-pip

Some Log Lines:

[DEBUG   ] Could not LazyLoad pip.installed: 'pip.installed' is not available.
[INFO    ] Running state [tensorflow] at time 10:32:31.413069
[DEBUG   ] Could not LazyLoad pip.installed: 'pip.installed' is not available.
[ERROR   ] State 'pip.installed' was not found in SLS 'research.seppi' 
Reason: 'pip.installed' is not available.

It works fine when I run the state explicitly, but not when running from highstate.