New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

pip.installed not available with pip 9.0.2 #46611

Open
autophagy opened this Issue Mar 19, 2018 · 7 comments

Comments

Projects
None yet
5 participants
@autophagy

autophagy commented Mar 19, 2018

Description of Issue/Question

Attempting to run highstate against a minion with multiple pip.installed definitions results in an exception:

State 'pip.installed' was not found in SLS 'k8s'
Reason: 'pip.installed' is not available.

However, this only occurs if the pip version is 9.0.2, and not if the pip version 9.0.1. When the pip version is 9.0.1, the highstate functions normally. This occurs regardless of whether I refresh the modules.

Setup

(Please provide relevant configs and/or SLS files (Be sure to remove sensitive info).)

kubernetes:
  pip.installed: 
    - name: kubernetes==3.0
    - reload_modules: True

Steps to Reproduce Issue

Define the pip.installed definition as above, install pip 9.0.2 on the minion. Results in:

          ID: kubernetes
    Function: pip.installed
        Name: kubernetes==3.0
      Result: False
     Comment: State 'pip.installed' was not found in SLS 'k8s'
              Reason: 'pip.installed' is not available.

If run on the minion:

[root@node ~]$ sudo salt-call state.highstate
[ERROR   ] Failed to import states pip_state, this is due most likely to a syntax error:
Traceback (most recent call last):
  File "/usr/lib/python2.7/site-packages/salt/loader.py", line 1412, in _load_module
    mod = imp.load_module(mod_namespace, fn_, fpath, desc)
  File "/usr/lib/python2.7/site-packages/salt/states/pip_state.py", line 37, in <module>
    import pip
  File "/usr/lib/python2.7/site-packages/pip-9.0.2-py2.7.egg/pip/__init__.py", line 45, in <module>
    from pip.vcs import git, mercurial, subversion, bazaar  # noqa
  File "/usr/lib/python2.7/site-packages/pip-9.0.2-py2.7.egg/pip/vcs/mercurial.py", line 9, in <module>
    from pip.download import path_to_url
  File "/usr/lib/python2.7/site-packages/pip-9.0.2-py2.7.egg/pip/download.py", line 40, in <module>
    from pip._vendor import requests, six
  File "/usr/lib/python2.7/site-packages/pip-9.0.2-py2.7.egg/pip/_vendor/requests/__init__.py", line 98, in <module>
    from . import packages
  File "/usr/lib/python2.7/site-packages/pip-9.0.2-py2.7.egg/pip/_vendor/requests/packages.py", line 12, in <module>
    sys.modules['pip._vendor.requests.packages.' + mod] = sys.modules["pip._vendor." + mod]
KeyError: 'pip._vendor.urllib3.urllib'

Downgrade pip to 9.0.1 (with pip install pip==9.0.1) and the highstate is correctly applied.

Versions Report

Salt Version:
           Salt: 2017.7.4
 
Dependency Versions:
           cffi: Not Installed
       cherrypy: Not Installed
       dateutil: 2.7.0
      docker-py: 1.10.6
          gitdb: Not Installed
      gitpython: Not Installed
          ioflo: Not Installed
         Jinja2: 2.7.2
        libgit2: Not Installed
        libnacl: Not Installed
       M2Crypto: Not Installed
           Mako: Not Installed
   msgpack-pure: Not Installed
 msgpack-python: 0.4.6
   mysql-python: Not Installed
      pycparser: Not Installed
       pycrypto: 2.6.1
   pycryptodome: Not Installed
         pygit2: Not Installed
         Python: 2.7.5 (default, Aug  4 2017, 00:39:18)
   python-gnupg: Not Installed
         PyYAML: 3.12
          PyZMQ: 15.3.0
           RAET: Not Installed
          smmap: Not Installed
        timelib: Not Installed
        Tornado: 4.2.1
            ZMQ: 4.1.4
 
System Versions:
           dist: centos 7.4.1708 Core
         locale: UTF-8
        machine: x86_64
        release: 3.10.0-693.21.1.el7.x86_64
         system: Linux
        version: CentOS Linux 7.4.1708 Core

No difference between master and minion versions.

@autophagy

This comment has been minimized.

autophagy commented Mar 19, 2018

I believe this might be the same issue mentioned here: #44980 (comment)

@diginc

This comment has been minimized.

diginc commented Mar 20, 2018

Attempted the locations.py patch from #44980 without luck. Downgrading to pip 9.0.1 did work though per this comment.

[ERROR   ] Failed to import states pip_state, this is due most likely to a syntax error:
Traceback (most recent call last):
  File "/usr/lib/python2.7/dist-packages/salt/loader.py", line 1411, in _load_module
    mod = imp.load_module(mod_namespace, fn_, fpath, desc)
  File "/usr/lib/python2.7/dist-packages/salt/states/pip_state.py", line 36, in <module>
    import pip
  File "/usr/local/lib/python2.7/dist-packages/pip/__init__.py", line 45, in <module>
    from pip.vcs import git, mercurial, subversion, bazaar  # noqa
  File "/usr/local/lib/python2.7/dist-packages/pip/vcs/mercurial.py", line 9, in <module>
    from pip.download import path_to_url
  File "/usr/local/lib/python2.7/dist-packages/pip/download.py", line 40, in <module>
    from pip._vendor import requests, six
  File "/usr/local/lib/python2.7/dist-packages/pip/_vendor/requests/__init__.py", line 98, in <module>
    from . import packages
  File "/usr/local/lib/python2.7/dist-packages/pip/_vendor/requests/packages.py", line 12, in <module>
    sys.modules['pip._vendor.requests.packages.' + mod] = sys.modules["pip._vendor." + mod]
KeyError: 'pip._vendor.urllib3.urllib'

pip 9.0.2 from /usr/local/lib/python2.7/dist-packages (python 2.7), Ubuntu 14.04

@garethgreenaway

This comment has been minimized.

Member

garethgreenaway commented Mar 20, 2018

Seems similar to #44980. The pip module isn't available because something isn't loading right, which definitely seems like an upstream issue.

@garethgreenaway garethgreenaway added this to the Approved milestone Mar 20, 2018

vdog added a commit to vdog/pip that referenced this issue Mar 21, 2018

@vdog

This comment has been minimized.

vdog commented Mar 21, 2018

pypa/pip#5101 actually should be the right fix.

@vdog

This comment has been minimized.

vdog commented Mar 21, 2018

This looks like it may not get fixed, or it will surely get broken in pip 10, which is going to be released soon @garethgreenaway

pypa/pip#5081

The pip maintainers are stressing that import pip should not be used by other python programs.

https://pip.pypa.io/en/latest/user_guide/#using-pip-from-your-program

@diginc

This comment has been minimized.

diginc commented Mar 22, 2018

Thankfully the fixed it with 9.0.3 - which is the right thing considering this wasn't an advertised breaking change or semantic versioned correctly.

I'm surprised by the best practice/recommendation to never use import pip...I'd expect the top python package provider to provide a stable python native (not subprocess shell calls) API interface.

@bryceml

This comment has been minimized.

bryceml commented Sep 25, 2018

I seem to have this issue running on version 9.0.1 on Ubuntu 18.04. It also still happens if I do pip install --upgrade pip and specify the bin_env to /usr/local/bin/pip and therefore attempt to use 18.0.

Relevant state:

# makes sure python-pip and python3-pip are installed
python-pip:
  pkg.installed:
    - pkgs:
      - python-pip
      - python3-pip
# installs python2 version of tensorflow
tensorflow:
  pip.installed:
    - name: tensorflow
    - require:
      - pkg: python-pip

Some Log Lines:

[DEBUG   ] Could not LazyLoad pip.installed: 'pip.installed' is not available.
[INFO    ] Running state [tensorflow] at time 10:32:31.413069
[DEBUG   ] Could not LazyLoad pip.installed: 'pip.installed' is not available.
[ERROR   ] State 'pip.installed' was not found in SLS 'research.seppi' 
Reason: 'pip.installed' is not available.

It works fine when I run the state explicitly, but not when running from highstate.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment