pip install --target fails

[chrismeyersfsu]

docker build -t pip_error .
docker run --name pip_error -it pip_error
pip install  --target /tmp/target_directory -vvv "psutil"

Exception:
Traceback (most recent call last):
  File "/usr/lib/python2.7/site-packages/pip/basecommand.py", line 223, in main
    status = self.run(options, args)
  File "/usr/lib/python2.7/site-packages/pip/commands/install.py", line 348, in run
    for item in os.listdir(lib_dir):
OSError: [Errno 2] No such file or directory: '/tmp/tmpO5kwWK/lib/python/'

Full Log: https://gist.github.com/chrismeyersfsu/6967e1f41db9660cdc1b

[root@59cecad2bf5a /]# pip --version
pip 7.1.0 from /usr/lib/python2.7/site-packages (python 2.7)

Also ran pip install pip --upgrade (to v 7.1.2) and still had the same error.

Dockerfile

FROM centos:7

RUN yum install -y epel-release http://yum.postgresql.org/9.4/redhat/rhel-7-x86_64/pgdg-redhat94-9.4-1.noarch.rpm
RUN yum install -y git rpmlink rpmdevtools gcc make gcc-c++ nodejs npm python-devel libxml2-devel libxslt-devel zeromq-devel mongodb-org python-pymongo-gridfs pymongo=2.8 nodejs-grunt-cli python-pip postgresql94-devel libffi-devel openldap-devel
#ENV PATH=$PATH:/usr/pgsql-9.4/bin/
CMD /bin/bash

Also tried
DISTUTILS_DEBUG=1 pip install -t /tmp/me --install-option="--install-purelib=\$base/lib64/python" billiard Fails and doesn't seem to effect the install_purelib disutils variable.

https://gist.github.com/chrismeyersfsu/709a80df91176bc9f7bc

Solution:
Creating a setup.cfg with below contents successfully effected the disutils install_purelib variable.
DISTUTILS_DEBUG=1 pip install -t /tmp/install_here -vvvv psutil
Log: https://gist.github.com/chrismeyersfsu/036b494a2a37d636839b

[install]
install-purelib=$base/lib64/python

Reflections:
My question now is, who is at 'fault' ? Should pip default to using the purelib directory? Or should a directive exist that complements --target to choose the variable from which to look for the artifacts (i.e. --target-use <purelib, lib>)
Further, why did --install-option="--install-purelib=\$base/lib64/python" have no effect?

[movermeyer]

I have independently ran into this issue while trying to install lxml.

It gives a similar trace as this other issue.

Here is the Dockerfile I created to reproduce the issue.

distutils_scheme seems to generate the wrong lib location. It produces a path in lib instead of lib64

[wm75]

see #3450

[andrewmichaelsmith]

Any pointers for working around this issue?

[wm75]

Shouldn't this be fixed in pip 8.1.2? See #3694

[andrewmichaelsmith]

@wm75 Aha, yes! Upgrading from pip 8.1.1 to 8.1.2 has fixed this issue.

Thanks.

[movermeyer]

The Dockerfile I submitted still reproduces the problem, even though it is using the latest pip (8.1.2).

[andrewmichaelsmith]

@movermeyer We are using CentOS 7 and upgrading to 8.1.2 resolved the issue for us.

[yamitzky]

I reproduce the issue on Amazon Linux Docker Container. Dockerfile is very very simple.

FROM 137112412989.dkr.ecr.ap-northeast-1.amazonaws.com/amazonlinux:latest
RUN curl -sL https://bootstrap.pypa.io/get-pip.py | python
RUN pip install lxml -t libs

Then, docker build fails.

Exception:
Traceback (most recent call last):
  File "/usr/local/lib/python2.7/site-packages/pip/basecommand.py", line 215, in main
    status = self.run(options, args)
  File "/usr/local/lib/python2.7/site-packages/pip/commands/install.py", line 357, in run
    for item in os.listdir(lib_dir):
OSError: [Errno 2] No such file or directory: '/tmp/tmpld2WbC/lib/python'

There is '/tmp/tmpld2WbC/lib64/python'.

Any workaroud?

[rickatnight11]

I'm also experiencing this issue on Fedora 25 with the python-pip-8.1.2-2.fc25.noarch package:

✔ ~/repos/test [master|✔] 
15:50 $ pip install -r source/requirements.txt -t source/vendor
Collecting jinja2 (from -r source/requirements.txt (line 1))
  Using cached Jinja2-2.9.6-py2.py3-none-any.whl
Collecting MarkupSafe>=0.23 (from jinja2->-r source/requirements.txt (line 1))
  Using cached MarkupSafe-1.0.tar.gz
Installing collected packages: MarkupSafe, jinja2
  Running setup.py install for MarkupSafe ... done
Successfully installed MarkupSafe-0.23 jinja2-2.8
Exception:
Traceback (most recent call last):
  File "/usr/lib/python2.7/site-packages/pip/basecommand.py", line 215, in main
    status = self.run(options, args)
  File "/usr/lib/python2.7/site-packages/pip/commands/install.py", line 366, in run
    for item in os.listdir(lib_dir):
OSError: [Errno 2] No such file or directory: '/tmp/tmphRkPu_/lib/python'
You are using pip version 8.1.2, however version 9.0.1 is available.
You should consider upgrading via the 'pip install --upgrade pip' command.

Allowing pip to upgrade itself to latest (not suggested if your pip was installed from an RPM, of course) fixed the issue:

✘-2 ~/repos/test [master|✔] 
15:51 $ sudo pip install --upgrade pip
Collecting pip
  Downloading pip-9.0.1-py2.py3-none-any.whl (1.3MB)
    100% |████████████████████████████████| 1.3MB 535kB/s 
Installing collected packages: pip
  Found existing installation: pip 8.1.2
    Uninstalling pip-8.1.2:
      Successfully uninstalled pip-8.1.2
Successfully installed pip-9.0.1
✔ ~/repos/test [master|✔] 
15:51 $ pip -V
pip 9.0.1 from /usr/lib/python2.7/site-packages (python 2.7)
✔ ~/repos/test [master|✔] 
15:51 $ pip install -r source/requirements.txt -t source/vendor
Collecting jinja2 (from -r source/requirements.txt (line 1))
  Using cached Jinja2-2.9.6-py2.py3-none-any.whl
Collecting MarkupSafe>=0.23 (from jinja2->-r source/requirements.txt (line 1))
  Using cached MarkupSafe-1.0.tar.gz
Installing collected packages: MarkupSafe, jinja2
  Running setup.py install for MarkupSafe ... done
Successfully installed MarkupSafe-1.0 jinja2-2.9.6

[pradyunsg]

Hey! It seems like it's been a while since this issue was active. I've spent some time fiddling with this option so, it might be that this has been fixed along the line.

Could someone try reproducing this issue on the current master?

Thanks and sorry for the wait.

[rickatnight11]

I'm no longer hitting this issue with Pip 9.0.1 on Fedora 27:

$ pip -V
pip 9.0.1 from /usr/lib/python2.7/site-packages (python 2.7)
$ cat /etc/fedora-release 
Fedora release 27 (Twenty Seven)
$ pip install -r source/requirements.txt -t source/vendor
Collecting jinja2 (from -r source/requirements.txt (line 1))
  Downloading Jinja2-2.10-py2.py3-none-any.whl (126kB)
    100% |████████████████████████████████| 133kB 3.0MB/s 
Collecting MarkupSafe>=0.23 (from -r source/requirements.txt (line 2))
  Downloading MarkupSafe-1.0.tar.gz
Installing collected packages: MarkupSafe, jinja2
  Running setup.py install for MarkupSafe ... done
Successfully installed MarkupSafe-1.0 jinja2-2.10

[pradyunsg]

Great! A second confirmation and I'll be happy to close.

[pradyunsg]

Well, one confirmation and a lack of activity should be enough reason to close. :)

[lock]

This thread has been automatically locked since there has not been any recent activity after it was closed. Please open a new issue for related bugs.