New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

pip 7.0.1: you should use "--trusted-host". Hey, no such option "--trusted-host"! #2822

Closed
the-efi opened this Issue May 25, 2015 · 23 comments

Comments

Projects
None yet
@the-efi

the-efi commented May 25, 2015

$ cat req.txt 
--extra-index-url http://pip.mycompany.com/simple 
mylib
myanotherlib

$ pip install -r req.txt
Collecting mylib (from -r req.txt (line 2))
.../urllib3/util/ssl_.py:90: InsecurePlatformWarning: A true SSLContext object is not available. This prevents urllib3 from configuring SSL appropriately and may cause certain SSL connections to fail. For more information, see https://urllib3.readthedocs.org/en/latest/security.html#insecureplatformwarning.
  InsecurePlatformWarning
  The repository located at pip.mycompany.com is not a trusted or secure host and is being ignored. If this repository is available via HTTPS it is recommended to use HTTPS instead, otherwise you may silence this warning and allow it anyways with '--trusted-host pip.mycompany.com'.
  Could not find a version that satisfies the requirement mylib (from -r req.txt (line 2)) (from versions: )
No matching distribution found for mylib (from -r req.txt (line 2))

$ cat req1.txt 
--extra-index-url http://pip.mycompany.com/simple 
--trusted-host pip.mycompany.com
mylib
myanotherlib

$ pip install -r req1.txt 
Usage: pip [options]

pip: error: no such option: --trusted-host
$ 

I know that i can run pip install -r file.txt --trusted-host=mycompany.com but I configure my servers with Chef, not running pip directly.

@the-efi

This comment has been minimized.

Show comment
Hide comment
@the-efi

the-efi May 27, 2015

Just wanted to note that the change to no longer allow insecure transport with no option to whitelist a host causes quite large amount of suffering on our side.

We have lots of internal libraries that are hosted on an internal pypi server (I guess it's "external" in pip-talk). Issuing a certificate won't help us, since sometimes it's resolved using client1.pip.mycompany.com and issuing certificates on the fly would be a huge pain.

Whitelisting all libraries with something like the following would also be inconvenient:

--extra-index-url http://pip.mycompany.com/simple 
--allow-external mylib
--allow-unverified mylib
mylib
--allow-external mylib2
--allow-unverified mylib2
mylib2
#etc.

A better solution at this stage would be just to downgrade all our pip usage to <7.0. Unfortunately, it's hard to do as well, since verses like this are used all across the place:

# chef configuration file
Chef::Log.info("About to upgrade pip & setuptools version to latest.")
%W{setuptools pip wheel}.each do |pkg|
  python_pip pkg do
    action :upgrade
  end
end

the-efi commented May 27, 2015

Just wanted to note that the change to no longer allow insecure transport with no option to whitelist a host causes quite large amount of suffering on our side.

We have lots of internal libraries that are hosted on an internal pypi server (I guess it's "external" in pip-talk). Issuing a certificate won't help us, since sometimes it's resolved using client1.pip.mycompany.com and issuing certificates on the fly would be a huge pain.

Whitelisting all libraries with something like the following would also be inconvenient:

--extra-index-url http://pip.mycompany.com/simple 
--allow-external mylib
--allow-unverified mylib
mylib
--allow-external mylib2
--allow-unverified mylib2
mylib2
#etc.

A better solution at this stage would be just to downgrade all our pip usage to <7.0. Unfortunately, it's hard to do as well, since verses like this are used all across the place:

# chef configuration file
Chef::Log.info("About to upgrade pip & setuptools version to latest.")
%W{setuptools pip wheel}.each do |pkg|
  python_pip pkg do
    action :upgrade
  end
end

@dstufft dstufft added this to the 7.0.2 milestone May 27, 2015

@smira

This comment has been minimized.

Show comment
Hide comment
@smira

smira May 30, 2015

I believe --trusted-host should go to pip.conf or to command line. It should not be part of requirements.txt.

smira commented May 30, 2015

I believe --trusted-host should go to pip.conf or to command line. It should not be part of requirements.txt.

@cknoblauch

This comment has been minimized.

Show comment
Hide comment
@cknoblauch

cknoblauch Jun 1, 2015

As @smira says, you can add trusted-host as a configuration option to your pip.conf; I just confirmed it works with our internal pip sever. You can set the env var PIP_CONFIG_FILE to point to a specific file if you can't set it up at a user/sytem level; see the Pip docs for more info.

cknoblauch commented Jun 1, 2015

As @smira says, you can add trusted-host as a configuration option to your pip.conf; I just confirmed it works with our internal pip sever. You can set the env var PIP_CONFIG_FILE to point to a specific file if you can't set it up at a user/sytem level; see the Pip docs for more info.

@dstufft dstufft modified the milestones: 7.1, 7.0.2 Jun 1, 2015

@the-efi

This comment has been minimized.

Show comment
Hide comment
@the-efi

the-efi Jun 2, 2015

@smira, using a command line is not an option for us, as we run chef recipes that may not even support it. chef.conf might or might not work as sudo pip install -r reqs.txt may look for that in a different place than pip install. Using PIP_CONFIG_FILE may create permission issues.

Moreover, I can't see what problem would it create if the --trusted-host would be an acceptable option for requirements files, and what makes --extra-index-url a valid option for requirements files, while making --trusted-host an invalid one from your point of view?

the-efi commented Jun 2, 2015

@smira, using a command line is not an option for us, as we run chef recipes that may not even support it. chef.conf might or might not work as sudo pip install -r reqs.txt may look for that in a different place than pip install. Using PIP_CONFIG_FILE may create permission issues.

Moreover, I can't see what problem would it create if the --trusted-host would be an acceptable option for requirements files, and what makes --extra-index-url a valid option for requirements files, while making --trusted-host an invalid one from your point of view?

@dstufft

This comment has been minimized.

Show comment
Hide comment
@dstufft

dstufft Jun 2, 2015

Member

I think it's fine for pip to add it to requirements.txt, the fact it isn't there is mostly an oversight.

Member

dstufft commented Jun 2, 2015

I think it's fine for pip to add it to requirements.txt, the fact it isn't there is mostly an oversight.

@trcarden

This comment has been minimized.

Show comment
Hide comment
@trcarden

trcarden Jun 5, 2015

We also would like to have the trusted host live in the requirements.txt file. The error message says we need to add it but when we do it throws an error saying the option "--trusted-host" doesn't exist.

trcarden commented Jun 5, 2015

We also would like to have the trusted host live in the requirements.txt file. The error message says we need to add it but when we do it throws an error saying the option "--trusted-host" doesn't exist.

@the-efi

This comment has been minimized.

Show comment
Hide comment
@the-efi

the-efi Jun 8, 2015

We have bought a wildcard certificate for our site, and it seems that the root CA certificate is not trusted by urllib3 by default.

So actually, even with a wildcard certificate in place, we still need to revert to using --trusted-host anyway.

the-efi commented Jun 8, 2015

We have bought a wildcard certificate for our site, and it seems that the root CA certificate is not trusted by urllib3 by default.

So actually, even with a wildcard certificate in place, we still need to revert to using --trusted-host anyway.

@dstufft dstufft modified the milestones: 7.1, 7.2 Jun 30, 2015

@maharg101

This comment has been minimized.

Show comment
Hide comment
@maharg101

maharg101 Jul 9, 2015

Using puppet here, simply write your pip.conf into the venv.

maharg101 commented Jul 9, 2015

Using puppet here, simply write your pip.conf into the venv.

@myoung8

This comment has been minimized.

Show comment
Hide comment
@myoung8

myoung8 Jul 14, 2015

+1 for adding --trusted-host to requirements files

myoung8 commented Jul 14, 2015

+1 for adding --trusted-host to requirements files

@trcarden

This comment has been minimized.

Show comment
Hide comment
@trcarden

trcarden Jul 14, 2015

@dstufft Isn't this just an inconsistency in pip? My understanding is that if you can place the option on the commandline you can place it in the requirements.txt file.

trcarden commented Jul 14, 2015

@dstufft Isn't this just an inconsistency in pip? My understanding is that if you can place the option on the commandline you can place it in the requirements.txt file.

@dstufft

This comment has been minimized.

Show comment
Hide comment
@dstufft

dstufft Jul 14, 2015

Member

Not every command line option is available in the requirements.txt, this jsut needs added to the list of options that are supported in requirements.txt.

Member

dstufft commented Jul 14, 2015

Not every command line option is available in the requirements.txt, this jsut needs added to the list of options that are supported in requirements.txt.

@rafaduran

This comment has been minimized.

Show comment
Hide comment
@rafaduran

rafaduran Jul 15, 2015

+1 for --trusted-host, we got same issue while using some of Docker oficial images and internal pypi server.

rafaduran commented Jul 15, 2015

+1 for --trusted-host, we got same issue while using some of Docker oficial images and internal pypi server.

@tshauck

This comment has been minimized.

Show comment
Hide comment
@tshauck

tshauck Jul 29, 2015

+1 for --trusted-host in requirements.txt, also causing some issues for me.

tshauck commented Jul 29, 2015

+1 for --trusted-host in requirements.txt, also causing some issues for me.

@ghost

This comment has been minimized.

Show comment
Hide comment
@ghost

ghost Aug 2, 2015

+1 for --trusted-host ... it's a job stopping for me.

ghost commented Aug 2, 2015

+1 for --trusted-host ... it's a job stopping for me.

@dragon788

This comment has been minimized.

Show comment
Hide comment
@dragon788

dragon788 Aug 5, 2015

What I discovered today is part of the issue may be that "which pip" and "whereis pip" report that I'm using /usr/bin/pip (which on Ubuntu 14.04 is 1.5.4 which does NOT have --trusted-host) and the upgraded pip that I deployed ended up in /usr/local/bin/pip (which is 7.1.0 and does support it just fine).

dragon788 commented Aug 5, 2015

What I discovered today is part of the issue may be that "which pip" and "whereis pip" report that I'm using /usr/bin/pip (which on Ubuntu 14.04 is 1.5.4 which does NOT have --trusted-host) and the upgraded pip that I deployed ended up in /usr/local/bin/pip (which is 7.1.0 and does support it just fine).

xavfernandez added a commit to xavfernandez/pip that referenced this issue Aug 14, 2015

xavfernandez added a commit to xavfernandez/pip that referenced this issue Sep 3, 2015

@hzbd

This comment has been minimized.

Show comment
Hide comment
@hzbd

hzbd Dec 19, 2015

cat ~/.pip/pip.conf
[global]
index-url = http://mirrors.aliyun.com/pypi/simple/

[install]
trusted-host=mirrors.aliyun.com

this should be feasible.

hzbd commented Dec 19, 2015

cat ~/.pip/pip.conf
[global]
index-url = http://mirrors.aliyun.com/pypi/simple/

[install]
trusted-host=mirrors.aliyun.com

this should be feasible.

@XiaoZYang

This comment has been minimized.

Show comment
Hide comment
@XiaoZYang

XiaoZYang Dec 29, 2015

+1 for --trusted-host it bothers a lot typing '--trusted-host foo' all the time

XiaoZYang commented Dec 29, 2015

+1 for --trusted-host it bothers a lot typing '--trusted-host foo' all the time

@dstufft dstufft modified the milestones: 8.0, 7.2 Jan 18, 2016

@tommyjcarpenter

This comment has been minimized.

Show comment
Hide comment
@tommyjcarpenter

tommyjcarpenter Feb 8, 2016

This is critical for us as well, since we use Cloudify (Similar to the Chef problem). We cannot write a conf and cannot control the pip command.

tommyjcarpenter commented Feb 8, 2016

This is critical for us as well, since we use Cloudify (Similar to the Chef problem). We cannot write a conf and cannot control the pip command.

@xavfernandez

This comment has been minimized.

Show comment
Hide comment
@xavfernandez

xavfernandez Feb 8, 2016

Contributor

@tommyjcarpenter --trusted-host is now allowed in requirements.txt files (since pip 8.0.0).

Contributor

xavfernandez commented Feb 8, 2016

@tommyjcarpenter --trusted-host is now allowed in requirements.txt files (since pip 8.0.0).

@tommyjcarpenter

This comment has been minimized.

Show comment
Hide comment
@tommyjcarpenter

tommyjcarpenter Feb 9, 2016

@xavfernandez Thanks! However, pip8 unfortunately does not ship with any standard python, which again creates problems for chef/cloudify envs: http://stackoverflow.com/questions/35295599/python2-which-python-comes-with-pip8

tommyjcarpenter commented Feb 9, 2016

@xavfernandez Thanks! However, pip8 unfortunately does not ship with any standard python, which again creates problems for chef/cloudify envs: http://stackoverflow.com/questions/35295599/python2-which-python-comes-with-pip8

@the-efi

This comment has been minimized.

Show comment
Hide comment
@the-efi

the-efi Feb 9, 2016

Could this help?

# chef configuration file
Chef::Log.info("About to upgrade pip & setuptools version to latest.")
%W{setuptools pip wheel}.each do |pkg|
  python_pip pkg do
    action :upgrade
  end
end

the-efi commented Feb 9, 2016

Could this help?

# chef configuration file
Chef::Log.info("About to upgrade pip & setuptools version to latest.")
%W{setuptools pip wheel}.each do |pkg|
  python_pip pkg do
    action :upgrade
  end
end
@dstufft

This comment has been minimized.

Show comment
Hide comment
@dstufft

dstufft Feb 9, 2016

Member

@tommyjcarpenter I mean, there's not a whole lot we can do about that... We can't go back in time and add the feature to an already released version of pip so your only real options are upgrade pip or wait until Python comes with it.

Member

dstufft commented Feb 9, 2016

@tommyjcarpenter I mean, there's not a whole lot we can do about that... We can't go back in time and add the feature to an already released version of pip so your only real options are upgrade pip or wait until Python comes with it.

@Zacmatix

This comment has been minimized.

Show comment
Hide comment
@Zacmatix

Zacmatix Jun 20, 2016

https://pip.pypa.io/en/stable/reference/pip_install/#requirements-file-format you should probably update (add --trusted-host to the list of options allowed in a requirements file) the docs regarding this particular issue

Zacmatix commented Jun 20, 2016

https://pip.pypa.io/en/stable/reference/pip_install/#requirements-file-format you should probably update (add --trusted-host to the list of options allowed in a requirements file) the docs regarding this particular issue

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment