Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Cargo doesn't respect git's http.sslverify #1180

Open
alexcrichton opened this issue Jan 16, 2015 · 36 comments
Open

Cargo doesn't respect git's http.sslverify #1180

alexcrichton opened this issue Jan 16, 2015 · 36 comments
Labels
A-git Area: anything dealing with git S-triage Status: This issue is waiting on initial triage.

Comments

@alexcrichton
Copy link
Member

This used to be supported in libgit2, but it was removed. We should probably add back in explicit support somewhere.

cc #636, this has come up once or twice there.

@alexcrichton alexcrichton added the A-git Area: anything dealing with git label Jan 16, 2015
@alexcrichton
Copy link
Member Author

This will likely be done by binding the certificate_check callback in remote_callbacks.rs of git2-rs and then using that in Cargo itself

@yug
Copy link

yug commented Jul 10, 2015

It seems a little bit complicated.

We are behind the R&D proxy, the company classifies the connection zones, we are unfortunately in a zone we can git clone the outside code, no write access to outside.

Cargo should get work by hacked locally, however, the process is that it firstly download a pre-built binary, that init binary couldn't work, so, all things blocked. I am not sure Cargo support cross-compile, if it does not we had to find a linux 64bit box without proxy access problem, build a hacked cargo, substitute the pre-built with that hacked, then get whole things work.

BTW, there should be configuration item to tell cargo to ignore the SSL_certificate_check, rather than people to hack an ad-hoc version

The problem is not solved, I will try it and update feedback here

@the-shank
Copy link

@yug any updates on this one? I am behind a http proxy and am unable to use cargo

@thebitllc
Copy link

I hardcoded the hostnames on my windows box and it worked with tinyproxy/proxifier:

c:\windows\system32\drivers\etc\hosts

174.129.244.175 www.crates.io crates.io s3-us-west-1-r-w.amazonaws.com

@alexcrichton
Copy link
Member Author

@thebitllc hm that's interesting, if you don't have that hardcoding, does crates.io resolve to a different IP than it normally does?

@chrisxue815
Copy link

I managed to bypass this problem by setting up a crates.io proxy and a local index repo. See detailed description in chrisxue815/crates.io-proxy.

@guillon
Copy link

guillon commented Feb 22, 2016

Related issue on my side, actually my company has a HTTP/HTTPS proxy service which fakes SSL certificates for doing HTTPS session monitoring (i.e. decrypts https on the fly). To achieve this the proxy provider fakes https certificates by returning its own certificate to the client.
This is quite unfortunate and forces us to always do HTTPS fetches with no server certificate checks.

Hence it would be great to have a cargo option similar to the --no-check-certificate flag of wget (or --insecure flag of curl, or as mentionned, sslVerify=false configuration of git) when downloading from crates.io.
Whatever the implementation in the background, I guess it would be better to have the flag honored independently of the actual git config as the problem is not specific to git fetches, but also to bare downloads.

@alexcrichton
Copy link
Member Author

One question here is how Cargo may want to export this. It'd undeniably pretty questionable security practice to ignore certificate checks, but the use cases brought up here are indeed legitimate. We may want to do something beyond http.sslVerify or the GIT_SSL_NO_VERIFY environment variable (e.g. also require cargo build --insecure), but if git itself has this sort of configuration then we arguably should just go ahead and respect it as well

@the-shank
Copy link

My case is the exact same as @guillon ... Behind a company wide proxy which fakes SSL certificates and am unable to use (and to ask others to use) cargo. Will give the crates.io-proxy thing a go but still, it would be so much better if this is resolved.

@guillon
Copy link

guillon commented Feb 23, 2016

@alexcrichton, I guess that if one provides a --insecure (or --no-check-certificate) flag, it would be natural for the user to expect that cargo will force the skip of certificate check for all transport connection initiated (git, curl or whatever application level client used by cargo to get artifacts from the network).
Hence I guess that if this flag is provided it should force this setting for file download and git fetches initiated as part of the cargo command.
Thus, actually overriding the defauly git setting if any or download client setting if any.

If, independently of this flag, cargo uses a git client which honors the git configuration, no issue with this, i.e. the user may expect that if he sets a git configuration a git client may honor it. It is not cargo responsibility in this case.
Same for instance if cargo would explicitly mention that the download client is curl (I don't know actually), in this case the ~/.curlrc configuration could be also used by the user.

In summary, it would be good if cargo uses git/download libraries that honor "standard" git/download configurations.
But in any case it seems to me that providing a flag and thus forcing the behavior in anycase is a must have and more maintainable solution (i.e. it becomes a feature of cargo itself and thus gives a guarantee of reliability).

@aldanor
Copy link

aldanor commented Feb 23, 2016

@the-shank @guillon Ditto, same case here, our company-wide proxy fakes SSL certificates, so we are basically forced to use wget --no-check-certificate, curl --insecure, etc, and sslVerify = false for git. And then of course it's a major PITA when a library/package/build systen doesn't comply with the configuration of standard tools like git.

I would, too, expect cargo to use whatever's listed in git configuration by default (if the user configured it in some specific way, that's the ground truth?), but maybe also provide a way to manually override this setting (--insecure) explicitly.

alexcrichton added a commit to alexcrichton/cargo that referenced this issue Feb 23, 2016
Git supports the `http.sslVerify` option as well as the `GIT_SSL_NO_VERIFY`
environment variable for disabling validation of SSL certificates. This was
removed from libgit2 in libgit2/libgit2@41698f22 citing bad security practice
and how applications now had enough information to decide for themselves.

There seems to be enough corporate environments (as mentioned in rust-lang#1180) where
certificate validation won't work that this seems prudent for Cargo to support.

Closes rust-lang#1180
@alexcrichton
Copy link
Member Author

I've pushed this up as #2408

@guillon
Copy link

guillon commented Feb 24, 2016

Great.
Note though, that this does not solve the file download transport as mentionned above, only the git transport.

This issue was initially related to the git config only thus the proposed fix seems sufficient for this issue, should we create another issue for the file download case?

For instance with the revision from pull request #2408, on a network with faked certificates:
$ cargo --version
cargo 0.9.0 (879766a 2016-02-23)
$ cargo build --verbose
Downloading bitflags v0.3.3
unable to get packages from source

Caused by:
failed to download package bitflags v0.3.3 from https://crates.io/api/v1/crates/bitflags/0.3.3/download

Caused by:
Peer certificate cannot be authenticated with given CA certificates

@alexcrichton
Copy link
Member Author

@guillon indeed! Out of curiosity, do you know if there's standard environment variable/config file configuration for curl to ignore certificates? I know there's a flag, but I'd hope to avoid a flag for now...

@guillon
Copy link

guillon commented Feb 24, 2016

@alexcrichton I do not now for env vars. Though, you have a ~/.curlrc file with the insecure option set as in:
$ cat $HOME/.curlrc
insecure

The curl command line will read it.
Note though that while the command line curl honors it, I'm not sure whether libcurl does it also.

@aldanor
Copy link

aldanor commented Feb 24, 2016

Yea, the best you can do is probably see if ~/.curlrc has an insecure line in it. I don't think curl picks up any environment variables.

@alexcrichton
Copy link
Member Author

Those who are running into this problem, one question that came up when @wycats and I were talking about this was why can't a local CA root be added? This is so blatantly a bad security practice it seems like the "standard way" of doing this would be for companies to install some trusted root CA which Cargo should then used to verify connections.

@guillon
Copy link

guillon commented Mar 11, 2016

Yes.
Actually the two cases are

  • repos/files hosted on self-signed certificate hosts, HTTPS proxy or not
  • HTTPS proxies with faked certificate (i.e. a private certificate returned for all sites), generally signed with a company or third party specific root CA.

In the second case, indeed, one should install the root CA to solve the issue or the company IT should do it.
Still, it remains more complicated when trying to set up containers or VMs, i.e. need a specific case for providing the root CA and installing it in the container, etc...
Thus we tend to go the easier way, indeed, and deactivate certs checks when possible for non production work.
=> the absurd idea of replacing real certs by actually making man-in-the-middle "the security guarant" lead us to absurdities also...

@alexcrichton
Copy link
Member Author

It was also pointed out that we could also make it easier to use a custom store of certificates through standard mechanisms if modifying the "root store" is too hard:

@aligature
Copy link

I'm new to rust, but I'm coming from the perspective of just trying to experiment / introduce it in our corporate environment. We have the same HTTPS proxy substituting certificates issue that's presented above.

The strategy of adding the private CA into the right spots in the OS makes sense in general, but isn't always practical in a corporate environment. This is true for policy reasons, not just technical. In our case, we don't have root access to our machines and trying to convince teams of people to make certificate changes for a "language experiment" is definitely an uphill battle. In the end, lack of an alternative solution just increases the barrier to entry for trying rust out.

From my point of view, either allowing me to force disable the certificate check or side load and point at our private CA certificate file would be workable solutions.

@aldanor
Copy link

aldanor commented Apr 15, 2016

I'm in exactly the same spot, was trying to introduce Rust to the team at work until we've hit this issue -- without root access (as is quite often the case) it's too much of a hassle to fix in a corporate environment and in some cases it's just plain implausible. This is pretty much the only reason we had to give up on Rust for now, at least until there's a solution that works (and doesn't require root rights).

@lilianmoraru
Copy link

I am also in a corporate environment.
On Linux it works through the proxy but on Windows it doesn't.
On Windows we use "wpad.dat"(Web Proxy Autodiscovery), even git clone doesn't work on Windows(with http_proxy and https_proxy setup)...
Web browsers work all ok.
Tried to call cargo(from MSYS2) with http_proxy=[the proxy] https_proxy=[the proxy] cargo build and it doesn't work.

@the-shank
Copy link

the-shank commented May 11, 2016

@alexcrichton I understand that support for using cargo behind company wide proxy that fakes SSL Certificates isn't going to land into cargo as it is insecure practice. However, we (as in me) could fork cargo and modify that fork to support working behind company wide proxies. This would atleast enable folks like me (and @guillon, @aldanor, @aligature etc.) to work with rust and introduce it at our work places.

The issue is that I am not very familiar with cargo. Could you give some pointers as to how to go about this.

We can even name this fork "cargo-insecure" 😉

@apaprocki
Copy link

@alexcrichton What are the next steps for adding support for alternative cert stores as mentioned in #1180 (comment) ?

@alexcrichton
Copy link
Member Author

@apaprocki Cargo should already respect platform-specific conventions like the environment variables SSL_CERT_FILE and SSL_CERT_DIR, so it may just be a matter of configuring those. Otherwise for a more cross-platform solution it'll likely involve binding libcurl's certificate information and then adding Cargo-specific configuration to inform libcurl of the various possible certificate stores.

@DemiMarie
Copy link

Using a company-wide proxy is not necessarily insecure, provided that the proxy checks TLS certificates AND the proxy's certificate is on the client machines.

In the relevant cases here, the proxy's certificate is known, so there should be a way to tell Cargo to use it.

@alexcrichton
Copy link
Member Author

@DemiMarie yeah I think that's definitely the next step to take on this issue, ensuring that you can configure the certificates that Cargo uses and it then communicates that to all of its components. It's also be great to read this configuration from a standard location if it's already there as well.

@worker2345234
Copy link

Very said that it is so complicated to use cargo behind our company wide proxy. I set up a localhost cntlm proxy for authentification. git works fine together with the cntlm, but still I need "sslVerify=false" in the git config.

I guess rust will have a serious problem to get used in company environements.
Should be possible to give rust a quick try without struggeling around with proxys and certificates.
By the way I don't understand why cargo shall be more secure than git...

alexcrichton added a commit to alexcrichton/cargo that referenced this issue Jul 25, 2016
This adds a `http.cainfo` option to Cargo which reads CA information from a
bundle to pass through to the underlying SSL implementation. This should allow
configuration of Cargo in situations where the default certificate store doesn't
contain the relevant certificates, such as behind corporate proxies.

cc rust-lang#1180
@alexcrichton
Copy link
Member Author

Ok, I've opened a PR (#2917) for adding http.cainfo config which should allow configuring Cargo to use a specific CA bundle for HTTP connections. If Cargo is then configured with an HTTP proxy, then all traffic should be routed with that cainfo information.

If you're on Linux you can also do that today by setting the SSL_CERT_FILE environment variable pointing at your cainfo file, but that won't work on OSX and Windows as OpenSSL isn't used there.

bors added a commit that referenced this issue Jul 26, 2016
Add `http.cainfo` config for custom certs

This adds a `http.cainfo` option to Cargo which reads CA information from a
bundle to pass through to the underlying SSL implementation. This should allow
configuration of Cargo in situations where the default certificate store doesn't
contain the relevant certificates, such as behind corporate proxies.

cc #1180
@alexcrichton
Copy link
Member Author

Now that #2917 is merged, if anyone runs into this issue, you can try to add:

[http]
cainfo = "path/to/cert.pem"

to your .cargo/config file (probably next to http.proxy. I've tested this on OSX/Linux, but I suspect Windows will work as well!

@yoava333
Copy link

@alexcrichton unfortunately Windows doesn't work with the new cainfo option.
Looking at the source it seems that CURLOPT_CAINFO is not supported with winssl.
I've tried to install the certificate (tested with chrome) but cargo is still unable to download crates.
I've tested this on windows 10 (10586.494) with cargo 0.13.0-nightly (c205132 2016-08-09).

@alexcrichton
Copy link
Member Author

Ah yeah I've since learned that unfortunately Windows handles certificate
trust in a very different fashion so this operation is a nop in curl on
Windows.

On Sunday, August 14, 2016, yoava333 notifications@github.com wrote:

@alexcrichton https://github.com/alexcrichton unfortunately Windows
doesn't work with the new cainfo option.
Looking at the source it seems that CURLOPT_CAINFO is not supported with
winssl.
I've tried to install the certificate (tested with chrome) but cargo is
still unable to download crates.
I've tested this on windows 10 (10586.494) with cargo 0.13.0-nightly (
c205132
c205132
2016-08-09).


You are receiving this because you were mentioned.
Reply to this email directly, view it on GitHub
#1180 (comment),
or mute the thread
https://github.com/notifications/unsubscribe-auth/AAD95BTiEvq0yY0T8EfP7OPzeFiN0it5ks5qfvskgaJpZM4DTdrr
.

@petrochenkov
Copy link
Contributor

So, http.cainfo wants a Certificate Authority bundle to point to.
In case you are wondering "what on earth is a 'Certificate Authority bundle' and where do I take one?", the working answer may be* the curl's site (cacert.pem).

* At least if you are on a server with sufficiently old version of some enterprise Linux, in this case the proxy may be a red herring.
(I spent too much time on this today.)

@dwijnand
Copy link
Member

With the "git the cli" config option (I forget what it is exactly) available as an alternative, do we still care to resolve this anyways?

@alexcrichton
Copy link
Member Author

I think we'll still want to consider this yeah for all of Cargo's other interactions, as there's often https issues with crates.io as well

@lihz6
Copy link

lihz6 commented Jan 22, 2020

If git is working, then open ~/.cargo/config and write:

[net]
git-fetch-with-cli = true

https://doc.rust-lang.org/cargo/reference/config.html

@epage epage added the S-triage Status: This issue is waiting on initial triage. label May 3, 2023
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
A-git Area: anything dealing with git S-triage Status: This issue is waiting on initial triage.
Projects
None yet
Development

No branches or pull requests