Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

unable to connect to index.crates.io #13457

Open
R-Goc opened this issue Feb 18, 2024 · 21 comments
Open

unable to connect to index.crates.io #13457

R-Goc opened this issue Feb 18, 2024 · 21 comments
Labels
A-interacts-with-crates.io Area: interaction with registries A-networking Area: networking issues, curl, etc. C-bug Category: bug S-needs-info Status: Needs more info, such as a reproduction or more background for a feature request.

Comments

@R-Goc
Copy link

R-Goc commented Feb 18, 2024

Problem

When attempting to download crates, I get a warning: spurious network error (3 tries remaining): [35] SSL connect error (Recv failure: Connection was reset). After the next tries it causes the install to fail with:
Caused by: [35] SSL connect error (Recv failure: Connection was reset)
This started happening on my new windows 11 machine.

Steps

Clean installation on a new windows 11 system, all updates installed. Attempting to install any crate fails.

Possible Solution(s)

No response

Notes

The toolchain used makes no difference. Already tried setting these options in the global config.toml, based on some previous issues:
[source.crates-io]
index = "https://github.com/rust-lang/crates.io-index" # I set the source to the official github index

[http]
proxy = "" # leaving this as a blank string resets proxy to default
http.check-revoke = false
#check-revoke = false

None of which worked. I am not using Windows defender. I am using Eset antivirus as I was on my previous computer, where I didn't have this problem.

Version

cargo 1.78.0-nightly (fc1d58fd0 2024-02-09)
release: 1.78.0-nightly
commit-hash: fc1d58fd0531a57a6b942a14cdcdbcb82ece16f3
commit-date: 2024-02-09
host: x86_64-pc-windows-msvc
libgit2: 1.7.2 (sys:0.18.2 vendored)
libcurl: 8.6.0-DEV (sys:0.4.71+curl-8.6.0 vendored ssl:Schannel)
os: Windows 10.0.22635 (Windows 11 Core) [64-bit]
@R-Goc R-Goc added C-bug Category: bug S-triage Status: This issue is waiting on initial triage. labels Feb 18, 2024
@MrMic
Copy link

MrMic commented Feb 19, 2024

Got the same issue but when I build an image with docker.
The update of index failed due to the same error.
Cannot build Rust App Image anymore ... :(

Any help welcome!
Rgds,

PS: cargo 1.76.0 (c84b367 2024-01-18)

@tusharxoxoxo
Copy link

seems to be an similar issue to this one #7096 i guess

@MrMic
Copy link

MrMic commented Feb 19, 2024

Not at all!
I am on Linux and it is a DNS issue when building an image
with a Rust App. On my host, everything works fine!
It seems that in the Dockerfile, when cargo build is triggered, cargo update
execute first to update the index with crates.io, and it is here I got the issue.
Even more, the initial image is downloaded successfully!

Something is wrong with index.crates.io and their DNS ...

@R-Goc
Copy link
Author

R-Goc commented Feb 19, 2024

For me it turns out that this is caused by curl and not by cargo. Don't know how, but sure.

@CodeDoctorDE
Copy link

CodeDoctorDE commented Feb 20, 2024

Got the same issue on my machine.
OS: Windows 11
Installed via rustup-init.exe
Uninstall doesn't help, config values doesn't help

    Updating crates.io index
warning: spurious network error (3 tries remaining): [35] SSL connect error (Recv failure: Connection was reset)
warning: spurious network error (2 tries remaining): [35] SSL connect error (Recv failure: Connection was reset)
warning: spurious network error (1 tries remaining): [35] SSL connect error (Recv failure: Connection was reset)
error: failed to get `ansi_term` as a dependency of package `cargo-leptos v0.2.7`

Caused by:
  download of config.json failed

Caused by:
  failed to download from `https://index.crates.io/config.json`

Caused by:
  [35] SSL connect error (Recv failure: Connection was reset)

When visiting this website using the browser it works.
But when running curl https://index.crates.io/config.json the same issue appears. I have currently no fix but I'm sure it's an issue on the windows/curl site.

> where.exe curl.exe
C:\Windows\System32\curl.exe

But other websites work with curl

> curl https://google.com

The main site crates.io also doesn't work

> curl https://crates.io
curl: (35) Recv failure: Connection was reset

@MrMic
Copy link

MrMic commented Feb 20, 2024

For me on Linux

$> curl https://crates.io
=> WORKS!

But:
$> curl https://index.crates.io
<Error><Code>NoSuchKey</Code><Message>The specified key does not exist.</Message><Key>index.html</Key><RequestId>EFEJM9PM6CVJRN9V</RequestId><HostId>iXnil6CJiEMRL4+scgXj AgNJyvrloC3rdfJNc7PU1nzBtflcfKlRSGB5/oDisRBT8N1T7BnNPEM=</HostId></Error>%

is NOT working...

@jdno
Copy link
Member

jdno commented Feb 20, 2024

@MrMic The last command works and does exactly what it is supposed to do, namely return HTTP 404 since there is no content hosted at the root of that domain (i.e. /):

$ curl -I https://index.crates.io/
HTTP/2 404

You have to access a specific path to test index.crates.io:

curl -I https://index.crates.io/config.json
HTTP/2 200

The issues you mention with Docker are unrelated to this issue, since your error is Couldn't resolve host name and here it is SSL connect error. I see that you've created rust-lang/rust#121143 and rust-lang/docker-rust#176 to track those.

@jdno
Copy link
Member

jdno commented Feb 20, 2024

For the folks with the SSL error, can you run curl with verbose output? That might reveal more details about the kind of SSL error:

curl -Iv https://index.crates.io/config.json

@MrMic
Copy link

MrMic commented Feb 20, 2024

@jdno You are right about the curl issue; indeed, my DNS issue is still open
and unfortunately not fixed. Something is wrong with the Dockerfile and cargo
build...

@R-Goc
Copy link
Author

R-Goc commented Feb 20, 2024

I located this to the issue being the curl connection, however this only happens on crates.io. It works for every other website.

@jdno
Copy link
Member

jdno commented Feb 21, 2024

Can you run curl -Iv https://index.crates.io/config.json to get more details on the SSL error?

@R-Goc
Copy link
Author

R-Goc commented Feb 21, 2024

Sure, here is the output. I'll preface it with saying that using -H"Pragma: no-cache" doesn't really help. The operation succeeds like 1 in 10 or 20.
The output from a failure:

  • Trying [2600:9000:25e8:a800:1f:a9f5:69c0:93a1]:443...
  • Connected to index.crates.io (2600:9000:25e8:a800:1f:a9f5:69c0:93a1) port 443
  • schannel: disabled automatic use of client certificate
  • ALPN: curl offers http/1.1
  • Recv failure: Connection was reset
  • schannel: failed to receive handshake, SSL/TLS connection failed
  • Closing connection
  • schannel: shutting down SSL/TLS connection with index.crates.io port 443
  • Send failure: Connection was reset
  • schannel: failed to send close msg: Failed sending data to the peer (bytes written: -1)
    curl: (35) Recv failure: Connection was reset

The output from a success:

  • Trying [2600:9000:25e8:a800:1f:a9f5:69c0:93a1]:443...
  • Trying 52.222.169.91:443...
  • Connected to index.crates.io (52.222.169.91) port 443
  • schannel: disabled automatic use of client certificate
  • ALPN: curl offers http/1.1
  • ALPN: server accepted http/1.1
  • using HTTP/1.1

HEAD /config.json HTTP/1.1
Host: index.crates.io
User-Agent: curl/8.4.0
Accept: /

  • schannel: remote party requests renegotiation
  • schannel: renegotiating SSL/TLS connection
  • schannel: SSL/TLS connection renegotiated
    < HTTP/1.1 200 OK
    HTTP/1.1 200 OK
    < Content-Type: application/json
    Content-Type: application/json
    < Content-Length: 76
    Content-Length: 76
    < Connection: keep-alive
    Connection: keep-alive
    < Last-Modified: Mon, 23 May 2022 19:23:53 GMT
    Last-Modified: Mon, 23 May 2022 19:23:53 GMT
    < x-amz-version-id: OLJqOQ3bTJ8rGepGZwmcEkt2gR_Fm1bY
    x-amz-version-id: OLJqOQ3bTJ8rGepGZwmcEkt2gR_Fm1bY
    < Accept-Ranges: bytes
    Accept-Ranges: bytes
    < Server: AmazonS3
    Server: AmazonS3
    < Date: Wed, 21 Feb 2024 19:38:52 GMT
    Date: Wed, 21 Feb 2024 19:38:52 GMT
    < ETag: "7a39851bdb021fcef53172317860f970"
    ETag: "7a39851bdb021fcef53172317860f970"
    < Vary: Accept-Encoding
    Vary: Accept-Encoding
    < X-Cache: Hit from cloudfront
    X-Cache: Hit from cloudfront
    < Via: 1.1 cf82d48fdf484813132bbd9c90904672.cloudfront.net (CloudFront)
    Via: 1.1 cf82d48fdf484813132bbd9c90904672.cloudfront.net (CloudFront)
    < X-Amz-Cf-Pop: CDG52-P2
    X-Amz-Cf-Pop: CDG52-P2
    < X-Amz-Cf-Id: 2VaixxGo5GWVsJaReNE98OmQGBgMVzxMxHuhwUmjeXXkArZejCXOsw==
    X-Amz-Cf-Id: 2VaixxGo5GWVsJaReNE98OmQGBgMVzxMxHuhwUmjeXXkArZejCXOsw==
    < Age: 2222
    Age: 2222

<

  • Connection #0 to host index.crates.io left intact

@weihanglo weihanglo added A-interacts-with-crates.io Area: interaction with registries S-needs-info Status: Needs more info, such as a reproduction or more background for a feature request. A-networking Area: networking issues, curl, etc. and removed S-triage Status: This issue is waiting on initial triage. labels Feb 21, 2024
@kdopen
Copy link

kdopen commented Feb 22, 2024

I can confirm @R-Goc 's output (#13457 (comment))

Running on Ubuntu 22.04, curl 7.81, openssl 3.x

Oddly, if I use the -Iv keys it nearly always works. If I drop the -I and just use -v it nearly fails about 50% of the time.

Success

$ curl -v https://index.crates.io/config.json
*   Trying 18.164.116.95:443...
* Connected to index.crates.io (18.164.116.95) port 443 (#0)
* ALPN, offering h2
* ALPN, offering http/1.1
*  CAfile: /etc/ssl/certs/ca-certificates.crt
*  CApath: /etc/ssl/certs
* TLSv1.0 (OUT), TLS header, Certificate Status (22):
* TLSv1.3 (OUT), TLS handshake, Client hello (1):
* TLSv1.2 (IN), TLS header, Certificate Status (22):
* TLSv1.3 (IN), TLS handshake, Server hello (2):
* TLSv1.2 (IN), TLS header, Finished (20):
* TLSv1.2 (IN), TLS header, Supplemental data (23):
* TLSv1.3 (IN), TLS handshake, Encrypted Extensions (8):
* TLSv1.2 (IN), TLS header, Supplemental data (23):
* TLSv1.3 (IN), TLS handshake, Certificate (11):
* TLSv1.2 (IN), TLS header, Supplemental data (23):
* TLSv1.3 (IN), TLS handshake, CERT verify (15):
* TLSv1.2 (IN), TLS header, Supplemental data (23):
* TLSv1.3 (IN), TLS handshake, Finished (20):
* TLSv1.2 (OUT), TLS header, Finished (20):
* TLSv1.3 (OUT), TLS change cipher, Change cipher spec (1):
* TLSv1.2 (OUT), TLS header, Supplemental data (23):
* TLSv1.3 (OUT), TLS handshake, Finished (20):
* SSL connection using TLSv1.3 / TLS_AES_128_GCM_SHA256
* ALPN, server accepted to use h2
* Server certificate:
*  subject: CN=crates.io
*  start date: Dec 26 00:00:00 2023 GMT
*  expire date: Jan 23 23:59:59 2025 GMT
*  subjectAltName: host "index.crates.io" matched cert's "index.crates.io"
*  issuer: C=US; O=Amazon; CN=Amazon RSA 2048 M02
*  SSL certificate verify ok.
* Using HTTP2, server supports multiplexing
* Connection state changed (HTTP/2 confirmed)
* Copying HTTP/2 data in stream buffer to connection buffer after upgrade: len=0
* TLSv1.2 (OUT), TLS header, Supplemental data (23):
* TLSv1.2 (OUT), TLS header, Supplemental data (23):
* TLSv1.2 (OUT), TLS header, Supplemental data (23):
* Using Stream ID: 1 (easy handle 0x561d774f1a60)
* TLSv1.2 (OUT), TLS header, Supplemental data (23):
> GET /config.json HTTP/2
> Host: index.crates.io
> user-agent: curl/7.81.0
> accept: */*
> 
* TLSv1.2 (IN), TLS header, Supplemental data (23):
* TLSv1.3 (IN), TLS handshake, Newsession Ticket (4):
* TLSv1.2 (IN), TLS header, Supplemental data (23):
* Connection state changed (MAX_CONCURRENT_STREAMS == 128)!
* TLSv1.2 (OUT), TLS header, Supplemental data (23):
* TLSv1.2 (IN), TLS header, Supplemental data (23):
< HTTP/2 200 
< content-type: application/json
< content-length: 76
< last-modified: Mon, 23 May 2022 19:23:53 GMT
< x-amz-version-id: OLJqOQ3bTJ8rGepGZwmcEkt2gR_Fm1bY
< accept-ranges: bytes
< server: AmazonS3
< date: Thu, 22 Feb 2024 17:12:16 GMT
< etag: "7a39851bdb021fcef53172317860f970"
< vary: Accept-Encoding
< x-cache: Hit from cloudfront
< via: 1.1 86a640712a72b4264f1681744fa48612.cloudfront.net (CloudFront)
< x-amz-cf-pop: JFK50-P6
< x-amz-cf-id: uPEO6kFszxuNO2X-bD9vxw9BUzcyi5qE6z2aA4VuWBBE89c9NzvEHw==
< age: 473
< 
{
  "dl": "https://crates.io/api/v1/crates",
  "api": "https://crates.io"
}
* Connection #0 to host index.crates.io left intact

Failure:

$ curl -v https://index.crates.io/config.json
*   Trying 18.164.116.62:443...
* Connected to index.crates.io (18.164.116.62) port 443 (#0)
* ALPN, offering h2
* ALPN, offering http/1.1
*  CAfile: /etc/ssl/certs/ca-certificates.crt
*  CApath: /etc/ssl/certs
* TLSv1.0 (OUT), TLS header, Certificate Status (22):
* TLSv1.3 (OUT), TLS handshake, Client hello (1):
* OpenSSL SSL_connect: Connection reset by peer in connection to index.crates.io:443 
* Closing connection 0
* TLSv1.0 (OUT), TLS header, Unknown (21):
* TLSv1.3 (OUT), TLS alert, decode error (562):
curl: (35) OpenSSL SSL_connect: Connection reset by peer in connection to index.crates.io:443 

Failure for me is happening on a simple cargo build from the command line of a mostly empty (new) repo - just two dependencies.

@Tremoneck
Copy link

Tremoneck commented Mar 9, 2024

I'm using Windows. For me curl 8.4 on Win seems to work if I specify -4. Specifying -6 makes it fail.

* Trying [2600:9000:2670:a600:1f:a9f5:69c0:93a1]:443...
* Connected to index.crates.io (2600:9000:2670:a600:1f:a9f5:69c0:93a1) port 443
* schannel: disabled automatic use of client certificate
* ALPN: curl offers http/1.0
* Send failure: Connection was reset
* schannel: failed to send initial handshake data: sent -1 of 201 bytes
* Closing connection
* schannel: shutting down SSL/TLS connection with index.crates.io port 443
* Send failure: Connection was reset
* schannel: failed to send close msg: Failed sending data to the peer (bytes written: -1)
* 
* 

Looking at it with wireshark, the handshake completes, but then the server immediately sends a RST Frame. The Client doesn't get the opportunity to send the tls client hello.

WSL curl 7.68.0 -4 works. v6 isn't configured for v6 so cargo defaults to ipv4 so it works. For me the workaround seems to be to disable ipv6 in my network adapter, at least when I need cargo to run. Is there an option to force cargo to v4?

At least for @R-Goc the failure is v6 and the success v4 as well. I don't think the other errors are of the same cause as they show a different error. @kdopen is able to send a Client Hello as far as I can tell.

@mjptree
Copy link

mjptree commented Mar 10, 2024

I believe this may have to do with Windows packet filtering. I have a Windows device and a Linux device in the same physical network, and I only observe the described problems from the Windows device. I looked at the Windows Event Viewer under Windows Logs/Security and noticed that the inbound packets from the IP(v6) address of index.crates.io generated Audit Failures. While the outbound request (obviously succeeded), the inbound responses where dropped by the packet filter.

In WireShark however, they appear as RSTs from the remote address, right after the client hello. This is odd, because the server keeps retransmitting SYN,ACKs a couple of times after the RST. Not sure how packet filtering would show up in WireShark, and if that looks from the application side as a RST. Most of the time the error manifests as a Recv failure. A few times it manifested as a Send failure, although I was not able to observe the firewall or packet filter behaviour in these cases.

About 5 - 10% of requests go through without being dropped. Based on the pcap, I cannot see any difference between the connection that goes through and the one that is imediately being RSTd, although the successful ones do not generate audit events. Allowing the application to communicate through Windows Defender Firewall, does not appear to help (added two inbound ALLOW rules for both TCP and UDP).

I suspect the issue to be related to IPv4/v6 differences. E.g. postman, which always uses IPv4, always succeeds for https://index.crates.io/config.json. Invoke-WebRequest -Uri https://index.crates.io/config.json works spuriously but appears to be able to recover from these errors if executed multiple times in short sequence, even when all requests (the failing and the successful ones) use the same remote IPv6 address.

@kinxyo
Copy link

kinxyo commented Mar 21, 2024

If someone is still going through this issue on Windows, please try by adding registries.crates-io.protocol = "git" to /.cargo/config.toml. It worked for me.

@Tremoneck
Copy link

If someone is still going through this issue on Windows, please try by adding registries.crates-io.protocol = "git" to /.cargo/config.toml. It worked for me.

I tried that before I found the workaround for me with ipv4 only. Getting the index worked. Downloading the crates didn't work for me.

@R-Goc
Copy link
Author

R-Goc commented Mar 31, 2024

How would you set cargo to use ipv4?

@yakrider
Copy link

a (clearly suboptimal) workaround is to make cargo use git for the protocol, and git-cli for its git work .. the CLI can be whatever you have installed and it usually uses ipv4 instead of ipv6 .. so ..

cargo-net.git-fetch-with-cli = true
registries.crates-io.protocol = "git"

in cargo.toml should be enough for that ..

now if whatever git-cli you have is still doing git via ipv6, you could configure it with something like :

git config --global url."https://github.com/".insteadOf git://github.com/
git config --global url."https://".insteadOf git://

there's also the more intrusive way of just going into windows network adapter advanced setting, finding the ipv4 and ipv6 specific advanced properties and configuring the metric manually to give ipv4 lower (i.e higher preference) metric compared to ipv6 instead of the typically selected 'Automatic metric' checkbox for both of them .. in theory that should make windows prefer the lower-metric option

and for those who might only occasionally run into the issue, you could prob simply get by increasing the number or retries cargo makes before it gives up .. by setting up CARGO_NET_RETRY=10 etc in the env (instead of the default 3)

(and for those who know what they're doing and find that it is indeed a ssl cert revocation issue for them, you could add CARGO_HTTP_CHECK_REVOKE=false to your env too)

@TheArchitect4855
Copy link

I can confirm that this is a bug with Windows - cargo is doing nothing wrong. Turning Windows Defender Firewall off has fixed the issue for me (however this may not be advisable). The firewall has been breaking other apps for me as well (mainly the Helldivers 2 updater), so I'm not too upset about turning this feature off.

If anyone has the guts to brave Microsoft's bug reporting process, just know that you have my support.

@R-Goc
Copy link
Author

R-Goc commented May 13, 2024

I now tried again and everything seems to be working perfectly. I don't know what fixed it sadly. I didn't change any network settings. My windows version is newer now, but I also updated cargo. Current version:
cargo 1.80.0-nightly (4de0094 2024-05-09)
release: 1.80.0-nightly
commit-hash: 4de0094
commit-date: 2024-05-09
host: x86_64-pc-windows-msvc
libgit2: 1.7.2 (sys:0.18.3 vendored)
libcurl: 8.6.0-DEV (sys:0.4.72+curl-8.6.0 vendored ssl:Schannel)
os: Windows 10.0.22635 (Windows 11 Core) [64-bit]

I don't know if I should close this issue or not.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
A-interacts-with-crates.io Area: interaction with registries A-networking Area: networking issues, curl, etc. C-bug Category: bug S-needs-info Status: Needs more info, such as a reproduction or more background for a feature request.
Projects
None yet
Development

No branches or pull requests