Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Network problems with gems fetching #2253

Closed
AlexWayfer opened this issue Mar 28, 2018 · 32 comments

Comments

@AlexWayfer
Copy link

@AlexWayfer AlexWayfer commented Mar 28, 2018

I'm having a problem.

My current problem is:

$ gem i browser --verbose
ERROR:  Could not find a valid gem 'browser' (>= 0), here is why:
          Unable to download data from https://rubygems.org/ - timed out (https://api.rubygems.org/specs.4.8.gz)

$ gem update --system --verbose
ERROR:  While executing gem ... (Gem::RemoteFetcher::UnknownHostError)
    timed out (https://api.rubygems.org/specs.4.8.gz)

$ curl -O https://rubygems.org/gems/browser-2.5.3.gem
  % Total    % Received % Xferd  Average Speed   Time    Time     Time  Current
                                 Dload  Upload   Total   Spent    Left  Speed
100 46592  100 46592    0     0  74428      0 --:--:-- --:--:-- --:--:-- 74428

This issue is related to:

  • Network problems
  • Installing a library
  • Publishing a library
  • The command line gem
  • Other

Here are my current environment details:

$ gem env version
2.7.3

I will abide by the code of conduct.

@duckinator

This comment has been minimized.

Copy link
Member

@duckinator duckinator commented Apr 12, 2018

hi,

If you're still having this issue, can you provide the output of running this command?:

curl -vO https://api.rubygems.org/specs.4.8.gz

As well as this command?:

ruby -r rubygems/remote_fetcher -e 'puts(Gem::RemoteFetcher.new.fetch_path("https://api.rubygems.org/specs.4.8.gz") ? "success" : "failure")'
@WedgeSparda

This comment has been minimized.

Copy link

@WedgeSparda WedgeSparda commented Apr 13, 2018

I'm having the same problem. Here's my output for those commands

curl -vO https://api.rubygems.org/specs.4.8.gz

  % Total    % Received % Xferd  Average Speed   Time    Time     Time  Current
                                 Dload  Upload   Total   Spent    Left  Speed
  0     0    0     0    0     0      0      0 --:--:-- --:--:-- --:--:--     0*   Trying 2a04:4e42:600::514...
* TCP_NODELAY set
*   Trying 151.101.194.2...
* TCP_NODELAY set
* Connected to api.rubygems.org (151.101.194.2) port 443 (#0)
* ALPN, offering h2
* ALPN, offering http/1.1
* Cipher selection: ALL:!EXPORT:!EXPORT40:!EXPORT56:!aNULL:!LOW:!RC4:@STRENGTH
* successfully set certificate verify locations:
*   CAfile: /etc/ssl/cert.pem
  CApath: none
* TLSv1.2 (OUT), TLS handshake, Client hello (1):
} [512 bytes data]
* TLSv1.2 (IN), TLS handshake, Server hello (2):
{ [102 bytes data]
* TLSv1.2 (IN), TLS handshake, Certificate (11):
{ [5039 bytes data]
* TLSv1.2 (IN), TLS handshake, Server key exchange (12):
{ [333 bytes data]
* TLSv1.2 (IN), TLS handshake, Server finished (14):
{ [4 bytes data]
* TLSv1.2 (OUT), TLS handshake, Client key exchange (16):
} [70 bytes data]
* TLSv1.2 (OUT), TLS change cipher, Client hello (1):
} [1 bytes data]
* TLSv1.2 (OUT), TLS handshake, Finished (20):
} [16 bytes data]
* TLSv1.2 (IN), TLS change cipher, Client hello (1):
{ [1 bytes data]
* TLSv1.2 (IN), TLS handshake, Finished (20):
{ [16 bytes data]
* SSL connection using TLSv1.2 / ECDHE-RSA-AES128-GCM-SHA256
* ALPN, server accepted to use h2
* Server certificate:
*  subject: C=US; ST=California; L=San Francisco; O=Fastly, Inc.; CN=f2.shared.global.fastly.net
*  start date: Mar 26 22:40:12 2018 GMT
*  expire date: Oct  6 14:45:12 2018 GMT
*  subjectAltName: host "api.rubygems.org" matched cert's "*.rubygems.org"
*  issuer: C=BE; O=GlobalSign nv-sa; CN=GlobalSign CloudSSL CA - SHA256 - G3
*  SSL certificate verify ok.
* Using HTTP2, server supports multi-use
* Connection state changed (HTTP/2 confirmed)
* Copying HTTP/2 data in stream buffer to connection buffer after upgrade: len=0
* Using Stream ID: 1 (easy handle 0x7faf32806000)
> GET /specs.4.8.gz HTTP/2
> Host: api.rubygems.org
> User-Agent: curl/7.54.0
> Accept: */*
> 
* Connection state changed (MAX_CONCURRENT_STREAMS updated)!
< HTTP/2 200 
< x-amz-id-2: dmdpMm1Zh7GoRcd4M+yenfHgEb2+GBEt3YWPqx9lWcuTxWu8bl+jVYefWSmPRGezKBGjyeExqcA=
< x-amz-request-id: 3CEED0923A9CC5FE
< x-amz-replication-status: COMPLETED
< last-modified: Fri, 13 Apr 2018 08:25:30 GMT
< etag: "1aa92c124ad6e4d87316ec18772ca172"
< cache-control: max-age=31536000
< x-amz-meta-surrogate-key: full-index
< x-amz-version-id: 2Oe30418VBqqyOzIjSENhDzSF_CXwQpr
< via: 1.1 varnish
< content-type: application/octet-stream
< x-backend: F_S3 52.218.212.73:443, fastlyshield--shield_ssl_cache_sea1037_SEA 157.52.67.37:443
< accept-ranges: bytes
< date: Fri, 13 Apr 2018 08:28:47 GMT
< via: 1.1 varnish
< age: 194
< x-served-by: cache-sea1037-SEA, cache-cdg8726-CDG
< x-cache: HIT, HIT
< x-cache-hits: 1, 2
< x-timer: S1523608128.514265,VS0,VE0
< vary: Fastly-SSL
< server: RubyGems.org
< content-length: 3541980
< 
{ [2213 bytes data]
100 3458k  100 3458k    0     0  2569k      0  0:00:01  0:00:01 --:--:-- 2569k
* Connection #0 to host api.rubygems.org left intact

And

ruby -r rubygems/remote_fetcher -e 'puts(Gem::RemoteFetcher.new.fetch_path("https://api.rubygems.org/specs.4.8.gz") ? "success" : "failure")'

/Users/Wedge/.rvm/rubies/ruby-2.3.0/lib/ruby/site_ruby/2.3.0/rubygems/remote_fetcher.rb:306:in `rescue in fetch_path': timed out (https://api.rubygems.org/specs.4.8.gz) (Gem::RemoteFetcher::UnknownHostError)
	from /Users/Wedge/.rvm/rubies/ruby-2.3.0/lib/ruby/site_ruby/2.3.0/rubygems/remote_fetcher.rb:284:in `fetch_path'
	from -e:1:in `<main>'
@AlexWayfer

This comment has been minimized.

Copy link
Author

@AlexWayfer AlexWayfer commented Apr 13, 2018

If you're still having this issue

No, I no longer have this problem:

$ gem i browser --verbose
HEAD https://api.rubygems.org/api/v1/dependencies
200 OK
GET https://api.rubygems.org/api/v1/dependencies?gems=browser
200 OK
Getting SRV record failed: DNS result has no information for _rubygems._tcp.api.rubygems.org
GET https://api.rubygems.org/quick/Marshal.4.8/browser-2.5.3.gemspec.rz
200 OK
...

But since @WedgeSparda has — I'll not close this issue for now.

@WedgeSparda

This comment has been minimized.

Copy link

@WedgeSparda WedgeSparda commented Apr 13, 2018

@AlexTalker Did you do something to fix your problem?

@AlexWayfer

This comment has been minimized.

Copy link
Author

@AlexWayfer AlexWayfer commented Apr 13, 2018

@AlexTalker Did you do something to fix your problem?

Sorry, I don't remember. I think no…

Probably, rebooted my Internet-connection (router, computer, etc.).

@WedgeSparda

This comment has been minimized.

Copy link

@WedgeSparda WedgeSparda commented Apr 13, 2018

This is really weird. I can't install anything using RubyGems, always getting a timeout. But If I connect remotely to another computer in another network it works. How can I get more info about this weird problem?

@WedgeSparda

This comment has been minimized.

Copy link

@WedgeSparda WedgeSparda commented Apr 13, 2018

Making more tests. Pings to rubygems.org and api.rubygems.org work perfectly.
Accesing to https://api.rubygems.org/specs.4.8.gz via web browser works too.
It seems that only happens from my terminal. I'm using macOS High Sierra 10.13.4.

Running gem i fastlane --verbose returns a timeout.

ERROR:  Could not find a valid gem 'fastlane' (>= 0), here is why:
          Unable to download data from https://rubygems.org/ - timed out (https://api.rubygems.org/specs.4.8.gz)

Anyone can help me? I'm unable of installing fastlane because of this and need to run a deploy.

Thanks.

@AlexWayfer

This comment has been minimized.

Copy link
Author

@AlexWayfer AlexWayfer commented Apr 13, 2018

Anyone can help me?

Did you try to reboot your computer and router?

@WedgeSparda

This comment has been minimized.

Copy link

@WedgeSparda WedgeSparda commented Apr 13, 2018

@AlexWayfer Yes. No luck.

@AlexWayfer

This comment has been minimized.

Copy link
Author

@AlexWayfer AlexWayfer commented Apr 13, 2018

But If I connect remotely to another computer in another network it works.

Maybe you can use some proxy or VPN temporarily, while problems are here?

@WedgeSparda

This comment has been minimized.

Copy link

@WedgeSparda WedgeSparda commented Apr 13, 2018

So I was able to run the installation using tethering with my phone. Any idea why a specific connection is giving error? Any port that I should open on my router?

@AlexWayfer

This comment has been minimized.

Copy link
Author

@AlexWayfer AlexWayfer commented Apr 13, 2018

@WedgeSparda You can try traceroute rubygems.org and traceroute api.rubygems.org, I guess, and paste outputs here.

@WedgeSparda

This comment has been minimized.

Copy link

@WedgeSparda WedgeSparda commented Apr 13, 2018

@AlexWayfer Sorry, I'm not familiar with traceroute, which options should I run it with?

Version 1.4a12+Darwin
Usage: traceroute [-adDeFInrSvx] [-A as_server] [-f first_ttl] [-g gateway] [-i iface]
	[-M first_ttl] [-m max_ttl] [-p port] [-P proto] [-q nqueries] [-s src_addr]
	[-t tos] [-w waittime] [-z pausemsecs] host [packetlen]
@AlexWayfer

This comment has been minimized.

Copy link
Author

@AlexWayfer AlexWayfer commented Apr 13, 2018

@WedgeSparda Without any option, I think.

@WedgeSparda

This comment has been minimized.

Copy link

@WedgeSparda WedgeSparda commented Apr 13, 2018

@AlexWayfer If I run just traceroute, that's what I'm getting

Version 1.4a12+Darwin
Usage: traceroute [-adDeFInrSvx] [-A as_server] [-f first_ttl] [-g gateway] [-i iface]
	[-M first_ttl] [-m max_ttl] [-p port] [-P proto] [-q nqueries] [-s src_addr]
	[-t tos] [-w waittime] [-z pausemsecs] host [packetlen]
@AlexWayfer

This comment has been minimized.

Copy link
Author

@AlexWayfer AlexWayfer commented Apr 13, 2018

@WedgeSparda Look at host — this is required argument for command. I wrote RubyGems hosts above.

@WedgeSparda

This comment has been minimized.

Copy link

@WedgeSparda WedgeSparda commented Apr 13, 2018

@AlexWayfer Sorry, I missed that. This is what I'm getting:

traceroute: Warning: rubygems.org has multiple addresses; using 151.101.66.2
traceroute to rubygems.org (151.101.66.2), 64 hops max, 52 byte packets
 1  192.168.1.1 (192.168.1.1)  1.390 ms  0.565 ms  0.421 ms
 2  109.230.217.254 (109.230.217.254)  2.840 ms  3.160 ms  2.568 ms
 3  10.3.1.1 (10.3.1.1)  1.871 ms  2.036 ms  1.960 ms
 4  10.255.2.4 (10.255.2.4)  8.921 ms  9.028 ms  9.024 ms
 5  213.27.208.165 (213.27.208.165)  9.157 ms  8.881 ms  8.734 ms
 6  212.74.70.51 (212.74.70.51)  31.261 ms  30.440 ms  31.703 ms
 7  212.74.73.241 (212.74.73.241)  29.560 ms
    te0-0-0-0-pr2.par.router.colt.net (212.74.88.161)  30.952 ms
    212.74.70.51 (212.74.70.51)  30.595 ms
 8  * * *
 9  * * *
10  * * *
11  * *

It just keeps adding lines like the last ones and nothing else happens.

@AlexWayfer

This comment has been minimized.

Copy link
Author

@AlexWayfer AlexWayfer commented Apr 13, 2018

@duckinator, is it helpful?

@AlexWayfer

This comment has been minimized.

Copy link
Author

@AlexWayfer AlexWayfer commented Apr 13, 2018

I don't like this warning:

traceroute: Warning: rubygems.org has multiple addresses; using 151.101.66.2

Here's my output:

traceroute to rubygems.org (151.101.2.2), 30 hops max, 60 byte packets
 1  _gateway (192.168.1.1)  0.632 ms  0.611 ms  0.600 ms
 2  193.104.234.41 (193.104.234.41)  3.612 ms  3.609 ms  3.905 ms
 3  195.16.99.42 (195.16.99.42)  3.889 ms  4.158 ms  4.149 ms
 4  * * *
 5  * * *
 6  * * *
 7  37.29.105.81 (37.29.105.81)  4.795 ms  2.862 ms  9.725 ms
 8  * * *
 9  * * *
10  83.169.204.82 (83.169.204.82)  28.429 ms  28.387 ms  28.641 ms
11  * * *
12  ae-2-3209.edge3.frankfurt1.level3.net (4.69.163.102)  46.961 ms  46.960 ms  47.190 ms
13  * * *
14  * * *
15  * * *
16  * * *
17  * * *
18  * * *
19  * * *
20  * * *
21  * * *
22  * * *
23  * * *
24  * * *
25  * * *
26  * * *
27  * * *
28  * * *
29  * * *
30  * * *

So, I have 151.101.2.2 IP for rubygems.org, you have multiple addresses and using 151.101.66.2 (another). This is strange for me.

Maybe you can do something with your DNS settings?

@WedgeSparda

This comment has been minimized.

Copy link

@WedgeSparda WedgeSparda commented Apr 13, 2018

@AlexWayfer I'm using the Google DNS (8.8.8.8 and 8.8.4.8). Maybe the problem is with them?

@AlexWayfer

This comment has been minimized.

Copy link
Author

@AlexWayfer AlexWayfer commented Apr 13, 2018

@WedgeSparda maybe, I don't know. You can change this and write here the results.

@WedgeSparda

This comment has been minimized.

Copy link

@WedgeSparda WedgeSparda commented Apr 13, 2018

@bronzdoc

This comment has been minimized.

Copy link
Member

@bronzdoc bronzdoc commented Apr 13, 2018

I'm going to close this since the original problem was fixed and it seems the problem is out of the scope of rubygems itself

@bronzdoc bronzdoc closed this Apr 13, 2018
@rrcobb

This comment has been minimized.

Copy link

@rrcobb rrcobb commented Oct 23, 2018

In case anyone else ends up running into this one - I was pretty stumped for a while. Could install fine from a hotspot, but wifi ended up with

$ gem install bundler
ERROR:  Could not find a valid gem 'bundler' (>= 0), here is why:
          Unable to download data from https://rubygems.org/ - timed out (https://api.rubygems.org/specs.4.8.gz)

Eventual fix and explanation came from

Rubygems.org seems like it filters some IPv6 requests from certain domains - not quite sure why.

You can see for yourself with

$ # ipv4 works fine
$ curl -v --head https://api.rubygems.org

$ # ipv6 broken
$ curl -6 -v --head https://api.rubygems.org

or more carefully,

$ dig AAAA api.rubygems.org +short
rubygems.org.
2a04:4e42:400::70
2a04:4e42:600::70
2a04:4e42:200::70
2a04:4e42::70
$ nmap -p 443 -6 -Pn -v -oG - 2a04:4e42:400::70 2a04:4e42:600::70 2a04:4e42:200::70 2a04:4e42::70
# Nmap 7.70 scan initiated Tue Oct 23 02:22:41 2018 as: nmap -p 443 -6 -Pn -v -oG - 2a04:4e42:400::70 2a04:4e42:600::70 2a04:4e42:200::70 2a04:4e42::70
# Ports scanned: TCP(1;443) UDP(0;) SCTP(0;) PROTOCOLS(0;)
Host: 2a04:4e42:400::70 ()	Status: Up
Host: 2a04:4e42:400::70 ()	Ports: 443/filtered/tcp//https///
Host: 2a04:4e42:600::70 ()	Status: Up
Host: 2a04:4e42:600::70 ()	Ports: 443/filtered/tcp//https///
Host: 2a04:4e42:200::70 ()	Status: Up
Host: 2a04:4e42:200::70 ()	Ports: 443/filtered/tcp//https///
Host: 2a04:4e42::70 ()	Status: Up
Host: 2a04:4e42::70 ()	Ports: 443/filtered/tcp//https///
# Nmap done at Tue Oct 23 02:22:54 2018 -- 4 IP addresses (4 hosts up) scanned in 13.25 seconds

compared to ipv4

$ dig api.rubygems.org +short
rubygems.org.
151.101.64.70
151.101.0.70
151.101.192.70
151.101.128.70
$ nmap -p 443 -Pn -v -oG - 151.101.64.70 151.101.0.70 151.101.192.70 151.101.128.70
# Nmap 7.70 scan initiated Tue Oct 23 02:25:45 2018 as: nmap -p 443 -Pn -v -oG - 151.101.64.70 151.101.0.70 151.101.192.70 151.101.128.70
# Ports scanned: TCP(1;443) UDP(0;) SCTP(0;) PROTOCOLS(0;)
Host: 151.101.64.70 ()	Status: Up
Host: 151.101.64.70 ()	Ports: 443/open/tcp//https///
Host: 151.101.0.70 ()	Status: Up
Host: 151.101.0.70 ()	Ports: 443/open/tcp//https///
Host: 151.101.192.70 ()	Status: Up
Host: 151.101.192.70 ()	Ports: 443/open/tcp//https///
Host: 151.101.128.70 ()	Status: Up
Host: 151.101.128.70 ()	Ports: 443/open/tcp//https///
# Nmap done at Tue Oct 23 02:25:56 2018 -- 4 IP addresses (4 hosts up) scanned in 11.10 seconds

(notice that these results for IPv4 addresses say 'open' instead of 'filtered' for IPv6)

The fix as a user is to change your network settings so that you make your requests over IPv4. The stackoverflow post above has more details, but for me, it's

System Preferences > Network > Advanced > TCP/IP > Configure IPv6: Link-local only

For rubygems, I'm not sure what the long-term fix is, except to dig into why some IPv6 requests are getting filtered.

@georgeu2000

This comment has been minimized.

Copy link

@georgeu2000 georgeu2000 commented Jan 23, 2019

I ran into this exact issue today. After using rubygems for 6 years it was a new issue for me and a bit of a surprise. Thanks @rrcobb.

@jlcarruda

This comment has been minimized.

Copy link

@jlcarruda jlcarruda commented Feb 27, 2019

I was stuck on this error for a week now. The workaround for me was to change my connection from a Wireless connection into a wired one. The strangest thing is that I've always used Wireless connection and this error never showed up.
Thanks @rrcobb to lightened those tech infos.

@sankalpanand

This comment has been minimized.

Copy link

@sankalpanand sankalpanand commented Apr 5, 2019

As of April 5th, even the IPv4 is also getting filtered-

$ dig api.rubygems.org +short
rubygems.org.
151.101.0.70
151.101.192.70
151.101.64.70
151.101.128.70

$ nmap -p 443 -Pn -v -oG - 151.101.0.70 151.101.192.70 151.101.64.70 151.101.128.70
# Nmap 7.70 scan initiated Fri Apr  5 00:21:42 2019 as: nmap -p 443 -Pn -v -oG - 151.101.0.70 151.101.192.70 151.101.64.70 151.101.128.70
# Ports scanned: TCP(1;443) UDP(0;) SCTP(0;) PROTOCOLS(0;)
Host: 151.101.0.70 ()	Status: Up
Host: 151.101.0.70 ()	Ports: 443/filtered/tcp//https///
Host: 151.101.192.70 ()	Status: Up
Host: 151.101.192.70 ()	Ports: 443/filtered/tcp//https///
Host: 151.101.64.70 ()	Status: Up
Host: 151.101.64.70 ()	Ports: 443/filtered/tcp//https///
Host: 151.101.128.70 ()	Status: Up
Host: 151.101.128.70 ()	Ports: 443/filtered/tcp//https///
@jalleyne

This comment has been minimized.

Copy link

@jalleyne jalleyne commented Jul 19, 2019

+1 that @rrcobb's solution switching to ip4 for wan traffic still works.

@tueksta

This comment has been minimized.

Copy link

@tueksta tueksta commented Sep 28, 2019

why is nobody at rubygems.org looking into their filters? This exists for years now, and still no fix. It looks like Germany is mostly affected by this?

@indirect

This comment has been minimized.

Copy link
Member

@indirect indirect commented Sep 28, 2019

@tueksta because the problem is with your ISP or their upstream providers. RubyGems.org is perfectly happy to accept IPv6 traffic, and I re-test on consumer and business IPv6 connections every time someone reports it isn’t working. It’s always working.

@undecided

This comment has been minimized.

Copy link

@undecided undecided commented Oct 3, 2019

EDIT: Looking further, it seems that ping6 google.com fails for me too. Apologies, I think you're right - some ISPs are just borked. Will take this up with them


HI @indirect - I also have this problem (United Kingdom, ISP = Sky), and it seems to be getting more consistent. It doesn't affect the website RubyGems.org - only bundler.

If I do the dance:

sudo sysctl -w net.ipv6.conf.default.disable_ipv6=1
sudo sysctl -w net.ipv6.conf.all.disable_ipv6=1

it fixes itself, straight away, 100% of the time. Have just switched back and bundler still works - possibly because it's cached the ipv4 address. curl -6 --head https://api.rubygems.org throws the error, regardless of the above sysctl changes.

As one affected, if there's anything I can do to help debug this, please let me know. I don't personally mind the workaround now that I know about it, but it'll be enough of a stumper for a newbie to walk away.

@duckinator

This comment has been minimized.

Copy link
Member

@duckinator duckinator commented Oct 3, 2019

For future reference, for folks who stumble across this issue:

  1. If you're on an operating system with the curl program available, compare curl -4 --head https://api.rubygems.org and curl -6 --head https://api.rubygems.org — if only the latter fails, it's specific to IPv6.
  2. If disabling IPv6 fixes it, it's specific to IPv6
  3. If the problem is specific to IPv6, it's probably your ISP.

ISPs providing IPv6 access is a requirement for people to access the entirety of the internet (as opposed to only part of it), at this point. Unfortunately some ISPs are bad at this.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
You can’t perform that action at this time.