Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Support IPv6 addresses in the :net_http adapter #621

Closed
wants to merge 3 commits into from

Conversation

foca
Copy link

@foca foca commented Oct 20, 2016

Fixes #589.

The problem comes from Net::HTTP not being able to understand brackets when passed a literal IPv6 address:

Net::HTTP.start("[::1]") #=> :(
Net::HTTP.start("::1") #=> :)

URI provides two different methods to extract the network host from the URI: #host, and #hostname.

The only difference being that #hostname will unwrap brackets for IPv6 addresses, which is what we want here.

There's a bit more background on why there's a different method in this ruby-core discussion/bug report: https://bugs.ruby-lang.org/issues/3788

Fixes lostisland#589.

The problem comes from Net::HTTP not being able to understand brackets
when passed a literal IPv6 address:

    Net::HTTP.start("[::1]") #=> :(
    Net::HTTP.start("::1") #=> :)

URI provides two different methods to extract the network host from the
URI: #host, and #hostname.

The only difference being that `#hostname` will unwrap brackets for IPv6
addresses, which is what we want here.
@iMacTia
Copy link
Member

iMacTia commented Oct 20, 2016

Hi @foca and thanks for your PR.
I would ask you to extend the test to all adapters rather than just the NET::HTTP one to understand if this is an issue affecting all of them. In that case we should find a solution that works for all adapters rather than fixing only one of them.
Thanks!

@mislav
Copy link
Contributor

mislav commented Oct 20, 2016

Thanks for the fix, @foca!

Do you think it would be possible to add an automated test for all adapters that makes a rudimentary IPv6 request to our test server on localhost?

@foca
Copy link
Author

foca commented Oct 20, 2016

Added an IPv6 test in a different branch. It fails (see test run at the end) for:

  • Net::HTTP
  • Net::HTTP::Persistent
  • EM::HTTP
  • EM::Synchrony

This isn't a problem with Faraday per-se, but with individual adapters (at least in the case of Net::HTTP, just the wrapper, not the library itself.)

I would like to argue in favor of fixing them separately, as I don't really have time right now to delve into EventMachine and figure out how EM::HTTP works, and I'm not sure when I'll be able to get around to fix it.

Would that be an acceptable path moving forward?

(edit: That branch doesn't have the commit in this one, which is why it fails for Net::HTTP there)


See the test failures

Run options: --seed 31509

# Running:

............E............................................................................................................................................................................E........................................E.................................................................................................................................................................E....................................................E.................................................................

Finished in 36.658117s, 13.8305 runs/s, 24.2784 assertions/s.

  1) Error:
Adapters::NetHttpPersistentTest#test_ipv6_requests:
Faraday::ConnectionFailed: Failed to open TCP connection to [::1]:3999 (getaddrinfo: nodename nor servname provided, or not known)
    /Users/foca/.rbenv/versions/2.3.1/lib/ruby/2.3.0/net/http.rb:882:in `rescue in block in connect'
    /Users/foca/.rbenv/versions/2.3.1/lib/ruby/2.3.0/net/http.rb:879:in `block in connect'
    /Users/foca/.rbenv/versions/2.3.1/lib/ruby/2.3.0/timeout.rb:91:in `block in timeout'
    /Users/foca/.rbenv/versions/2.3.1/lib/ruby/2.3.0/timeout.rb:101:in `timeout'
    /Users/foca/.rbenv/versions/2.3.1/lib/ruby/2.3.0/net/http.rb:878:in `connect'
    /Users/foca/.rbenv/versions/2.3.1/lib/ruby/2.3.0/net/http.rb:863:in `do_start'
    /Users/foca/.rbenv/versions/2.3.1/lib/ruby/2.3.0/net/http.rb:858:in `start'
    /Users/foca/Projects/self/faraday/.gs/gems/net-http-persistent-2.9.4/lib/net/http/persistent.rb:700:in `start'
    /Users/foca/Projects/self/faraday/.gs/gems/net-http-persistent-2.9.4/lib/net/http/persistent.rb:631:in `connection_for'
    /Users/foca/Projects/self/faraday/.gs/gems/net-http-persistent-2.9.4/lib/net/http/persistent.rb:994:in `request'
    /Users/foca/Projects/self/faraday/lib/faraday/adapter/net_http_persistent.rb:25:in `perform_request'
    /Users/foca/Projects/self/faraday/lib/faraday/adapter/net_http.rb:40:in `block in call'
    /Users/foca/Projects/self/faraday/lib/faraday/adapter/net_http_persistent.rb:21:in `with_net_http_connection'
    /Users/foca/Projects/self/faraday/lib/faraday/adapter/net_http.rb:32:in `call'
    /Users/foca/Projects/self/faraday/lib/faraday/response.rb:8:in `call'
    /Users/foca/Projects/self/faraday/lib/faraday/request/url_encoded.rb:15:in `call'
    /Users/foca/Projects/self/faraday/lib/faraday/request/multipart.rb:14:in `call'
    /Users/foca/Projects/self/faraday/lib/faraday/rack_builder.rb:139:in `build_response'
    /Users/foca/Projects/self/faraday/lib/faraday/connection.rb:377:in `run_request'
    /Users/foca/Projects/self/faraday/lib/faraday/connection.rb:140:in `get'
    /Users/foca/Projects/self/faraday/test/adapters/integration.rb:233:in `test_ipv6_requests'


  2) Error:
Adapters::DefaultTest#test_ipv6_requests:
Faraday::ConnectionFailed: Failed to open TCP connection to [::1]:3999 (getaddrinfo: nodename nor servname provided, or not known)
    /Users/foca/.rbenv/versions/2.3.1/lib/ruby/2.3.0/net/http.rb:882:in `rescue in block in connect'
    /Users/foca/.rbenv/versions/2.3.1/lib/ruby/2.3.0/net/http.rb:879:in `block in connect'
    /Users/foca/.rbenv/versions/2.3.1/lib/ruby/2.3.0/timeout.rb:91:in `block in timeout'
    /Users/foca/.rbenv/versions/2.3.1/lib/ruby/2.3.0/timeout.rb:101:in `timeout'
    /Users/foca/.rbenv/versions/2.3.1/lib/ruby/2.3.0/net/http.rb:878:in `connect'
    /Users/foca/.rbenv/versions/2.3.1/lib/ruby/2.3.0/net/http.rb:863:in `do_start'
    /Users/foca/.rbenv/versions/2.3.1/lib/ruby/2.3.0/net/http.rb:852:in `start'
    /Users/foca/.rbenv/versions/2.3.1/lib/ruby/2.3.0/net/http.rb:1398:in `request'
    /Users/foca/.rbenv/versions/2.3.1/lib/ruby/2.3.0/net/http.rb:1156:in `get'
    /Users/foca/Projects/self/faraday/lib/faraday/adapter/net_http.rb:80:in `perform_request'
    /Users/foca/Projects/self/faraday/lib/faraday/adapter/net_http.rb:40:in `block in call'
    /Users/foca/Projects/self/faraday/lib/faraday/adapter/net_http.rb:87:in `with_net_http_connection'
    /Users/foca/Projects/self/faraday/lib/faraday/adapter/net_http.rb:32:in `call'
    /Users/foca/Projects/self/faraday/lib/faraday/response.rb:8:in `call'
    /Users/foca/Projects/self/faraday/lib/faraday/request/url_encoded.rb:15:in `call'
    /Users/foca/Projects/self/faraday/lib/faraday/rack_builder.rb:139:in `build_response'
    /Users/foca/Projects/self/faraday/lib/faraday/connection.rb:377:in `run_request'
    /Users/foca/Projects/self/faraday/lib/faraday/connection.rb:140:in `get'
    /Users/foca/Projects/self/faraday/test/adapters/integration.rb:233:in `test_ipv6_requests'


  3) Error:
Adapters::EMSynchronyTest#test_ipv6_requests:
RuntimeError: unable to resolve server address: Undefined error: 0
    /Users/foca/Projects/self/faraday/lib/faraday/adapter/em_synchrony.rb:55:in `call'
    /Users/foca/Projects/self/faraday/lib/faraday/response.rb:8:in `call'
    /Users/foca/Projects/self/faraday/lib/faraday/request/url_encoded.rb:15:in `call'
    /Users/foca/Projects/self/faraday/lib/faraday/request/multipart.rb:14:in `call'
    /Users/foca/Projects/self/faraday/lib/faraday/rack_builder.rb:139:in `build_response'
    /Users/foca/Projects/self/faraday/lib/faraday/connection.rb:377:in `run_request'
    /Users/foca/Projects/self/faraday/lib/faraday/connection.rb:140:in `get'
    /Users/foca/Projects/self/faraday/test/adapters/integration.rb:233:in `test_ipv6_requests'


  4) Error:
Adapters::NetHttpTest#test_ipv6_requests:
Faraday::ConnectionFailed: Failed to open TCP connection to [::1]:3999 (getaddrinfo: nodename nor servname provided, or not known)
    /Users/foca/.rbenv/versions/2.3.1/lib/ruby/2.3.0/net/http.rb:882:in `rescue in block in connect'
    /Users/foca/.rbenv/versions/2.3.1/lib/ruby/2.3.0/net/http.rb:879:in `block in connect'
    /Users/foca/.rbenv/versions/2.3.1/lib/ruby/2.3.0/timeout.rb:91:in `block in timeout'
    /Users/foca/.rbenv/versions/2.3.1/lib/ruby/2.3.0/timeout.rb:101:in `timeout'
    /Users/foca/.rbenv/versions/2.3.1/lib/ruby/2.3.0/net/http.rb:878:in `connect'
    /Users/foca/.rbenv/versions/2.3.1/lib/ruby/2.3.0/net/http.rb:863:in `do_start'
    /Users/foca/.rbenv/versions/2.3.1/lib/ruby/2.3.0/net/http.rb:852:in `start'
    /Users/foca/.rbenv/versions/2.3.1/lib/ruby/2.3.0/net/http.rb:1398:in `request'
    /Users/foca/.rbenv/versions/2.3.1/lib/ruby/2.3.0/net/http.rb:1156:in `get'
    /Users/foca/Projects/self/faraday/lib/faraday/adapter/net_http.rb:80:in `perform_request'
    /Users/foca/Projects/self/faraday/lib/faraday/adapter/net_http.rb:40:in `block in call'
    /Users/foca/Projects/self/faraday/lib/faraday/adapter/net_http.rb:87:in `with_net_http_connection'
    /Users/foca/Projects/self/faraday/lib/faraday/adapter/net_http.rb:32:in `call'
    /Users/foca/Projects/self/faraday/lib/faraday/response.rb:8:in `call'
    /Users/foca/Projects/self/faraday/lib/faraday/request/url_encoded.rb:15:in `call'
    /Users/foca/Projects/self/faraday/lib/faraday/request/multipart.rb:14:in `call'
    /Users/foca/Projects/self/faraday/lib/faraday/rack_builder.rb:139:in `build_response'
    /Users/foca/Projects/self/faraday/lib/faraday/connection.rb:377:in `run_request'
    /Users/foca/Projects/self/faraday/lib/faraday/connection.rb:140:in `get'
    /Users/foca/Projects/self/faraday/test/adapters/integration.rb:233:in `test_ipv6_requests'


  5) Error:
Adapters::EMHttpTest#test_ipv6_requests:
Faraday::ClientError: unable to resolve server address: Undefined error: 0
    /Users/foca/Projects/self/faraday/lib/faraday/adapter/em_http.rb:168:in `raise_error'
    /Users/foca/Projects/self/faraday/lib/faraday/adapter/em_http.rb:113:in `perform_request'
    /Users/foca/Projects/self/faraday/lib/faraday/adapter/em_http.rb:90:in `call'
    /Users/foca/Projects/self/faraday/lib/faraday/response.rb:8:in `call'
    /Users/foca/Projects/self/faraday/lib/faraday/request/url_encoded.rb:15:in `call'
    /Users/foca/Projects/self/faraday/lib/faraday/request/multipart.rb:14:in `call'
    /Users/foca/Projects/self/faraday/lib/faraday/rack_builder.rb:139:in `build_response'
    /Users/foca/Projects/self/faraday/lib/faraday/connection.rb:377:in `run_request'
    /Users/foca/Projects/self/faraday/lib/faraday/connection.rb:140:in `get'
    /Users/foca/Projects/self/faraday/test/adapters/integration.rb:233:in `test_ipv6_requests'

507 runs, 890 assertions, 0 failures, 5 errors, 0 skips

@mislav
Copy link
Contributor

mislav commented Oct 20, 2016

Would that be an acceptable path moving forward?

  1. Replace host with hostname in other adapters and see if they start passing;
  2. For the remaining adapters that fail, check if their underlying HTTP client even supports IPv6;
  3. If it doesn't, selectively disable the IPv6 test for those adapters.

@foca
Copy link
Author

foca commented Oct 20, 2016

👌

I'll do this today or tomorrow.

Thanks ❤️

@foca
Copy link
Author

foca commented Oct 20, 2016

Ok, I opened issues upstream with both Net::HTTP::Persistent and em-http-request for fixing IPv6 support internally. In the meantime, added tests for IPv6 requests in the other adapters.

…and now it turns out that Travis's cloud providers don't support IPv6 on the loopback interface, so we can't really test against ::1 (which is why the build is failing now.)

😢

See: travis-ci/travis-ci#4964 and travis-ci/travis-ci#5200

@foca
Copy link
Author

foca commented Dec 28, 2016

Given there's no guarantee Travis will support IPv6 anytime soon, is there any way we can get the original patch (using hostname instead of host in :net_http) merged, without adding IPv6-specific tests for other adapters?

@iMacTia
Copy link
Member

iMacTia commented Dec 29, 2016

Hi @foca, I understand your frustration with all these and that the fix you're proposing for Net::HTTP adapter is extremely easy (using hostname in place of host).

However, I hope you'll understand that we can't merge this branch adding IPv6 support for Net::HTTP adapter only if we don't even know if it's supported by the other adapters.
This will only create confusion as people will ask "Does Faraday officially support IPv6?" and the answer will be something odd like "Well... we support it for Net::HTTP, but for others you'll need to test and keep your finger crossed".
Even the fact that it works fine for Net::HTTP is undermined but the lack of tests. We could potentially merge a new feature in the future that is somehow breaking compatibility with IPv6 without knowing that.

I know this all sounds exaggerated, but it's our duty to guarantee that a feature is working (and will keep doing so) once we decide to support it.

Maybe there's a way to mock these IPv6 calls? As far as I understood IPv6 support is disabled on containers used by Travis-CI, but at least mocking the calls will allow us to test if adapters are parsing them as expected...

@iMacTia
Copy link
Member

iMacTia commented Dec 29, 2016

I would also argue that we probably want the PR/Issues on net-http-persistent and em-http-request to be merged before merging this one as that will grant compatibility with those 2 adapters (and we should enable IPv6 tests for them as well)

@iMacTia iMacTia added this to the v0.11.0 milestone Dec 29, 2016
@iMacTia iMacTia removed this from the v0.11.0 milestone Dec 30, 2016
@foca
Copy link
Author

foca commented Jan 2, 2017

"Does Faraday officially support IPv6?"

No, it does not. There, hard question answered 😛

If you read through the original ruby-core issue where they end up adding hostname, you'll see that the only reason they did this was to preserve backwards compatibility, but that moving forward people are supposed to be using URI#hostname, and not URI#host, as that works exactly as before, but works with IPv6 addresses.

Also, most of Net::HTTP's usage examples tell you to use hostname, not host, and it uses it internally, too.

Let's forget about IPv6 for a second here, and just consider "should Faraday use the recommended URI APIs, or the old ones"?

Waiting for every single adapter to support IPv6 shouldn't hold that patch. Waiting on Travis to support IPv6 (which atm is, at best, a… "wellllllll, maybe someday?" scenario) should not hold that patch.

I'd send a patch that just changes URI#host for URI#hostname without adding any IPv6-specific tests (but keeping all the existing tests passing), just as a "let's use the API recommended by ruby-core".

If you disagree this is a reasonable / pragmatic course of action, then feel free to close this PR 😄

@iMacTia
Copy link
Member

iMacTia commented Jan 10, 2017

I understand your point @foca and the change itself seems reasonable.
However, I expressed my concerns about it above and they haven't changed.

I added @mislav as a reviewer and will merge this in only if he agrees with it (of course AFTER the failing tests have been removed).

@Fryguy
Copy link

Fryguy commented Jul 19, 2017

I would like to see this PR merged to move this forward. IMO, Faraday can state they "support IPv6 if the underlying adapter supports it". Then, raise a NotImplementedError for adapters that don't support IPv6 (not sure on the how the actual implementation would be done, offhand, but that would solidify the question)

@iMacTia
Copy link
Member

iMacTia commented Jul 19, 2017

@Fryguy we would all like to move towards IPv6, but apparently there's not much support on it (yet!).
I still have the same concerns I had last time I looked at this PR.
All PRs opened on other adapters are also still open and I don't think Travis now supports the IPv6 loopback. I suggested to add stubs for tests but that didn't happen.
If you want to simply change from .host to .hostname because that works for you, then you just need to Monkey-patch 5 lines of code

@iMacTia
Copy link
Member

iMacTia commented Jul 26, 2017

@foca @Fryguy I've reluctantly merged #714 with the motivations expressed in that PR, so that should be a good news for you.
I'd still like to keep this PR open as it offer tracking for the works on net-http-persistent and em-http-request clients.
It would be great to have Faraday v1.0 officially supporting IPv6, even though I understand this is not something entirely under our control.

@iMacTia iMacTia added this to the v1.0 milestone Jul 26, 2017
@iMacTia iMacTia added this to Backlog in v1.0 Jan 31, 2018
@iMacTia iMacTia closed this Feb 20, 2019
v1.0 automation moved this from Backlog to Done Feb 20, 2019
@foca foca deleted the net-http/ipv6_support branch November 17, 2021 12:58
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
No open projects
v1.0
Done
Development

Successfully merging this pull request may close these issues.

None yet

4 participants