net_http_connect_on_start: true does not correctly stub out URLs if it cannot be resolved by the DNS

[kryptonat]

I'm running into file descriptor leakages and found out that you can set net_http_connect_on_start option to true as recommended here.

However when this option is enabled, it's not able to correctly stub out a request with subdomains URLs that fail to resolve DNS lookup. This seems like a bug.

I expect mocked URL's to not actually try to HTTP connect when net_http_connect_on_start is enabled, similar to the behavior when net_http_connect_on_start is disabled.

Actual

    WebMock.disable_net_connect!(net_http_connect_on_start: true)

    # Stubbing URL with subdomain, results in an error.
    stub_request(:get, 'http://api.foobar.com').to_return(body: 'hi')
    Net::HTTP.get(URI('http://api.foobar.com'))
    # => SocketError: Failed to open TCP connection to api.foobar.com:80 (getaddrinfo: nodename nor servname provided, or not known)
 
    # Stubbing URL with no subdomain, can be stubbed successfully.
    stub_request(:get, 'http://foobar.com').to_return(body: 'hi')
    Net::HTTP.get(URI('http://foobar.com'))
    # => "hi"

    # When net_http_connect_on_start defaults to false...
    WebMock.disable_net_connect!

    # Stubbing URL with subdomain, can be stubbed successfully.
    Net::HTTP.get(URI('http://api.foobar.com'))
    # => "hi"

Expected

    WebMock.disable_net_connect!(net_http_connect_on_start: true)
    stub_request(:get, 'http://api.foobar.com').to_return(body: 'hi')
    # => "hi"

[kryptonat]

Just stepped through the code a bit, looks like enabling net_http_connect_on_start invokes connect from Net::HTTP which consequently creates a TCPSocket with the stubbed URL

https://github.com/ruby/net-http/blob/1502552ce444c122b7920d8254c79a6cd2e94155/lib/net/http.rb#L987

If that URL host cannot be resolved, it throws the SocketError (getaddrinfo: nodename nor servname provided, or not known). I'm not sure if it's intentionally written this way but if that's the case, enabling net_http_connect_on_start is not an option for resolving the file descriptor leakages if any of the stubbed requests do not resolve in a successful DNS lookup.

[privorotskii]

Actual issue for me as well

[bblimke]

I expect mocked URL's to not actually try to HTTP connect when net_http_connect_on_start is enabled, similar to the behavior when net_http_connect_on_start is disabled.

By enabling net_http_connect_on_start , you are telling WebMock to allow Net::HTTP to make the connection to the domain before the actual request to a specific url is made. At the time of connection being made there is no information about any request or URL yet, therefore WebMock is not capable to decide whether to stub a request or not.

This is all explained in README https://github.com/bblimke/webmock#connecting-on-nethttpstart

I'd be very happy to improve the current solution if anyone can suggest a better one, taking into account the information above.

[rzane]

@bblimke, would it be possible to at specify which hosts should be connected to on Net::HTTP#start? It seems like address, port, and use_ssl? could be used here to make that filtering possible.

$ irb -r net/http                                   
irb(main):001:0>  http = Net::HTTP.new('www.example.org', 80)
=> #<Net::HTTP www.example.org:80 open=false>
irb(main):002:0> http.address
=> "www.example.org"
irb(main):003:0> http.port
=> 80
irb(main):004:0> http.use_ssl?
=> false