Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

TCP Connection refused #5767

Closed
petermumford opened this issue Jun 21, 2019 · 13 comments

Comments

@petermumford
Copy link

commented Jun 21, 2019

  • jruby 9.2.7.0 - pure ruby application
  • MacOS - Darwin 18.6.0
  • Ruby application

I can't send an HTTP request to my Faye server (using Sinatra) from my JRuby application. When I do the same in a Ruby application by running ruby app.rb everything works as normal. Below is the code I am using to send a message to my Faye server:

uri = URI.parse("http://localhost:9292/faye")
Net::HTTP.post_form(uri, :message => message.to_json)

When I do the above in my JRuby application I get the following error message errno::ECONNREFUSED: Connection refused - Failed to open TCP connection to localhost:9292 (Connection refused - connect(2) for "localhost" port 9292) I've tried a different port 4567 and I still can't connect, but again, it works when I run the above code in my Ruby application or in a Rails application.

It seems to only happen when I try to connect to the Faye server. If I change the url above to point to my Rails application http://localhost:3000 it works in both my Ruby and JRuby applications.

Has anyone else had this problem? I've been stuck on this for days.

@petermumford

This comment has been minimized.

Copy link
Author

commented Jun 21, 2019

Looking at this issue #3067

if I do the following command jruby -rsocket -e "ai = Addrinfo.tcp('localhost', 9292); ai.connect; p ai" I get the following error

Errno::EBADF: Bad file descriptor - No message available
             close at org/jruby/ext/socket/RubySocket.java:694
  connect_internal at /Users/petermumford/.rbenv/versions/jruby-9.2.7.0/lib/ruby/stdlib/socket.rb:69
           connect at /Users/petermumford/.rbenv/versions/jruby-9.2.7.0/lib/ruby/stdlib/socket.rb:139
            <main> at -e:1
@enebo

This comment has been minimized.

Copy link
Member

commented Jun 21, 2019

@petermumford can you make a tiny sinatra app so we can try this locally? I can make one myself but then I won't know if it matches something you are doing or not.

@enebo enebo added this to the JRuby 9.2.8.0 milestone Jun 21, 2019

@petermumford

This comment has been minimized.

Copy link
Author

commented Jun 21, 2019

@enebo I've created a simple Faye server built with Sinatra just run ruby app.rb -p 4567 to start it. I've tested this with my JRuby application and it doesn't work, does what I described above. Please let me know how you get on.

faye.zip

@enebo

This comment has been minimized.

Copy link
Member

commented Jun 24, 2019

I can confrm we see an exception in calling finishConnect() in TCPSocket.new. Not really sure why. We select on OP_CONNECT so when that finishes we should have a connection. One thought is it connects then immediately disconnects and when finishConnect() happens it is already shut down.

I tried running faye specs as a second activity and we pass all of them so that did not yield any extra interesting info.

@enebo

This comment has been minimized.

Copy link
Member

commented Jun 24, 2019

Interesting to see when we call select we pass in this socket which is initializing and select basically depends on getOpenFile() to return something and it returns null which pushes it down an else which returns true and this error happens because it did not really select() on anything...So something is not hooked up right here. Perhaps it happens to work in cases because select is always true but in many cases perhaps it happens to be selectable anyways so it just happens to work?

@headius

This comment has been minimized.

Copy link
Member

commented Jun 24, 2019

We had some discussion on Matrix about this: https://matrix.to/#/!vyEDBdbmzqApWaugdd:matrix.org/$156140639460961GtQuJ:matrix.org?via=matrix.org

Basically it appears that on some systems, MRI is binding to IPv6 rather than IPv4 and when we try to connect with localhost we only use IPv4 and it fails to connect. The connection refused is actually connection refused.

It may be that MRI is trying both IPv4 and IPv6, but I do not have information to support that. Obviously MRI connecting to MRI works and JRuby connecting to JRuby works, but there may still be something we're not doing right when it comes to resolving and connecting to localhost in a mixed IPv4/IPv6 environment.

@headius

This comment has been minimized.

Copy link
Member

commented Jun 24, 2019

I added some info to the JRuby FAQ about this issue: https://github.com/jruby/jruby/wiki/FAQs#networking

@enebo

This comment has been minimized.

Copy link
Member

commented Jun 25, 2019

@headius I would say we definitely are doing something differently with client connections than MRI. MRI can bind to the ipv6 server using either ipv6 or ipv4. We need to match up on family to work. I will see what I can learn from MRI source code.

@enebo

This comment has been minimized.

Copy link
Member

commented Jun 25, 2019

Ok back on this a bit...

MRI has a getaddrinfo method which loops through a lookup order table (which can be ifdef'd so it is not always the same) but by default is looking in an order of: PF_INET, PF_INET6, PF_UNSPEC. The ipv4 misses for PF_INET and then when called with PF_INET6 is returns something so it uses ipv6 instead on the client side to make a connection.

We use INetAddress.getByName and I think if we use ipv4 name it makes entry which is only ipv4 and likewise the same for v6. I wonder if getAllByName only gives valid ones or not (note: this will give v4 and v6 hosts but this obivously is missing port so is not what I am talking about with regards to getaddrinfo)? The other obvious fix if the Java core APIs don't work properly is to use jnr-posix here and perform a native call.

@enebo

This comment has been minimized.

Copy link
Member

commented Jun 25, 2019

In reading on getaddrinfo a little more MRI is being a little extra specific. If hints was just PF_UNSPEC then it would return ipv6 if only the equivalent of the same host had ipv6. If somehow you can address from V4 OR V6 the ordering in last comment is used to try V4 before V6 and then I guess use unspec for the hell of it (what could return if both those fail?). Java seems totally incapable of this thus far in my exploration so this is looking more and more like it requires another native call out to match this behavior.

@enebo

This comment has been minimized.

Copy link
Member

commented Jun 25, 2019

Ok there seems to be a bug on this: https://bugs.openjdk.java.net/browse/JDK-8170568

I guess we could loop like is as suggested in this issue to solve this without using a native call but is this more work then just making the native call?

Basic logic here:
http://beej.us/guide/bgnet/html/single/bgnet.html#simpleclient

@headius

This comment has been minimized.

Copy link
Member

commented Jun 25, 2019

@enebo Seems like the looping would be fine for cases like TCPSocket, which do address resolution and connection all in one go. Our Socket doesn't support PF_UNSPEC, so at least this fix will provide a good TCP client for mixed IP4/6 envs.

@enebo

This comment has been minimized.

Copy link
Member

commented Jun 25, 2019

@headius yeah I will try this loop but in the future we can look at the native solution. Upon reflection the native solution has maintenance issues in that MRI has multiple implementations of getaddrinfo depending on platform so it may be more work than the loop on connect method. It is still a future to consider if this does not work out well...

@enebo enebo closed this in b030db5 Jun 25, 2019

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
3 participants
You can’t perform that action at this time.