Cannot recover from Connection refused during connect/reconnect

[damienlevin]

When testing the recovering of disconnects by creating a ipfw rule on OSX, I find that Brando doesn't seems to recover.

To block/allow a port I'm using :
sudo ipfw add 1 deny tcp from any to any 6379
sudo ipfw del 1 deny tcp from any to any 6379

Seems like when the connection is setup, Brando is able to detect the disconnection when the rule is set. However, it cannot connect or reconnect.

It might be an akka IO issue I just posted something on the user list : https://groups.google.com/forum/#!topic/akka-user/sL_wL_OQgbo

[damienlevin]

Here is how it looks with actor logging turned on.
In this case I set the following rule :

sudo ipfw add 1 deny tcp from any to rds1.blabla.com 6379
and removed it latter on with
sudo ipfw del 1 deny tcp from any to rds1.blabla.com 6379

As you can see an Akka actor crashed and Brando don't get notified of it no Tcp.CommandFailed(_: Connect) sent.

[DEBUG] [08/08/2013 09:43:53.307] [system-akka.actor.default-dispatcher-2] [akka://system/system/IO-TCP/selectors/$a/0] Attempting connection to [rds1.blabla.com/174.129.176.148:6379]
[DEBUG] [08/08/2013 09:43:53.334] [system-akka.actor.default-dispatcher-2] [akka://system/system/IO-TCP/selectors/$a/0] Connection established to [rds1.blabla.com/174.129.176.148:6379]
Connected
0 Success Some(Pong)
1 Success Some(Pong)
2 Success Some(Pong)
3 Success Some(Pong)
[DEBUG] [08/08/2013 09:43:57.834] [system-akka.actor.default-dispatcher-4] [akka://system/system/IO-TCP/selectors/$a/0] Closing connection due to IO error java.io.IOException: Permission denied
ConnectionClosed
[DEBUG] [08/08/2013 09:43:57.850] [system-akka.actor.default-dispatcher-3] [akka://system/system/IO-TCP/selectors/$a] received AutoReceiveMessage Envelope(Terminated(Actor[akka://system/system/IO-TCP/selectors/$a/0#958831268]),Actor[akka://system/system/IO-TCP/selectors/$a/0#958831268])
Connect
[DEBUG] [08/08/2013 09:43:59.872] [system-akka.actor.default-dispatcher-2] [akka://system/system/IO-TCP/selectors/$a/1] Attempting connection to [rds1.blabla.com/174.129.176.148:6379]
[DEBUG] [08/08/2013 09:43:59.880] [system-akka.actor.default-dispatcher-3] [akka://system/system/IO-TCP/selectors/$a/1] Permission denied
[DEBUG] [08/08/2013 09:43:59.883] [system-akka.actor.default-dispatcher-3] [akka://system/system/IO-TCP/selectors/$a] received AutoReceiveMessage Envelope(Terminated(Actor[akka://system/system/IO-TCP/selectors/$a/1#94740691]),Actor[akka://system/system/IO-TCP/selectors/$a/1#94740691])
4 Failure Timed out
5 Failure Timed out
6 Failure Timed out
7 Failure Timed out
8 Failure Timed out

[damienlevin]

Roland asked me to open a ticket :
akka/akka#1637

[damienlevin]

Fixed with Akka 2.2.1 update.
Akka issue 3535 Akka IO - CommandFailed(_: Connect) not sent for Connection refused