client: do not try to connect to an unreachable server #448
Conversation
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Motivation: When the client wants to connect to a server it lookups the all IP addresses of servers in the **connectString** and randomly picks one of them. However, if the client has only IPv4 configured of IPv6 with link local only, then connection will fail: --- Welcome to ZooKeeper! JLine support is enabled 2018-01-12 11:19:02,902 [myid:] - INFO [main-SendThread(lab004:2181):ClientCnxn$SendThread@1035] - Opening socket connection to server lab004/xxxx:xxx:xxx:1062:0:0:1:60:2181. Will not attempt to authenticate using SASL (unknown error) [zk: lab004:2181(CONNECTING) 0] ls / 2018-01-12 11:19:17,853 [myid:] - WARN [main-SendThread(lab004:2181):ClientCnxn$SendThread@1111] - Client session timed out, have not heard from server in 15011ms for sessionid 0x0 2018-01-12 11:19:17,853 [myid:] - INFO [main-SendThread(lab004:2181):ClientCnxn$SendThread@1159] - Client session timed out, have not heard from server in 15011ms for sessionid 0x0, closing socket connection and attempting reconnect 2018-01-12 11:19:17,956 [myid:] - INFO [main-SendThread(lab004:2181):ClientCnxn$SendThread@1035] - Opening socket connection to server lab004/x.x.x.x:2181. Will not attempt to authenticate using SASL (unknown error) 2018-01-12 11:19:17,956 [myid:] - INFO [main-SendThread(lab004:2181):ClientCnxn$SendThread@877] - Socket connection established to lab004/x.x.x.x:2181, initiating session Exception in thread "main" org.apache.zookeeper.KeeperException$ConnectionLossException: KeeperErrorCode = ConnectionLoss for / at org.apache.zookeeper.KeeperException.create(KeeperException.java:102) at org.apache.zookeeper.KeeperException.create(KeeperException.java:54) at org.apache.zookeeper.ZooKeeper.getChildren(ZooKeeper.java:1535) at org.apache.zookeeper.ZooKeeper.getChildren(ZooKeeper.java:1563) at org.apache.zookeeper.ZooKeeperMain.processZKCmd(ZooKeeperMain.java:732) at org.apache.zookeeper.ZooKeeperMain.processCmd(ZooKeeperMain.java:600) at org.apache.zookeeper.ZooKeeperMain.executeLine(ZooKeeperMain.java:372) at org.apache.zookeeper.ZooKeeperMain.run(ZooKeeperMain.java:332) at org.apache.zookeeper.ZooKeeperMain.main(ZooKeeperMain.java:291) ---- Modification: Auto-discover supported IP version and filter out server IP addresses which can't be reached. If Auto-discovery failed, then client will have the old behaviour. Result: Client does not attempts to use IPv6 if it's not configured. Signed-off-by: Tigran Mkrtchyan <tigran.mkrtchyan@desy.de>
anmolnar
requested changes
Feb 4, 2018
There was a problem hiding this comment.
@kofemann Thanks for the contribution.
The change generally looks good to me, I only have some minor nitpicks.
Also there're 2 things that should be considered:
- It'd be nice to create a Jira for the issue and
- Please consider adding some unit tests to reproduce the problem.
| } | ||
| } | ||
| } catch (SocketException e) { | ||
| LOG.error("Failed to resolve supported address types: ", e); |
There was a problem hiding this comment.
Catching this exception here could result the method returning empty list of supported address types which later will make the client unable to connect to any servers.
I think the client should be terminated either in the case of empty list or the first time when this is exception is caught.
| for (InetSocketAddress address : serverAddresses) { | ||
| try { | ||
| InetAddress ia = address.getAddress(); | ||
| String addr = (ia != null) ? ia.getHostAddress() : address.getHostString(); | ||
| InetAddress resolvedAddresses[] = InetAddress.getAllByName(addr); | ||
| for (InetAddress resolvedAddress : resolvedAddresses) { | ||
| InetAddress taddr = InetAddress.getByAddress(address.getHostString(), resolvedAddress.getAddress()); | ||
| tmpList.add(new InetSocketAddress(taddr, address.getPort())); | ||
| // try to use IP address only if it's supported by our network stack | ||
| if (!supprtedInetTypes.isEmpty() && supprtedInetTypes.contains(taddr.getClass())) { |
There was a problem hiding this comment.
It'd also work to skip the validation if supportedInetTypes is empty.
| private List<InetSocketAddress> resolveAndShuffle(Collection<InetSocketAddress> serverAddresses) { | ||
| List<InetSocketAddress> tmpList = new ArrayList<InetSocketAddress>(serverAddresses.size()); | ||
| List<InetSocketAddress> tmpList = new ArrayList<InetSocketAddress>(serverAddresses.size()); | ||
| Set<Class<? extends InetAddress>> supprtedInetTypes = supportedAddressTypes(); |
|
@kofemann Not sure if this is still an issue with the new implementation of |
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
Motivation:
When the client wants to connect to a server it lookups the all IP
addresses of servers in the connectString and randomly picks one of
them. However, if the client has only IPv4 configured of IPv6 with link
local only, then connection will fail:
Welcome to ZooKeeper!
JLine support is enabled
2018-01-12 11:19:02,902 [myid:] - INFO [main-SendThread(lab004:2181):ClientCnxn$SendThread@1035] - Opening socket connection to server lab004/xxxx:xxx:xxx:1062:0:0:1:60:2181. Will not attempt to authenticate using SASL (unknown error)
[zk: lab004:2181(CONNECTING) 0] ls /
2018-01-12 11:19:17,853 [myid:] - WARN [main-SendThread(lab004:2181):ClientCnxn$SendThread@1111] - Client session timed out, have not heard from server in 15011ms for sessionid 0x0
2018-01-12 11:19:17,853 [myid:] - INFO [main-SendThread(lab004:2181):ClientCnxn$SendThread@1159] - Client session timed out, have not heard from server in 15011ms for sessionid 0x0, closing socket connection and attempting reconnect
2018-01-12 11:19:17,956 [myid:] - INFO [main-SendThread(lab004:2181):ClientCnxn$SendThread@1035] - Opening socket connection to server lab004/x.x.x.x:2181. Will not attempt to authenticate using SASL (unknown error)
2018-01-12 11:19:17,956 [myid:] - INFO [main-SendThread(lab004:2181):ClientCnxn$SendThread@877] - Socket connection established to lab004/x.x.x.x:2181, initiating session
Exception in thread "main" org.apache.zookeeper.KeeperException$ConnectionLossException: KeeperErrorCode = ConnectionLoss for /
at org.apache.zookeeper.KeeperException.create(KeeperException.java:102)
at org.apache.zookeeper.KeeperException.create(KeeperException.java:54)
at org.apache.zookeeper.ZooKeeper.getChildren(ZooKeeper.java:1535)
at org.apache.zookeeper.ZooKeeper.getChildren(ZooKeeper.java:1563)
at org.apache.zookeeper.ZooKeeperMain.processZKCmd(ZooKeeperMain.java:732)
at org.apache.zookeeper.ZooKeeperMain.processCmd(ZooKeeperMain.java:600)
at org.apache.zookeeper.ZooKeeperMain.executeLine(ZooKeeperMain.java:372)
at org.apache.zookeeper.ZooKeeperMain.run(ZooKeeperMain.java:332)
at org.apache.zookeeper.ZooKeeperMain.main(ZooKeeperMain.java:291)
Modification:
Auto-discover supported IP version and filter out server IP addresses
which can't be reached. If Auto-discovery failed, then client will have
the old behaviour.
Result:
Client does not attempts to use IPv6 if it's not configured.
Signed-off-by: Tigran Mkrtchyan tigran.mkrtchyan@desy.de