[SPARK-24029][core] Set SO_REUSEADDR on listen sockets. #21110
Conversation
This allows sockets to be bound even if there are still sockets from a previous application that are still pending closure. It avoids bind issues when, for example, re-starting the SHS. Don't enable the option on Windows though. The following page explains some odd behavior that this option can have there: https://msdn.microsoft.com/en-us/library/windows/desktop/ms740621%28v=vs.85%29.aspx I intentionally ignored server sockets that always bind to ephemeral ports, since those don't benefit from this options.
| @@ -98,7 +99,8 @@ private void init(String hostToBind, int portToBind) { | |||
| .group(bossGroup, workerGroup) | |||
| .channel(NettyUtils.getServerChannelClass(ioMode)) | |||
| .option(ChannelOption.ALLOCATOR, allocator) | |||
| .childOption(ChannelOption.ALLOCATOR, allocator); | |||
| .childOption(ChannelOption.ALLOCATOR, allocator) | |||
| .childOption(ChannelOption.SO_REUSEADDR, !SystemUtils.IS_OS_WINDOWS); | |||
There was a problem hiding this comment.
It could be even nicer if we leave a link and few comments here why we don't enable on Windows specifically though.
There was a problem hiding this comment.
Where would you put it? Every place where this is set? Not a fan of that.
There was a problem hiding this comment.
I usually put some comments when it takes longer than usual to read tho. It's sometimes hard to debug in particular when it's OS specific. It bothered me to leave a comment saying you could leave it optionally if you feel in the same way. am fine as is too.
|
Test build #89602 has finished for PR 21110 at commit
|
|
retest this please |
|
From a very quick look, the flakiness looks global. |
|
Test build #89611 has finished for PR 21110 at commit
|
|
retest this please |
1 similar comment
|
retest this please |
|
Test build #89667 has finished for PR 21110 at commit
|
|
Merged to master. |
This allows sockets to be bound even if there are sockets
from a previous application that are still pending closure. It
avoids bind issues when, for example, re-starting the SHS.
Don't enable the option on Windows though. The following page
explains some odd behavior that this option can have there:
https://msdn.microsoft.com/en-us/library/windows/desktop/ms740621%28v=vs.85%29.aspx
I intentionally ignored server sockets that always bind to
ephemeral ports, since those don't benefit from this option.