aSocket().bind() sometimes throws Already bound SocketException

[vclvm]

Ktor Version

1.1.3

Ktor Engine Used

Netty, Socket

JVM Version, Operating System and Relevant Context

openjdk version "10.0.2" 2018-07-17, Ubuntu 18.04.2 LTS

Feedback

It's hard to reproduce, the problem appears and disappears without observable reasons. I have a server starting two Netty instances and one UDP socket

fun Application.main() {
    embeddedServer(Netty, port = 18080) {
        routing {
            get("/") {
                call.respondText("Hello World from what is supposed to be WWW server!", ContentType.Text.Plain)
            }
        }
    }.start()
    embeddedServer(Netty, port = 18090) {
        routing {
            get("/") {
                call.respondText("Hello World from what is supposed to be RESt server!", ContentType.Text.Plain)
            }
        }
    }.start()
    runBlocking {
        org.looksworking.ha.controller.Application.logger.info("Starting UDP")
        val server = aSocket(ActorSelectorManager(Dispatchers.IO))
            .udp()
            .bind(InetSocketAddress("0.0.0.0", 18070))
        while (true) {
            println(server.incoming.receive().packet.readText())
        }
    }
}

Console output:

...
Starting UDP
Exception in thread "main" java.lang.reflect.InvocationTargetException
	at java.base/jdk.internal.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
	at java.base/jdk.internal.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
	at java.base/jdk.internal.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
	at java.base/java.lang.reflect.Method.invoke(Method.java:564)
	at kotlin.reflect.jvm.internal.calls.CallerImpl$Method.callMethod(CallerImpl.kt:97)
	at kotlin.reflect.jvm.internal.calls.CallerImpl$Method$Static.call(CallerImpl.kt:106)
	at kotlin.reflect.jvm.internal.KCallableImpl.call(KCallableImpl.kt:106)
	at kotlin.reflect.jvm.internal.KCallableImpl.callDefaultMethod$kotlin_reflect_api(KCallableImpl.kt:152)
	at kotlin.reflect.jvm.internal.KCallableImpl.callBy(KCallableImpl.kt:110)
	at io.ktor.server.engine.ApplicationEngineEnvironmentReloading.callFunctionWithInjection(ApplicationEngineEnvironmentReloading.kt:349)
	at io.ktor.server.engine.ApplicationEngineEnvironmentReloading.executeModuleFunction(ApplicationEngineEnvironmentReloading.kt:299)
	at io.ktor.server.engine.ApplicationEngineEnvironmentReloading.instantiateAndConfigureApplication(ApplicationEngineEnvironmentReloading.kt:275)
	at io.ktor.server.engine.ApplicationEngineEnvironmentReloading.createApplication(ApplicationEngineEnvironmentReloading.kt:127)
	at io.ktor.server.engine.ApplicationEngineEnvironmentReloading.start(ApplicationEngineEnvironmentReloading.kt:247)
	at io.ktor.server.netty.NettyApplicationEngine.start(NettyApplicationEngine.kt:106)
	at io.ktor.server.netty.NettyApplicationEngine.start(NettyApplicationEngine.kt:18)
	at io.ktor.server.engine.ApplicationEngine$DefaultImpls.start$default(ApplicationEngine.kt:52)
	at io.ktor.server.netty.EngineMain.main(EngineMain.kt:17)
Caused by: java.net.SocketException: Already bound
	at java.base/sun.nio.ch.Net.translateToSocketException(Net.java:136)
	at java.base/sun.nio.ch.DatagramSocketAdaptor.bind(DatagramSocketAdaptor.java:93)
	at io.ktor.network.sockets.UDPSocketBuilder.bind(Builders.kt:150)
	at io.ktor.network.sockets.UDPSocketBuilder.bind$default(Builders.kt:141)
	at org.looksworking.ha.controller.AppKt$main$3.invokeSuspend(App.kt:44)
	at kotlin.coroutines.jvm.internal.BaseContinuationImpl.resumeWith(ContinuationImpl.kt:32)
	at kotlinx.coroutines.DispatchedTask.run(Dispatched.kt:233)
	at kotlinx.coroutines.EventLoopImplBase.processNextEvent(EventLoop.kt:116)
	at kotlinx.coroutines.BlockingCoroutine.joinBlocking(Builders.kt:76)
	at kotlinx.coroutines.BuildersKt__BuildersKt.runBlocking(Builders.kt:53)
	at kotlinx.coroutines.BuildersKt.runBlocking(Unknown Source)
	at kotlinx.coroutines.BuildersKt__BuildersKt.runBlocking$default(Builders.kt:35)
	at kotlinx.coroutines.BuildersKt.runBlocking$default(Unknown Source)
	at org.looksworking.ha.controller.AppKt.main(App.kt:38)
	... 18 more
Caused by: java.nio.channels.AlreadyBoundException
	at java.base/sun.nio.ch.DatagramChannelImpl.bind(DatagramChannelImpl.java:669)
	at java.base/sun.nio.ch.DatagramSocketAdaptor.bind(DatagramSocketAdaptor.java:91)
	... 30 more

I checked many times - no other instance is running, no ports are busy. At the same time, tcp ports are successfully taken, while this problem with udp socket sometimes persists even after reboot. And then just goes away without any changes.
Is it possible that like in most upvoted answer here, ktor sometimes creates an already bound socket?

[rjhdby]

Have similar problem
Ktor 1.2.1
Darwin Kernel Version 18.6.0
Java 1.0.3

    private suspend fun eventLoop(port: Int) {
        val server = aSocket(ActorSelectorManager(Dispatchers.IO))
            .udp()
            .bind(InetSocketAddress(port))       <<<<<<<<<<< Exception

50/50 for normal start/"already bound" exception

Exception in thread "main" java.net.SocketException: Already bound
	at java.base/sun.nio.ch.Net.translateToSocketException(Net.java:164)
	at java.base/sun.nio.ch.DatagramSocketAdaptor.bind(DatagramSocketAdaptor.java:105)
	at io.ktor.network.sockets.UDPSocketBuilder.bind(Builders.kt:152)
	at io.ktor.network.sockets.UDPSocketBuilder.bind$default(Builders.kt:143)
	at co.fun.multiplayer.udp.ServerListener.eventLoop(ServerListener.kt:34)
	at co.fun.multiplayer.udp.ServerListener$start$1.invokeSuspend(ServerListener.kt:26)
	at kotlin.coroutines.jvm.internal.BaseContinuationImpl.resumeWith(ContinuationImpl.kt:33)
	at kotlinx.coroutines.DispatchedTask.run(Dispatched.kt:238)
	at kotlinx.coroutines.EventLoopImplBase.processNextEvent(EventLoop.kt:116)
	at kotlinx.coroutines.BlockingCoroutine.joinBlocking(Builders.kt:80)
	at kotlinx.coroutines.BuildersKt__BuildersKt.runBlocking(Builders.kt:54)
	at kotlinx.coroutines.BuildersKt.runBlocking(Unknown Source)
	at kotlinx.coroutines.BuildersKt__BuildersKt.runBlocking$default(Builders.kt:36)
	at kotlinx.coroutines.BuildersKt.runBlocking$default(Unknown Source)
	at co.fun.multiplayer.udp.ServerListener.start(ServerListener.kt:25)
	at co.fun.multiplayer.udp.MainKt.main(Main.kt:14)
Caused by: java.nio.channels.AlreadyBoundException
	at java.base/sun.nio.ch.DatagramChannelImpl.bind(DatagramChannelImpl.java:784)
	at java.base/sun.nio.ch.DatagramSocketAdaptor.bind(DatagramSocketAdaptor.java:103)
	... 14 more

Dirty workaround

var server: BoundDatagramSocket
        while (true) {
            try {
                server = aSocket(ActorSelectorManager(Dispatchers.IO))
                    .udp()
                    .bind(InetSocketAddress(port))
                break
            } catch (e: Exception) {
                println("Oops!")
            }
        }

        println("Each!")

But it is produce other non-fatal exception

Exception in thread "DefaultDispatcher-worker-5" java.nio.channels.ClosedChannelException
	at java.base/java.nio.channels.spi.AbstractSelectableChannel.register(AbstractSelectableChannel.java:206)
	at io.ktor.network.selector.SelectorManagerSupport.applyInterest(SelectorManagerSupport.kt:116)
	at io.ktor.network.selector.ActorSelectorManager.process(ActorSelectorManager.kt:87)
	at io.ktor.network.selector.ActorSelectorManager$process$1.invokeSuspend(ActorSelectorManager.kt)
	at kotlin.coroutines.jvm.internal.BaseContinuationImpl.resumeWith(ContinuationImpl.kt:33)
	at io.ktor.network.selector.ActorSelectorManager$ContinuationHolder.resume(ActorSelectorManager.kt:187)
	at io.ktor.network.selector.ActorSelectorManager.publishInterest(ActorSelectorManager.kt:141)
	at io.ktor.network.selector.SelectorManagerSupport.select(SelectorManagerSupport.kt:46)
	at io.ktor.network.sockets.DatagramSocketImpl.receiveSuspend(DatagramSocketImpl.kt:70)
	at io.ktor.network.sockets.DatagramSocketImpl.receiveImpl(DatagramSocketImpl.kt:59)
	at io.ktor.network.sockets.DatagramSocketImpl$receiver$1.invokeSuspend(DatagramSocketImpl.kt:36)
	at kotlin.coroutines.jvm.internal.BaseContinuationImpl.resumeWith(ContinuationImpl.kt:33)
	at kotlinx.coroutines.DispatchedTask.run(Dispatched.kt:238)
	at kotlinx.coroutines.scheduling.CoroutineScheduler.runSafely(CoroutineScheduler.kt:594)
	at kotlinx.coroutines.scheduling.CoroutineScheduler.access$runSafely(CoroutineScheduler.kt:60)
	at kotlinx.coroutines.scheduling.CoroutineScheduler$Worker.run(CoroutineScheduler.kt:742)

[JesusMcCloud]

Perfectly reproducible on Java 8 and Java 11 even when explicitly trying to create an unbound UDP Socket:

    @Test
    fun ktorio() {

        val socketBuilder = aSocket(ActorSelectorManager(Dispatchers.IO)).udp()
        repeat(100) {
            while (true)
                try {
                    socketBuilder.bind(null).close()
                    println("$it ok")
                    break
                } catch (t: Throwable) {
                    println("$it FAIL (${t.message}")
                    //   t.printStackTrace()
                }
        }
    }

And it produces the same errors:

0 FAIL (Already bound)
0 ok
[…]
35 ok
Exception in thread "DefaultDispatcher-worker-2 @coroutine#18" java.nio.channels.ClosedChannelException
	at java.base/java.nio.channels.spi.AbstractSelectableChannel.register(AbstractSelectableChannel.java:206)
	at io.ktor.network.selector.SelectorManagerSupport.applyInterest(SelectorManagerSupport.kt:116)
	at io.ktor.network.selector.ActorSelectorManager.process(ActorSelectorManager.kt:87)
	at io.ktor.network.selector.ActorSelectorManager$process$1.invokeSuspend(ActorSelectorManager.kt)
	(Coroutine boundary)
	at io.ktor.network.selector.SelectorManagerSupport.select(SelectorManagerSupport.kt:177)
	at io.ktor.network.sockets.DatagramSocketImpl.receiveSuspend(DatagramSocketImpl.kt:70)
	at io.ktor.network.sockets.DatagramSocketImpl$receiver$1.invokeSuspend(DatagramSocketImpl.kt:36)
Caused by: java.nio.channels.ClosedChannelException
	at java.base/java.nio.channels.spi.AbstractSelectableChannel.register(AbstractSelectableChannel.java:206)
	at io.ktor.network.selector.SelectorManagerSupport.applyInterest(SelectorManagerSupport.kt:116)
	at io.ktor.network.selector.ActorSelectorManager.process(ActorSelectorManager.kt:87)
	at io.ktor.network.selector.ActorSelectorManager$process$1.invokeSuspend(ActorSelectorManager.kt)
	at kotlin.coroutines.jvm.internal.BaseContinuationImpl.resumeWith(ContinuationImpl.kt:33)
	at io.ktor.network.selector.ActorSelectorManager$ContinuationHolder.resume(ActorSelectorManager.kt:187)
	at io.ktor.network.selector.ActorSelectorManager.publishInterest(ActorSelectorManager.kt:141)
	at io.ktor.network.selector.SelectorManagerSupport.select(SelectorManagerSupport.kt:46)
	at io.ktor.network.sockets.DatagramSocketImpl.receiveSuspend(DatagramSocketImpl.kt:70)
	at io.ktor.network.sockets.DatagramSocketImpl.receiveImpl(DatagramSocketImpl.kt:59)
	at io.ktor.network.sockets.DatagramSocketImpl$receiver$1.invokeSuspend(DatagramSocketImpl.kt:36)
	at kotlin.coroutines.jvm.internal.BaseContinuationImpl.resumeWith(ContinuationImpl.kt:33)
	at kotlinx.coroutines.DispatchedTask.run(Dispatched.kt:238)
	at kotlinx.coroutines.scheduling.CoroutineScheduler.runSafely(CoroutineScheduler.kt:594)
	at kotlinx.coroutines.scheduling.CoroutineScheduler.access$runSafely(CoroutineScheduler.kt:60)
	at kotlinx.coroutines.scheduling.CoroutineScheduler$Worker.run(CoroutineScheduler.kt:742)
36 ok
37 ok
38 ok
[…]

[magillus]

I run into same issue with UDP socket.
Thank you for the work around applied and it works usually after first loop cycle it binds.

[deepthought-64]

I seem to come across the same bug...
io.ktor:ktor-network:1.3.1
The workaround (just try until it works) does the trick... but is there a solution on the horizon?

[oleg-larshin]

Please check the following ticket on YouTrack for follow-ups to this issue. GitHub issues will be closed in the coming weeks.