ArangoDB cluster mode :: VstConnection :: java.net.SocketException: Socket closed

[wajda]

The error only occurs when ArangoDB is in cluster mode and is running in a docker:

java -jar /opt/admin.jar db-init arangodb://host:8529/spline-dev-latest -f

fails with the following:

11:48:32.165 [pool-1-thread-1] ERROR c.a.i.v.internal.VstConnection - Error on Connection connection_1618912112157_0.6407627766229647
11:48:32.166 [pool-3-thread-1] ERROR c.a.i.v.internal.MessageStore - Socket closed
java.net.SocketException: Socket closed
        at java.base/java.net.SocketInputStream.socketRead0(Native Method)
        at java.base/java.net.SocketInputStream.socketRead(SocketInputStream.java:115)
        at java.base/java.net.SocketInputStream.read(SocketInputStream.java:168)
        at java.base/java.net.SocketInputStream.read(SocketInputStream.java:140)
        at com.arangodb.internal.velocystream.internal.VstConnection.readBytesIntoBuffer(VstConnection.java:337)
        at com.arangodb.internal.velocystream.internal.VstConnection.readBytes(VstConnection.java:331)
        at com.arangodb.internal.velocystream.internal.VstConnection.readChunk(VstConnection.java:306)
        at com.arangodb.internal.velocystream.internal.VstConnection.lambda$open$0(VstConnection.java:203)
        at java.base/java.util.concurrent.FutureTask.run(FutureTask.java:264)
        at java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1128)
        at java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:628)
        at java.base/java.lang.Thread.run(Thread.java:834)
Exception in thread "main" java.util.concurrent.ExecutionException: com.arangodb.ArangoDBException: java.net.SocketException: Socket closed
        at java.base/java.util.concurrent.CompletableFuture.reportGet(CompletableFuture.java:395)
        at java.base/java.util.concurrent.CompletableFuture.get(CompletableFuture.java:1999)
        at za.co.absa.spline.persistence.ArangoDatabaseFacade$.$anonfun$warmUpDb$1(ArangoDatabaseFacade.scala:81)
        at scala.concurrent.BlockContext$DefaultBlockContext$.blockOn(BlockContext.scala:57)
        at scala.concurrent.package$.blocking(package.scala:146)
        at za.co.absa.spline.persistence.ArangoDatabaseFacade$.za$co$absa$spline$persistence$ArangoDatabaseFacade$$warmUpDb(ArangoDatabaseFacade.scala:84)
        at za.co.absa.spline.persistence.ArangoDatabaseFacade.db$lzycompute(ArangoDatabaseFacade.scala:62)
        at za.co.absa.spline.persistence.ArangoDatabaseFacade.db(ArangoDatabaseFacade.scala:60)
        at za.co.absa.spline.persistence.AutoClosingArangoManagerProxy.$anonfun$withManager$1(AutoClosingArangoManagerProxy.scala:42)
        at scala.util.Try$.apply(Try.scala:213)
        at za.co.absa.spline.persistence.AutoClosingArangoManagerProxy.withManager(AutoClosingArangoManagerProxy.scala:42)
        at za.co.absa.spline.persistence.AutoClosingArangoManagerProxy.initialize(AutoClosingArangoManagerProxy.scala:31)
        at za.co.absa.spline.admin.AdminCLI.exec(AdminCLI.scala:144)
        at za.co.absa.spline.admin.AdminCLI$.delayedEndpoint$za$co$absa$spline$admin$AdminCLI$1(AdminCLI.scala:43)
        at za.co.absa.spline.admin.AdminCLI$delayedInit$body.apply(AdminCLI.scala:34)
        at scala.Function0.apply$mcV$sp(Function0.scala:39)
        at scala.Function0.apply$mcV$sp$(Function0.scala:39)
        at scala.runtime.AbstractFunction0.apply$mcV$sp(AbstractFunction0.scala:17)
        at scala.App.$anonfun$main$1$adapted(App.scala:80)
        at scala.collection.immutable.List.foreach(List.scala:392)
        at scala.App.main(App.scala:80)
        at scala.App.main$(App.scala:78)
        at za.co.absa.spline.admin.AdminCLI$.main(AdminCLI.scala:34)
        at za.co.absa.spline.admin.AdminCLI.main(AdminCLI.scala)
Caused by: com.arangodb.ArangoDBException: java.net.SocketException: Socket closed
        at com.arangodb.internal.velocystream.internal.VstConnection.writeIntern(VstConnection.java:285)
        at com.arangodb.internal.velocystream.internal.VstConnectionSync.write(VstConnectionSync.java:96)
        at com.arangodb.internal.velocystream.VstCommunicationSync.send(VstCommunicationSync.java:138)
        at com.arangodb.internal.velocystream.VstCommunicationSync.execute(VstCommunicationSync.java:124)
        at com.arangodb.internal.velocystream.VstCommunicationSync.execute(VstCommunicationSync.java:40)
        at com.arangodb.internal.velocystream.VstCommunication.execute(VstCommunication.java:139)
        at com.arangodb.internal.velocystream.VstProtocol.execute(VstProtocol.java:46)
        at com.arangodb.internal.ArangoExecutorSync.execute(ArangoExecutorSync.java:71)
        at com.arangodb.internal.net.ExtendedHostResolver.resolveFromServer(ExtendedHostResolver.java:127)
        at com.arangodb.internal.net.ExtendedHostResolver.resolve(ExtendedHostResolver.java:85)
        at com.arangodb.internal.net.FallbackHostHandler.confirm(FallbackHostHandler.java:80)
        at com.arangodb.internal.net.DirtyReadHostHandler.confirm(DirtyReadHostHandler.java:70)
        at com.arangodb.internal.velocystream.VstCommunication.connect(VstCommunication.java:97)
        at com.arangodb.internal.velocystream.VstCommunication.execute(VstCommunication.java:138)
        at com.arangodb.async.internal.ArangoExecutorAsync.lambda$execute$2(ArangoExecutorAsync.java:69)
        at java.base/java.util.concurrent.CompletableFuture$UniCompose.tryFire(CompletableFuture.java:1072)
        at java.base/java.util.concurrent.CompletableFuture$Completion.run(CompletableFuture.java:478)
        at java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1128)
        at java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:628)
        at java.base/java.lang.Thread.run(Thread.java:834)
Caused by: java.net.SocketException: Socket closed
        at java.base/java.net.SocketOutputStream.socketWrite(SocketOutputStream.java:113)
        at java.base/java.net.SocketOutputStream.write(SocketOutputStream.java:150)
        at java.base/java.io.BufferedOutputStream.flushBuffer(BufferedOutputStream.java:81)
        at java.base/java.io.BufferedOutputStream.flush(BufferedOutputStream.java:142)
        at com.arangodb.internal.velocystream.internal.VstConnection.writeIntern(VstConnection.java:282)
        ... 19 more

(reported by @dvagapov)

[wajda]

The error occurs when using arangodb-java-driver:6.9.1, while version 6.9.0 works.
See 732ff72

[cerveada]

There is a newer arangodb-java-driver:6.10.0. Maybe it fixes the issues?
https://github.com/arangodb/arangodb-java-driver/releases/tag/v6.10.0

[dvagapov]

Checked 6.10.0, without workaround - same error like in 6.9.1

[wajda]

Found the reason, see the above ticket created.

A temporary workaround is for the client to address the arangodb server via the IP or a host name that matches (as a string) the IP/host visible from the inside Docker network. E.g. instead of arangodb://localhost:8529/blah use arangodb://172.17.0.1:8529/blah