Issues when using pipes in the windows-latest github actions image

[mdedetrich]

Select Topic Area

Question

Body

For background context on the problem see sbt/sbt#7082. In summary I am currently experiencing problems where scripted tests in sbt fail in github actions ONLY on the windows-latest image because the port on which the sbt server and client communicate through appears to be closed, i.e.

Error:  java.net.ConnectException: Connection refused: connect
Error:  	at java.base/java.net.PlainSocketImpl.connect0(Native Method)
Error:  	at java.base/java.net.PlainSocketImpl.socketConnect(PlainSocketImpl.java:101)
Error:  	at java.base/java.net.AbstractPlainSocketImpl.doConnect(AbstractPlainSocketImpl.java:412)
Error:  	at java.base/java.net.AbstractPlainSocketImpl.connectToAddress(AbstractPlainSocketImpl.java:255)
Error:  	at java.base/java.net.AbstractPlainSocketImpl.connect(AbstractPlainSocketImpl.java:237)
Error:  	at java.base/java.net.SocksSocketImpl.connect(SocksSocketImpl.java:392)
Error:  	at java.base/java.net.Socket.connect(Socket.java:609)
Error:  	at java.base/java.net.Socket.connect(Socket.java:558)
Error:  	at java.base/java.net.Socket.<init>(Socket.java:454)
Error:  	at java.base/java.net.Socket.<init>(Socket.java:264)
Error:  	at xsbt.IPC$.client(IPC.scala:21)
Error:  	at sbt.BasicCommands$.readMessage(BasicCommands.scala:472)
Error:  	at sbt.BasicCommands$.doRead(BasicCommands.scala:448)
Error:  	at sbt.BasicCommands$.$anonfun$read$2(BasicCommands.scala:435)
Error:  	at sbt.Command$.$anonfun$applyEffect$4(Command.scala:150)
Error:  	at sbt.Command$.$anonfun$applyEffect$2(Command.scala:145)
Error:  	at sbt.Command$.process(Command.scala:189)
Error:  	at sbt.MainLoop$.$anonfun$processCommand$5(MainLoop.scala:245)
Error:  	at scala.Option.getOrElse(Option.scala:189)
Error:  	at sbt.MainLoop$.process$1(MainLoop.scala:245)
Error:  	at sbt.MainLoop$.processCommand(MainLoop.scala:276)
Error:  	at sbt.MainLoop$.$anonfun$next$5(MainLoop.scala:163)
Error:  	at sbt.State$StateOpsImpl$.runCmd$1(State.scala:289)
Error:  	at sbt.State$StateOpsImpl$.process$extension(State.scala:325)
Error:  	at sbt.MainLoop$.$anonfun$next$4(MainLoop.scala:163)
Error:  	at sbt.internal.util.ErrorHandling$.wideConvert(ErrorHandling.scala:23)
Error:  	at sbt.MainLoop$.next(MainLoop.scala:163)
Error:  	at sbt.MainLoop$.run(MainLoop.scala:144)
Error:  	at sbt.MainLoop$.$anonfun$runWithNewLog$1(MainLoop.scala:119)
Error:  	at sbt.io.Using.apply(Using.scala:27)
Error:  	at sbt.MainLoop$.runWithNewLog(MainLoop.scala:112)
Error:  	at sbt.MainLoop$.runAndClearLast(MainLoop.scala:66)
Error:  	at sbt.MainLoop$.runLoggedLoop(MainLoop.scala:51)
Error:  	at sbt.MainLoop$.runLogged(MainLoop.scala:42)
Error:  	at sbt.StandardMain$.runManaged(Main.scala:218)
Error:  	at sbt.xMain$.$anonfun$run$11(Main.scala:133)
Error:  	at sbt.internal.util.Terminal$.withStreams(Terminal.scala:[381](https://github.com/sbt/sbt-github-actions/actions/runs/3587764952/jobs/6038462004#step:8:382))
Error:  	at sbt.xMain$.withStreams$1(Main.scala:87)
Error:  	at sbt.xMain$.run(Main.scala:121)
Error:  	at java.base/jdk.internal.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
Error:  	at java.base/jdk.internal.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
Error:  	at java.base/jdk.internal.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
Error:  	at java.base/java.lang.reflect.Method.invoke(Method.java:566)
Error:  	at sbt.internal.XMainConfiguration.run(XMainConfiguration.java:56)
Error:  	at sbt.xMain.run(Main.scala:46)
Error:  	at xsbt.boot.Launch$.$anonfun$run$1(Launch.scala:149)
Error:  	at xsbt.boot.Launch$.withContextLoader(Launch.scala:176)
Error:  	at xsbt.boot.Launch$.run(Launch.scala:149)
Error:  	at xsbt.boot.Launch$.$anonfun$apply$1(Launch.scala:44)
Error:  	at xsbt.boot.Launch$.launch(Launch.scala:159)
Error:  	at xsbt.boot.Launch$.apply(Launch.scala:44)
Error:  	at xsbt.boot.Launch$.apply(Launch.scala:21)
Error:  	at xsbt.boot.Boot$.runImpl(Boot.scala:78)
Error:  	at xsbt.boot.Boot$.run(Boot.scala:73)
Error:  	at xsbt.boot.Boot$.main(Boot.scala:21)
Error:  	at xsbt.boot.Boot.main(Boot.scala)

For more details about the sbt server/client you can read https://www.scala-sbt.org/1.x/docs/sbt-server.html#Server+discovery+and+authentication but in summary the default port (5010) appears to be closed. Is there any documented way on how you can open the port so that the build can run?

Alternately if you are not able to open ports on the windows-latest image, is there a list and/or a way to get the available open ports so that sbt can be configured to use it? Another thing to note is that this problem doesn't occur on my local Windows 11 pro installation, so I think the root cause is that windows-latest is using Windows Server that has a default firewall setup that may be blocking the ports?

EDIT: Turns out the issue is with Windows pipe which sbt uses as an IPC mechanism.

[eed3si9n]

Looking into it a bit more, I think scripted test tries to open a random TCP port between 1025 and 65536:
https://github.com/sbt/sbt/blob/v1.8.0/main-command/src/main/scala/xsbt/IPC.scala#L17-L18

On "windows-latest" jobs, can we assume that we can open any random ports that aren't yet opened by ourselves?

[mdedetrich]

This is very likely the root cause of the problem, creating a PR on sbt to fix it. We should be getting ports by using this technique https://stackoverflow.com/a/2675416

[mdedetrich]

So the usage of static ports random parts rather than using new SocketServer(0) to find free ports is definitely suspicious however I wasn't able to replicate the problem within SBT (see sbt/sbt#7084). I think some more investigation is required, I am trying to see whether the core problem is a socket timeout issue but that also seems to be failing.

The other option may be that there is a firewall in Windows Server that could be effecting the test, i.e. if it sees too many connections being made then it will start blocking connections?

[github-actions[bot]]

🕒 Discussion Activity Reminder 🕒

This Discussion has been labeled as dormant by an automated system for having no activity in the last 60 days. Please consider one the following actions:

1️⃣ Close as Out of Date: If the topic is no longer relevant, close the Discussion as out of date at the bottom of the page.

2️⃣ Provide More Information: Share additional details or context — or let the community know if you've found a solution on your own.

3️⃣ Mark a Reply as Answer: If your question has been answered by a reply, mark the most helpful reply as the solution.

Note: This dormant notification will only apply to Discussions with the Question label. To learn more, see our recent announcement.

Thank you for helping bring this Discussion to a resolution! 💬