windows-2025 runs have stopped working; can't work out why

[mpilgrem]

Description

I apologise in advance if this is not an appropriate place to raise this.

Stack (the Haskell build tool, which emphasises reproducible builds) uses GitHub Actions for its own CI, running integration tests on various OS. One test is called upload. On 2025-09-28, that test passed on all OS, the run starting (on Windows) with:

2025-09-28T23:25:53.8723798Z Current runner version: '2.328.0'
2025-09-28T23:25:53.8760063Z ##[group]Runner Image Provisioner
2025-09-28T23:25:53.8761117Z Hosted Compute Agent
2025-09-28T23:25:53.8761934Z Version: 20250912.392
2025-09-28T23:25:53.8762851Z Commit: d921fda672a98b64f4f82364647e2f10b2267d0b
2025-09-28T23:25:53.8763918Z Build Date: 2025-09-12T15:23:14Z
2025-09-28T23:25:53.8764981Z ##[endgroup]
2025-09-28T23:25:53.8765738Z ##[group]Operating System
2025-09-28T23:25:53.8766690Z Microsoft Windows Server 2025
2025-09-28T23:25:53.8767506Z 10.0.26100
2025-09-28T23:25:53.8768328Z Datacenter
2025-09-28T23:25:53.8769050Z ##[endgroup]
2025-09-28T23:25:53.8769750Z ##[group]Runner Image
2025-09-28T23:25:53.8771015Z Image: windows-2025
2025-09-28T23:25:53.8771882Z Version: 20250921.36.1
2025-09-28T23:25:53.8773691Z Included Software: https://github.com/actions/runner-images/blob/win25/20250921.36/images/windows/Windows2025-Readme.md
2025-09-28T23:25:53.8777026Z Image Release: https://github.com/actions/runner-images/releases/tag/win25%2F20250921.36

But now that same test fails repeatedly on Windows (only). However, the run still starts with:

2025-10-01T14:40:16.3143743Z Current runner version: '2.328.0'
2025-10-01T14:40:16.3169841Z ##[group]Runner Image Provisioner
2025-10-01T14:40:16.3170591Z Hosted Compute Agent
2025-10-01T14:40:16.3171062Z Version: 20250912.392
2025-10-01T14:40:16.3172007Z Commit: d921fda672a98b64f4f82364647e2f10b2267d0b
2025-10-01T14:40:16.3172724Z Build Date: 2025-09-12T15:23:14Z
2025-10-01T14:40:16.3173242Z ##[endgroup]
2025-10-01T14:40:16.3173708Z ##[group]Operating System
2025-10-01T14:40:16.3174211Z Microsoft Windows Server 2025
2025-10-01T14:40:16.3174722Z 10.0.26100
2025-10-01T14:40:16.3175100Z Datacenter
2025-10-01T14:40:16.3175530Z ##[endgroup]
2025-10-01T14:40:16.3175934Z ##[group]Runner Image
2025-10-01T14:40:16.3176433Z Image: windows-2025
2025-10-01T14:40:16.3176896Z Version: 20250921.36.1
2025-10-01T14:40:16.3177826Z Included Software: https://github.com/actions/runner-images/blob/win25/20250921.36/images/windows/Windows2025-Readme.md
2025-10-01T14:40:16.3179433Z Image Release: https://github.com/actions/runner-images/releases/tag/win25%2F20250921.36

The upload test creates a fake server and tries to upload to it. It now fails with:

2025-10-01T15:55:27.8146985Z HttpExceptionRequest Request {
2025-10-01T15:55:27.8147191Z   host                 = "localhost"
2025-10-01T15:55:27.8147384Z   port                 = 12415
2025-10-01T15:55:27.8147557Z   secure               = False
2025-10-01T15:55:27.8148046Z   requestHeaders       = [("Content-Type","multipart/form-data; boundary=----WebKitFormBoundaryB5ydf7n26lbqQEiR"),("Accept","text/plain")]
2025-10-01T15:55:27.8148547Z   path                 = "/packages/"
2025-10-01T15:55:27.8148753Z   queryString          = ""
2025-10-01T15:55:27.8148929Z   method               = "POST"
2025-10-01T15:55:27.8149114Z   proxy                = Nothing
2025-10-01T15:55:27.8149289Z   rawBody              = False
2025-10-01T15:55:27.8149470Z   redirectCount        = 10
2025-10-01T15:55:27.8149690Z   responseTimeout      = ResponseTimeoutDefault
2025-10-01T15:55:27.8149931Z   requestVersion       = HTTP/1.1
2025-10-01T15:55:27.8150164Z   proxySecureMode      = ProxySecureWithConnect
2025-10-01T15:55:27.8150381Z }
2025-10-01T15:55:27.8150719Z  (ConnectionFailure Network.Socket.connect: <socket: 808>: failed (Connection refused (WSAECONNREFUSED)))
2025-10-01T15:55:27.8151158Z Main.hs: Exited with exit code: ExitFailure 1

I am confident that nothing has changed with the Stack or upload code between the runs. I wondered if something was 'up' somewhere else with GitHub?

Platforms affected

• Azure DevOps
• GitHub Actions - Standard Runners
• GitHub Actions - Larger Runners

Runner images affected

• Ubuntu 22.04
• Ubuntu 24.04
• macOS 13
• macOS 13 Arm64
• macOS 14
• macOS 14 Arm64
• macOS 15
• macOS 15 Arm64
• macOS 26 Arm64
• Windows Server 2019
• Windows Server 2022
• Windows Server 2025

Image version and build link

Works:

• https://github.com/commercialhaskell/stack/actions/runs/18081292760/job/51444921991

Fails:

• https://github.com/commercialhaskell/stack/actions/runs/18146388346/job/51707777989

Is it regression?

I think yes, see above.

Expected behavior

I can't think of a reason why this test has started failing.

Actual behavior

The test has started failing.

Repro steps

See above.

[mpilgrem]

One other data point: if I substitute runner windows-2022 for windows-latest (now equivalent to windows-2025), and make no other changes, upload does not fail.

(EDIT: However, to be clear, my issue is intra-windows-2025 (as between 2025-09-28 and currently) not inter-windows-2022 and windows-2025 (currently).)

[shamil-mubarakshin]

Hey @mpilgrem, at first glance there were no changes to the windows-2025 image between the 2 runs you've attached, both are running 20250921.36.1. Let me have a closer look and see if things can be done on image side

[shamil-mubarakshin]

@mpilgrem, ran your workflow on different commits, but couldn't reproduce the issue.

• on main
• on f03d53a54d1d93af1d7adc88ff9edf4169a99bae - from provided successful run
• on ebaab83f52e81147cb6f7f00e6ee4aba6a567624 - from provided failing run

There aren't many options left for me to check. Does your workflow still faces the issue?

[mpilgrem]

@shamil-mubarakshin, thanks for looking at this. I need to work out why:

• your 'main' works, but
• Stack's 'master' still fails

with (a) the same repository commit (bcec6aa7b2b2c9d1b632d955c1afbc5c1cf330af) (b) the same runner image and (c) both starting from 'no cache'.

EDIT: This is odd: with some further commits that touch only documentation files, the CI now passes.