cmd/link: binary failed to execute with "This version of %1 is not compatible with the version of Windows you're running."

[bcmills]

#!watchflakes
post <- builder ~ `windows` && `This version of %1 is not compatible`

2021-03-05T06:15:22-60b500d/windows-amd64-race

CC @alexbrainman @zx2c4 @jeremyfaller

[bcmills]

Very likely related to #34774.

[zx2c4]

Is this a rare bug or is it happening all the time?

[bcmills]

Very rare.

[alexbrainman]

I don't have time even to look at the code. But the error message is, probably, because of executable is corrupted.

Alex

[bcmills]

2021-04-22T04:07:38-14a18b7/windows-amd64-2016

[bcmills]

We're now up to four of these total, so far:
2021-04-30T17:48:17-95c5f4d/windows-amd64-longtest
2021-04-22T04:07:38-14a18b7/windows-amd64-2016
2021-03-05T06:15:22-60b500d/windows-amd64-race
2019-11-20T17:13:59-b89b462/windows-amd64-longtest

[bcmills]

greplogs -l -e 'not compatible with the version of Windows' --since=2021-05-01
2022-05-10T21:29:25-d28616d/windows-amd64-longtest
2022-04-19T23:20:21-104742f/windows-amd64-longtest
2021-11-04T21:52:06-37634ee/windows-amd64-race
2021-08-24T15:55:38-d70c69d/windows-amd64-race
2021-07-01T01:38:42-9d65578/windows-amd64-longtest

[bcmills]

This seems to reproduce most often (but not exclusively) when running cmd/go.TestWindowsDefaultBuildmodIsPIE/cgo.

--- FAIL: TestWindowsDefaultBuildmodIsPIE (0.00s)
    --- FAIL: TestWindowsDefaultBuildmodIsPIE/cgo (2.98s)
        go_test.go:2197: running testgo [build -o C:\Users\gopher\AppData\Local\Temp\1\cmd-go-test-3173413636\gotest3609786383\main.exe C:\Users\gopher\AppData\Local\Temp\1\cmd-go-test-3173413636\gotest3609786383\main.go]
        go_test.go:2286: fork/exec C:\Users\gopher\AppData\Local\Temp\1\cmd-go-test-3173413636\gotest3609786383\main.exe: This version of %1 is not compatible with the version of Windows you're running. Check your computer's system information and then contact the software publisher.
go test proxy running at GOPROXY=http://127.0.0.1:51709/mod
FAIL
FAIL	cmd/go	174.990s

greplogs -l -e 'not compatible with the version of Windows' --since=2022-05-11
2022-05-13T19:45:43-ba8310c/windows-amd64-race

[bcmills]

(CC @randall77 @griesemer for cmd/compile, @cherrymui @rsc @ianlancetaylor for cmd/link. It isn't clear to me which binary is producing the corrupted parts of the executable.)

[randall77]

It's almost certainly the linker (or, perhaps, the host linker, because cgo?). If the compiler generated garbage instructions, say, it wouldn't make this kind of error. It would just be a SIGILL when running.
Maybe the linker needs some extra flush somewhere?

[ianlancetaylor]

Several cases are failing when running the (newly created) linker.

Building Go cmd/dist using C:\workdir\go1.4
Building Go toolchain1 using C:\workdir\go1.4.
Building Go bootstrap cmd/go (go_bootstrap) using Go toolchain1.
Building Go toolchain2 using go_bootstrap and Go toolchain1.
Building Go toolchain3 using go_bootstrap and Go toolchain2.
go tool link: fork/exec C:\workdir\go\pkg\tool\windows_amd64\link.exe: This version of %1 is not compatible with the version of Windows you're running. Check your computer's system information and then contact the software publisher.

All cases are failing to run a newly created binary.

I have a vague recollection that we've had trouble in the past running a newly created executable on Windows, but I couldn't pin it down.

I wonder if it would be appropriate for cmd/link to call FlushFileBuffers in OutBuf.munmap in cmd/link/internal/ld/outbuf_windows.go. CC @cherrymui @thanm

[thanm]

I wonder if it would be appropriate for cmd/link to call FlushFileBuffers in OutBuf.munmap

Seems like a promising idea. Testing such a change seems tricky given the infrequency of the problem.

Documentation for FlushFileBuffers says:

... The caller must have administrative privileges. ... https://docs.microsoft.com/en-us/windows/win32/secbp/running-with-special-privileges

Not sure if we can guarantee that?

In any case, I can give it a try.

[thanm]

I wrote a CL to do this at https://go-review.googlesource.com/c/go/+/406814 , but it doesn't seem to work (I must be holding something wrong). Fails bootstrap on windows with

Building Go cmd/dist using C:\workdir\go1.4
Building Go toolchain1 using C:\workdir\go1.4.
Building Go bootstrap cmd/go (go_bootstrap) using Go toolchain1.
C:\workdir\go\pkg\tool\windows_amd64\link: FlushFileBuffers failed: The handle is invalid.

If anyone can see where I went wrong, let me know.

[cherrymui]

Issue #38440 is a previous issue about newly created executables and file mappings on Windows. I added FlushViewOfFile for that. Maybe it is not enough?

[cherrymui]

@thanm I think FlushFileBuffers takes a file handle, not a file mapping handle. Perhaps syscall.Handle(out.f.Fd()) is the one we want.

[thanm]

Thanks, I suspected it was something like that. I'll try your suggestion.

[gopherbot]

Change https://go.dev/cl/406814 mentions this issue: cmd/link: call syscall.FlushFileBuffers on outbuf Unmap

[bcmills]

One more today. (Is CL 406814 still waiting on a gomote swarm?)

greplogs -l -e 'not compatible with the version of Windows' --since=2022-05-14
2022-05-24T21:25:09-b6bcd0d/windows-amd64-longtest

##### ../misc/reboot
Building Go cmd/dist using C:\workdir\go
Building Go toolchain1 using C:\workdir\go.
Building Go bootstrap cmd/go (go_bootstrap) using Go toolchain1.
Building Go toolchain2 using go_bootstrap and Go toolchain1.
Building Go toolchain3 using go_bootstrap and Go toolchain2.
go tool link: fork/exec C:\Users\gopher\AppData\Local\Temp\1\TestRepeatBootstrap2327678893\001\pkg\tool\windows_amd64\link.exe: This version of %1 is not compatible with the version of Windows you're running. Check your computer's system information and then contact the software publisher.

go tool dist: FAILED: C:\Users\gopher\AppData\Local\Temp\1\TestRepeatBootstrap2327678893\001\pkg\tool\windows_amd64\go_bootstrap list -f={{if .Stale}}	STALE {{.ImportPath}}: {{.StaleReason}}{{end}} cmd/asm cmd/cgo cmd/compile cmd/link runtime/internal/sys: exit status 1
--- FAIL: TestRepeatBootstrap (85.00s)
    reboot_test.go:30: GOROOT/src overlay set up in 1.0830123s
    reboot_test.go:52: exit status 1
FAIL

[thanm]

Swarm running last week generated a bunch of failures (mostly TestCallbackInAnotherThread) but nothing that matched this problem.

Consensus seems to be that submitting CL 406814 would be a positive step, so I will go ahead and check in the CL but leave this issue open.

[thanm]

No new failures so far after CL 406814 ... of course it has only been a couple of weeks. Probably best to leave open for a few more.

[gopherbot]

Found new dashboard test flakes for:

#!watchflakes
post <- builder ~ `windows` && `This version of %1 is not compatible`

2022-09-26 14:16 windows-amd64-longtest go@1e91ffc8 (log)

go: error obtaining buildID for go tool link: fork/exec C:\workdir\go\pkg\tool\windows_amd64\link.exe: This version of %1 is not compatible with the version of Windows you're running. Check your computer's system information and then contact the software publisher.

go tool dist: FAILED: C:\workdir\go\pkg\tool\windows_amd64\go_bootstrap list -f={{if .Stale}}	STALE {{.ImportPath}}: {{.StaleReason}}{{end}} cmd/asm cmd/cgo cmd/compile cmd/link runtime/internal/sys: exit status 1

— watchflakes