Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

syscall: ntdll.dll errors in rtlGetNtVersionNumbers via os.StartProcess #49731

Open
bcmills opened this issue Nov 22, 2021 · 5 comments
Open

syscall: ntdll.dll errors in rtlGetNtVersionNumbers via os.StartProcess #49731

bcmills opened this issue Nov 22, 2021 · 5 comments

Comments

@bcmills
Copy link
Member

@bcmills bcmills commented Nov 22, 2021

# go run run.go -G=0 fixedbugs/issue26411.go
exit status 2
panic: Failed to find RtlGetNtVersionNumbers procedure in ntdll.dll: The specified procedure could not be found.

goroutine 165 [running]:
panic({0x163b3a0, 0xc0003027e0})
	C:/workdir/go/src/runtime/panic.go:941 +0x397 fp=0xc0002b68c8 sp=0xc0002b6808 pc=0x1116437
syscall.(*LazyProc).mustFind(...)
	C:/workdir/go/src/syscall/dll_windows.go:293
syscall.(*LazyProc).Addr(...)
	C:/workdir/go/src/syscall/dll_windows.go:300
syscall.rtlGetNtVersionNumbers(0xc000546480?, 0x2?, 0x1bff6b55d00?)
	C:/workdir/go/src/syscall/zsyscall_windows.go:1219 +0x7a fp=0xc0002b6920 sp=0xc0002b68c8 pc=0x115d33a
syscall.StartProcess({0xc0004061e0?, 0xc0005004c8?}, {0xc00029b000, 0x2, 0x2}, 0xc0000ba000?)
	C:/workdir/go/src/syscall/exec_windows.go:317 +0x26b fp=0xc0002b6b88 sp=0xc0002b6920 pc=0x115424b
os.startProcess({0xc0004061e0, 0x30}, {0xc00029b000, 0x2, 0x2}, 0xc0002b6e60)
	C:/workdir/go/src/os/exec_posix.go:54 +0x305 fp=0xc0002b6c68 sp=0xc0002b6b88 pc=0x11be445
os.StartProcess({0xc0004061e0, 0x30}, {0xc00029b000, 0x2, 0x2}, 0xf6df31f8?)
	C:/workdir/go/src/os/exec.go:109 +0x5a fp=0xc0002b6cb0 sp=0xc0002b6c68 pc=0x11bde9a
os/exec.(*Cmd).Start(0xc00037ec60)
	C:/workdir/go/src/os/exec/exec.go:422 +0x611 fp=0xc0002b6eb8 sp=0xc0002b6cb0 pc=0x11ef631
os/exec.(*Cmd).Run(0xc0004061e0?)
	C:/workdir/go/src/os/exec/exec.go:338 +0x1e fp=0xc0002b6ed8 sp=0xc0002b6eb8 pc=0x11eed3e
cmd/go/internal/work.(*Builder).toolID(0xc00007d220, {0x16b450c, 0x7})
	C:/workdir/go/src/cmd/go/internal/work/buildid.go:167 +0x425 fp=0xc0002b70b8 sp=0xc0002b6ed8 pc=0x15599c5
cmd/go/internal/work.(*Builder).buildActionID(0xc00007d220, 0xc0001c9180)
	C:/workdir/go/src/cmd/go/internal/work/exec.go:312 +0xef5 fp=0xc0002b74c0 sp=0xc0002b70b8 pc=0x155faf5
cmd/go/internal/work.(*Builder).build(0xc00007d220, {0xc000018130?, 0xc000406180?}, 0xc0001c9180)
	C:/workdir/go/src/cmd/go/internal/work/exec.go:472 +0x199 fp=0xc0002b7db8 sp=0xc0002b74c0 pc=0x1560f99
cmd/go/internal/work.(*Builder).Do.func2({0x17b24b8, 0xc000018130}, 0xc0001c9180)
	C:/workdir/go/src/cmd/go/internal/work/exec.go:139 +0x5ec fp=0xc0002b7f20 sp=0xc0002b7db8 pc=0x155e8ec
cmd/go/internal/work.(*Builder).Do.func3()
	C:/workdir/go/src/cmd/go/internal/work/exec.go:201 +0xb9 fp=0xc0002b7fe0 sp=0xc0002b7f20 pc=0x155e119
runtime.goexit()
	C:/workdir/go/src/runtime/asm_amd64.s:1571 +0x1 fp=0xc0002b7fe8 sp=0xc0002b7fe0 pc=0x1143761
created by cmd/go/internal/work.(*Builder).Do
	C:/workdir/go/src/cmd/go/internal/work/exec.go:187 +0x3da

greplogs --dashboard -md -l -e 'Failed to find RtlGetNtVersionNumbers'

2021-11-22T18:57:33-5a3d871/windows-amd64-longtest

CC @bufflig

@bcmills
Copy link
Member Author

@bcmills bcmills commented Nov 22, 2021

Given the failure mode here I suspect some kind of race. No idea beyond that. 🤷‍♂️

Loading

@bufflig
Copy link

@bufflig bufflig commented Nov 23, 2021

I have no idea, the dll is loaded but the basic function is not found. The only reasonable cause for this would be that some other dll was found under this name, but it's a system dll, so that also seems improbable. I don't know of any reasonable event in the OS that would do this to a program, so I feel it has to be internal. I could imagine something getting garbage collected that messes up the call to fetch the function pointer, but I can't see anything obvious in the code - or something messing with the syscall structure of the m, but I can't see anything that doesn't first lockOsThread that touches that.

Loading

@bcmills
Copy link
Member Author

@bcmills bcmills commented Nov 23, 2021

Another one, same symbol but different error:
https://build.golang.org/log/aab177b601b0996b06fcb82738a63407e213c2fc

--- FAIL: TestPErsrcLLVM (0.21s)
    link_test.go:858: building failed: exit status 2, output:
        panic: Failed to load ntdll.dll: The specified module could not be found.
        
        goroutine 13 [running]:
        panic({0x7fb3a0, 0xc0003860f0})
        	C:/workdir/go/src/runtime/panic.go:941 +0x397 fp=0xc00044a8c8 sp=0xc00044a808 pc=0x2d6437
        syscall.(*LazyProc).mustFind(...)
        	C:/workdir/go/src/syscall/dll_windows.go:293
        syscall.(*LazyProc).Addr(...)
        	C:/workdir/go/src/syscall/dll_windows.go:300
        syscall.rtlGetNtVersionNumbers(0xc000380640?, 0x2?, 0x1c2344f8a80?)
        	C:/workdir/go/src/syscall/zsyscall_windows.go:1219 +0x7a fp=0xc00044a920 sp=0xc00044a8c8 pc=0x31d33a
        syscall.StartProcess({0xc000218060?, 0xc000362df8?}, {0xc0000963a0, 0x2, 0x2}, 0xc000315000?)
        	C:/workdir/go/src/syscall/exec_windows.go:317 +0x26b fp=0xc00044ab88 sp=0xc00044a920 pc=0x31424b
        os.startProcess({0xc000218060, 0x30}, {0xc0000963a0, 0x2, 0x2}, 0xc00044ae60)
        	C:/workdir/go/src/os/exec_posix.go:54 +0x305 fp=0xc00044ac68 sp=0xc00044ab88 pc=0x37e445
        os.StartProcess({0xc000218060, 0x30}, {0xc0000963a0, 0x2, 0x2}, 0x34611190?)
        	C:/workdir/go/src/os/exec.go:109 +0x5a fp=0xc00044acb0 sp=0xc00044ac68 pc=0x37de9a
        os/exec.(*Cmd).Start(0xc000254b00)
        	C:/workdir/go/src/os/exec/exec.go:422 +0x611 fp=0xc00044aeb8 sp=0xc00044acb0 pc=0x3af631
        os/exec.(*Cmd).Run(0xc000218060?)
        	C:/workdir/go/src/os/exec/exec.go:338 +0x1e fp=0xc00044aed8 sp=0xc00044aeb8 pc=0x3aed3e
        cmd/go/internal/work.(*Builder).toolID(0xc0000952c0, {0x87450c, 0x7})
        	C:/workdir/go/src/cmd/go/internal/work/buildid.go:167 +0x425 fp=0xc00044b0b8 sp=0xc00044aed8 pc=0x7199c5
        cmd/go/internal/work.(*Builder).buildActionID(0xc0000952c0, 0xc00035c780)
        	C:/workdir/go/src/cmd/go/internal/work/exec.go:312 +0xef5 fp=0xc00044b4c0 sp=0xc00044b0b8 pc=0x71faf5
        cmd/go/internal/work.(*Builder).build(0xc0000952c0, {0xc000018110?, 0xc000218000?}, 0xc00035c780)
        	C:/workdir/go/src/cmd/go/internal/work/exec.go:472 +0x199 fp=0xc00044bdb8 sp=0xc00044b4c0 pc=0x720f99
        cmd/go/internal/work.(*Builder).Do.func2({0x9724b8, 0xc000018110}, 0xc00035c780)
        	C:/workdir/go/src/cmd/go/internal/work/exec.go:139 +0x5ec fp=0xc00044bf20 sp=0xc00044bdb8 pc=0x71e8ec
        cmd/go/internal/work.(*Builder).Do.func3()
        	C:/workdir/go/src/cmd/go/internal/work/exec.go:201 +0xb9 fp=0xc00044bfe0 sp=0xc00044bf20 pc=0x71e119
        runtime.goexit()
        	C:/workdir/go/src/runtime/asm_amd64.s:1571 +0x1 fp=0xc00044bfe8 sp=0xc00044bfe0 pc=0x303761
        created by cmd/go/internal/work.(*Builder).Do
        	C:/workdir/go/src/cmd/go/internal/work/exec.go:187 +0x3da

Loading

@bcmills
Copy link
Member Author

@bcmills bcmills commented Nov 23, 2021

Broadening the search, looks like it's still just those two, and both very recently. (This looks like a 1.18 regression to me.)

greplogs --dashboard -md -l -e 'panic.*ntdll\.dll.*\n\s*\n\s*goroutine \d+.*:\n(?:\s*.+\n\s*\t.+\n)*\s*syscall\.' --since=2021-01-01

2021-11-22T21:52:20-100d7ea/windows-amd64-longtest
2021-11-22T18:57:33-5a3d871/windows-amd64-longtest

Loading

@bcmills bcmills removed this from the Backlog milestone Nov 23, 2021
@bcmills bcmills added this to the Go1.18 milestone Nov 23, 2021
@bcmills bcmills changed the title syscall: "Failed to find RtlGetNtVersionNumbers procedure in ntdll.dll" on windows-amd64-longtest builder syscall: ntdll.dll errors in rtlGetNtVersionNumbers via os.StartProcess Nov 23, 2021
@bufflig
Copy link

@bufflig bufflig commented Nov 23, 2021

I did run a test over night to try to provoke this on 32bit+64bit amd and arm64, it would not happen (surprise). I'll try to see if I can provoke a gomote, so far it has only happened on amd64-longtest, so maybe I need that specific setup to get the timing right...

Loading

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
Linked pull requests

Successfully merging a pull request may close this issue.

None yet
2 participants