Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

net: lookup_windows.go uses APIs that are no longer in Nano Server #21867

Closed
thecloudtaylor opened this issue Sep 13, 2017 · 25 comments · Fixed by StefanScherer/dockerfiles-windows#231
Labels
NeedsFix The path to resolution is known, but the work has not been done. OS-Windows
Milestone

Comments

@thecloudtaylor
Copy link

thecloudtaylor commented Sep 13, 2017

The fall release of Nano Server removed a number of APIs with the effort of reducing the size of Nano Server. One of the DLLs removed was NetApi32.dll. Currently there are a few places that lookup_windows.go calls APIs in NetApi32.dll - specifically isDomainJoined(..); lookupFullNameDomain(..) which is in turn called by newUser(..).

It may be possible to refactor a bit to remove this dependency - I will think about it but I wanted to get this logged.

Examples:

func isDomainJoined() (bool, error) {
var domain *uint16
var status uint32
err := syscall.NetGetJoinInformation(nil, &domain, &status)
if err != nil {
return false, err
}
syscall.NetApiBufferFree((*byte)(unsafe.Pointer(domain)))
return status == syscall.NetSetupDomainName, nil
}

procGetUserNameExW = modsecur32.NewProc("GetUserNameExW")
procNetUserGetInfo = modnetapi32.NewProc("NetUserGetInfo")
procNetGetJoinInformation = modnetapi32.NewProc("NetGetJoinInformation")

EDITED by @odeke-em to use permalinks and format the target examples

@odeke-em odeke-em changed the title lookup_windows.go uses APIs that are no longer in Nano Server net: lookup_windows.go uses APIs that are no longer in Nano Server Sep 13, 2017
@odeke-em
Copy link
Member

/cc @mikioh @dsnet @ianlancetaylor

@dsnet
Copy link
Member

dsnet commented Sep 13, 2017

\cc @alexbrainman. I have almost zero knowledge of Windows.

@ianlancetaylor ianlancetaylor added this to the Go1.10 milestone Sep 13, 2017
@alexbrainman
Copy link
Member

@taylorb-microsoft

It may be possible to refactor a bit to remove this dependency - I will think about it

SGTM.

We have couple of functions in syscall with name that starts with Load (for example syscall.LoadCancelIoEx). These were created because some versions of Windows did not have those functions - so we used syscall.Load... functions in standard library to program around that limitation. You can do something similar.

If you cannot replace the functionality netapi.dll provides, it is OK to just return error / skip tests. I suspect Nano Server users do not expect that functionality anyway.

Alex

@StefanScherer
Copy link

StefanScherer commented Sep 16, 2017

This can be reproduced in a microsoft/nanoserver-insider image with this example code:

package main

import (
	"fmt"
	"os/user"
)

func main() {
	fmt.Println("Hello")
	currentuser, err := user.Current()
	if err == nil {
		fmt.Println("Current user", currentuser)
	} else {
		fmt.Println("Error", err)
	}
}

Running the binary crashes

C:\gopath>currentuser.exe
Hello
panic: Failed to load netapi32.dll: The specified module could not be found.

goroutine 1 [running]:
syscall.(*LazyProc).mustFind(0xc04203b8c0)
	C:/go/src/syscall/dll_windows.go:280 +0x5f
syscall.(*LazyProc).Addr(0xc04203b8c0, 0x1)
	C:/go/src/syscall/dll_windows.go:287 +0x32
syscall.NetGetJoinInformation(0x0, 0xc04205fca8, 0xc04205fca4, 0x0, 0x0)
	C:/go/src/syscall/zsyscall_windows.go:1784 +0x38
os/user.isDomainJoined(0x4623e4, 0x29f8600, 0x0)
	C:/go/src/os/user/lookup_windows.go:21 +0x5a
os/user.lookupFullName(0xc0420440f0, 0xc, 0xc0420440e0, 0xd, 0xc0420421c0, 0x1a, 0xd, 0xc0420421c0, 0x1a, 0x0)
	C:/go/src/os/user/lookup_windows.go:58 +0x2d
os/user.newUser(0xc042068050, 0xc0420440c0, 0xc, 0xc042042180, 0x16, 0x53f1f0, 0x0, 0x0)
	C:/go/src/os/user/lookup_windows.go:88 +0x178
os/user.current(0x0, 0x0, 0x0)
	C:/go/src/os/user/lookup_windows.go:124 +0x18c
os/user.Current.func1()
	C:/go/src/os/user/lookup.go:11 +0x2d
sync.(*Once).Do(0x544660, 0x4d36e8)
	C:/go/src/sync/once.go:44 +0xc5
os/user.Current(0xc04205ff20, 0x1, 0x1)
	C:/go/src/os/user/lookup.go:11 +0x44
main.main()
	C:/gopath/currentuser.go:10 +0x85

StefanScherer added a commit to StefanScherer/dockerfiles-windows that referenced this issue Sep 16, 2017
StefanScherer added a commit to StefanScherer/dockerfiles-windows that referenced this issue Sep 16, 2017
@alexbrainman
Copy link
Member

This can be reproduced in a microsoft/nanoserver-insider image with this example code:

Thank you @StefanScherer

Alex

@StefanScherer
Copy link

Any updates? I found so far class swarm.exe, Traefik reverse proxy and Prometheus that need the netapi32.dll to work correctly in NanoServer 1709 image.

@alexbrainman
Copy link
Member

I found so far class swarm.exe, Traefik reverse proxy and Prometheus that need the netapi32.dll to work correctly in NanoServer 1709 image.

What exactly do these programs require? What netapi32.dll functionality do these programs use?

Alex

@StefanScherer
Copy link

swarm

Swarm has vendored glog which uses user.Current() - https://github.com/docker/swarm/blob/439031691c38c0904c6f833cbb583c330e201705/vendor/github.com/golang/glog/glog_file.go#L63

C:\>swarm.exe
panic: Failed to load netapi32.dll: The specified module could not be found.

goroutine 1 [running]:
syscall.(*LazyProc).mustFind(0xc04204f950)
	C:/go/src/syscall/dll_windows.go:280 +0x5f
syscall.(*LazyProc).Addr(0xc04204f950, 0xc000000000)
	C:/go/src/syscall/dll_windows.go:287 +0x32
syscall.NetGetJoinInformation(0x0, 0xc0421fbc00, 0xc0421fbbfc, 0x0, 0x0)
	C:/go/src/syscall/zsyscall_windows.go:1803 +0x38
os/user.isDomainJoined(0x4795c3, 0xd9b60, 0x0)
	C:/go/src/os/user/lookup_windows.go:21 +0x5a
os/user.lookupFullName(0xc042058540, 0xc, 0xc042056500, 0x16, 0xc042076510, 0x23, 0x16, 0xc042076510, 0x23, 0x0)
	C:/go/src/os/user/lookup_windows.go:58 +0x2d
os/user.newUser(0xc0421020d0, 0xc042058520, 0xc, 0xc0420764b0, 0x1f, 0xc042076450, 0xc, 0xc)
	C:/go/src/os/user/lookup_windows.go:88 +0x16e
os/user.current(0x0, 0x0, 0x0)
	C:/go/src/os/user/lookup_windows.go:124 +0x17f
os/user.Current.func1()
	C:/go/src/os/user/lookup.go:11 +0x29
sync.(*Once).Do(0x11ad3a0, 0xd71120)
	C:/go/src/sync/once.go:44 +0xc5
os/user.Current(0xc042058510, 0xc, 0xc042058510)
	C:/go/src/os/user/lookup.go:11 +0x44
github.com/docker/swarm/vendor/github.com/golang/glog.init.1()
	C:/go/src/github.com/docker/swarm/vendor/github.com/golang/glog/glog_file.go:63 +0x40

traefik

The vendored Kubernetes package k8s.io/client-go also uses glog:

C:\>traefik.exe
panic: Failed to load netapi32.dll: The specified module could not be found.

goroutine 1 [running]:
syscall.(*LazyProc).mustFind(0xc04203d9b0)
	/usr/local/go/src/syscall/dll_windows.go:280 +0x5f
syscall.(*LazyProc).Addr(0xc04203d9b0, 0x1)
	/usr/local/go/src/syscall/dll_windows.go:287 +0x32
syscall.NetGetJoinInformation(0x0, 0xc0426957b8, 0xc0426957b4, 0x0, 0x0)
	/usr/local/go/src/syscall/zsyscall_windows.go:1784 +0x38
os/user.isDomainJoined(0x489354, 0xa5f30, 0x0)
	/usr/local/go/src/os/user/lookup_windows.go:21 +0x5a
os/user.lookupFullName(0xc0424c10c0, 0xc, 0xc04232ce40, 0x16, 0xc042294ff0, 0x23, 0x16, 0xc042294ff0, 0x23, 0x0)
	/usr/local/go/src/os/user/lookup_windows.go:58 +0x2d
os/user.newUser(0xc042390b10, 0xc0424c1090, 0xc, 0xc042294f90, 0x1f, 0x0, 0xc042294f00, 0xc)
	/usr/local/go/src/os/user/lookup_windows.go:88 +0x178
os/user.current(0x0, 0x0, 0x0)
	/usr/local/go/src/os/user/lookup_windows.go:124 +0x18c
os/user.Current.func1()
	/usr/local/go/src/os/user/lookup.go:11 +0x2d
sync.(*Once).Do(0x3388d60, 0x221f4e8)
	/usr/local/go/src/sync/once.go:44 +0xc5
os/user.Current(0xc0424c1070, 0xc, 0xc0424c1070)
	/usr/local/go/src/os/user/lookup.go:11 +0x44
github.com/containous/traefik/vendor/github.com/golang/glog.init.1()
	/go/src/github.com/containous/traefik/vendor/github.com/golang/glog/glog_file.go:63 +0x40
github.com/containous/traefik/vendor/github.com/golang/glog.init()
	<autogenerated>:1 +0x1d3
github.com/containous/traefik/vendor/k8s.io/client-go/pkg/labels.init()
	<autogenerated>:1 +0x6b
github.com/containous/traefik/vendor/k8s.io/client-go/pkg/api/unversioned.init()
	<autogenerated>:1 +0x93
github.com/containous/traefik/vendor/k8s.io/client-go/pkg/api.init()
	<autogenerated>:1 +0x82
github.com/containous/traefik/vendor/k8s.io/client-go/pkg/apis/extensions/v1beta1.init()
	<autogenerated>:1 +0x6e
github.com/containous/traefik/provider/kubernetes.init()
	<autogenerated>:1 +0x63
github.com/containous/traefik/configuration.init()
	<autogenerated>:1 +0xa4
main.init()
	<autogenerated>:1 +0x8b

prometheus

Also prometheus uses glog :-)

C:\bin>prometheus.exe
panic: Failed to load netapi32.dll: The specified module could not be found.

goroutine 1 [running]:
syscall.(*LazyProc).mustFind(0xc04206b950)
	/usr/local/go/src/syscall/dll_windows.go:280 +0x5f
syscall.(*LazyProc).Addr(0xc04206b950, 0xc000000000)
	/usr/local/go/src/syscall/dll_windows.go:287 +0x32
syscall.NetGetJoinInformation(0x0, 0xc042093c00, 0xc042093bfc, 0x0, 0x0)
	/usr/local/go/src/syscall/zsyscall_windows.go:1803 +0x38
os/user.isDomainJoined(0x478d33, 0x6727af0, 0x0)
	/usr/local/go/src/os/user/lookup_windows.go:21 +0x5a
os/user.lookupFullName(0xc042074540, 0xc, 0xc042072540, 0x16, 0xc042088330, 0x23, 0x16, 0xc042088330, 0x23, 0x0)
	/usr/local/go/src/os/user/lookup_windows.go:58 +0x2d
os/user.newUser(0xc04209a210, 0xc042074530, 0xc, 0xc0420882d0, 0x1f, 0xc042088270, 0xc, 0xc)
	/usr/local/go/src/os/user/lookup_windows.go:88 +0x16e
os/user.current(0x0, 0x0, 0x0)
	/usr/local/go/src/os/user/lookup_windows.go:124 +0x17f
os/user.Current.func1()
	/usr/local/go/src/os/user/lookup.go:11 +0x29
sync.(*Once).Do(0x28c6660, 0x1bb1330)
	/usr/local/go/src/sync/once.go:44 +0xc5
os/user.Current(0xc042074520, 0xc, 0xc042074520)
	/usr/local/go/src/os/user/lookup.go:11 +0x44
github.com/prometheus/prometheus/vendor/github.com/golang/glog.init.1()
	/go/src/github.com/prometheus/prometheus/vendor/github.com/golang/glog/glog_file.go:63 +0x40

@alexbrainman
Copy link
Member

@StefanScherer thank you

Swarm has vendored glog which uses user.Current() - https://github.com/docker/swarm/blob/439031691c38c0904c6f833cbb583c330e201705/vendor/github.com/golang/glog/glog_file.go#L63

That uses user.Current().Username. Maybe we can get current username without netapi32.dll? @taylorb-microsoft can we do that? How?

Thank you.

Alex

@alexbrainman
Copy link
Member

@StefanScherer I think the quickest and easiest way to fix swarm.exe, Traefik reverse proxy and Prometheus is to fix glog package.

I looked at the code you have pointed to:

https://github.com/docker/swarm/blob/439031691c38c0904c6f833cbb583c330e201705/vendor/github.com/golang/glog/glog_file.go#L63

and glog uses os/user package just to fetch current user name. Is there a way to get current user name on NanoServer without calling netapi32.dll ? Can we get current user name by reading %USERNAME% environment variable or something? If yes, we can change glog code to do that if os/user.Current fails.

Alex

@StefanScherer
Copy link

Thanks @alexbrainman, I don't know if this would be accepted in glog package.
I did a short try calling GetUserNameExW from Golang, but inside a container I get an error No mapping between account names and security IDs was done., the same binary works on the Docker host.

@alexbrainman
Copy link
Member

I don't know if this would be accepted in glog package.

Why would not they. If they want their package work on NanoServer, they will make it work somehow. I did not look at their code, but I don't see how they package could not work without knowing current user name.

I did a short try calling GetUserNameExW from Golang, but inside a container I get an error No mapping between account names and security IDs was done.,

Can you list all environment variables. Maybe user name is there. Then you could use that environment variable in glog package.

Also maybe you have some contacts from Microsoft - they would tell you how to get current user name in NanoServer.

Alex

@StefanScherer
Copy link

Output of user.Current() in a Windows container:

$ docker run currentuser
Hello
Current user &{Uid:S-1-5-93-2-2 Gid:S-1-5-93-2-2 Username:User Manager\ContainerUser Name:ContainerUser HomeDir:C:\Users\ContainerUser}

So user.Current().Username maps to User Manager\ContainerUser and glog replaces the backslash to an underscore.

Listing all environments:

$ docker run currentuser cmd /c set
ALLUSERSPROFILE=C:\ProgramData
APPDATA=C:\Users\ContainerUser\AppData\Roaming
CommonProgramFiles=C:\Program Files\Common Files
CommonProgramFiles(x86)=C:\Program Files (x86)\Common Files
CommonProgramW6432=C:\Program Files\Common Files
COMPUTERNAME=86183E2D492D
ComSpec=C:\Windows\system32\cmd.exe
LOCALAPPDATA=C:\Users\ContainerUser\AppData\Local
NUMBER_OF_PROCESSORS=2
OS=Windows_NT
Path=C:\Windows\system32;C:\Windows;C:\Users\ContainerUser\AppData\Local\Microsoft\WindowsApps
PATHEXT=.COM;.EXE;.BAT;.CMD
PROCESSOR_ARCHITECTURE=AMD64
PROCESSOR_IDENTIFIER=Intel64 Family 6 Model 70 Stepping 1, GenuineIntel
PROCESSOR_LEVEL=6
PROCESSOR_REVISION=4601
ProgramData=C:\ProgramData
ProgramFiles=C:\Program Files
ProgramFiles(x86)=C:\Program Files (x86)
ProgramW6432=C:\Program Files
PROMPT=$P$G
PUBLIC=C:\Users\Public
SystemDrive=C:
SystemRoot=C:\Windows
TEMP=C:\Users\ContainerUser\AppData\Local\Temp
TMP=C:\Users\ContainerUser\AppData\Local\Temp
USERDOMAIN=User Manager
USERNAME=ContainerUser
USERPROFILE=C:\Users\ContainerUser
windir=C:\Windows

We have USERDOMAIN and USERNAME set.

So something like this?

package main

import (
	"fmt"
	"os"
	"os/user"
)

func currentUsernameFromEnv() string {
	return fmt.Sprintf("%s\\%s", os.Getenv("USERDOMAIN"), os.Getenv("USERNAME"))
}

func currentUsername() (me string) {
	defer func() {
		if r := recover(); r != nil {
			fmt.Println("Recovered!")
			me = currentUsernameFromEnv()
		}
	}()

	current, err := user.Current()
	if err == nil {
		me = current.Username
	} else {
		me = currentUsernameFromEnv()
	}
	return
}

func main() {
	fmt.Println("Hello")
	fmt.Printf("Current user %s\n", currentUsername())
}

When run on the host or in a microsoft/nanoserver:sac2016 container it works without panic/recover

Hello
Current user User Manager\ContainerUser

When run in microsoft/nanoserver:1709 container it shows

Hello
Recovered!
Current user User Manager\ContainerUser

@alexbrainman
Copy link
Member

We have USERDOMAIN and USERNAME set.

Sounds good to me. I would change your code to always use USERDOMAIN and USERNAME environment variables on Windows ( if runtime.GOOS == "windows" then ... else ... ) then you won't need to recover from panic.

I looked at https://github.com/golang/glog and it says: "Send bug reports to golang-nuts@googlegroups.com.". But I would create new issue here on https://github.com/golang/go/issues explaining the problem, and suggested solution. Perhaps someone will submit your change.

If that change is submitted, I am not sure how to update vendor folder in warm.exe, Traefik reverse proxy and Prometheus. I suppose you could always create an issue there too.

Alex

@ianlancetaylor ianlancetaylor added the NeedsInvestigation Someone must examine and confirm this is a valid issue and not a duplicate of an existing one. label Jun 27, 2018
@ianlancetaylor ianlancetaylor modified the milestones: Go1.11, Go1.12 Jun 27, 2018
@ukai
Copy link
Contributor

ukai commented Jan 28, 2022

can't we make user.Current return error instead of panic for this case?
Most calling side should handle error properly.

@qmuntal
Copy link
Contributor

qmuntal commented Jul 11, 2024

Note that if/when CL 597255 is merged (to fix #68312), then user.Current won't call netapi32 functions anymore. As this issue is mostly about user.Current(), I'm inclined to say that CL 597255 is a fix for the problem stated here. There are other functions in the os/user package that will still use netapi32, but they are not as widely used as user.Current, by far, so they can be tracked in dedicated issues.

@gopherbot
Copy link
Contributor

Change https://go.dev/cl/597255 mentions this issue: os/user: speed up Current on Windows

@gopherbot
Copy link
Contributor

Change https://go.dev/cl/604395 mentions this issue: os/user: document Current improvements

@gopherbot
Copy link
Contributor

Change https://go.dev/cl/604415 mentions this issue: os/user: test that Current does not depend on netapi32.dll

@dmitshur dmitshur added NeedsFix The path to resolution is known, but the work has not been done. and removed NeedsInvestigation Someone must examine and confirm this is a valid issue and not a duplicate of an existing one. labels Aug 9, 2024
@dmitshur dmitshur modified the milestones: Backlog, Go1.24 Aug 9, 2024
@gopherbot
Copy link
Contributor

Change https://go.dev/cl/605135 mentions this issue: os/user: speed up Current on Windows

gopherbot pushed a commit that referenced this issue Aug 21, 2024
[This is a roll-forward of CL 597255, which had to be rolled back
because it broke the windows-arm64 builder, whose current user display
name is unavailable. This new CL fixes the issue by reintroducing the
historical behavior of falling back to the user name instead of
returning an error].

user.Current is slow on Windows sessions connected to an Active
Directory domain. This is because it uses Windows APIs that do RPC
calls to the domain controller, such as TranslateAccountW and
NetUserGetInfo.

This change speeds up user.Current by using the GetUserNameEx API
instead, which is already optimized for retrieving the current user
name in different formats.

These are the improvements I see with the new implementation:

goos: windows
goarch: amd64
pkg: os/user
cpu: Intel(R) Core(TM) i7-10850H CPU @ 2.70GHz
           │   old.txt   │               new.txt                │
           │   sec/op    │    sec/op     vs base                │
Current-12   501.8µ ± 7%   118.6µ ± 11%  -76.36% (p=0.000 n=10)

           │  old.txt   │              new.txt              │
           │    B/op    │    B/op     vs base               │
Current-12   888.0 ± 0%   832.0 ± 0%  -6.31% (p=0.000 n=10)

           │  old.txt   │              new.txt               │
           │ allocs/op  │ allocs/op   vs base                │
Current-12   15.00 ± 0%   11.00 ± 0%  -26.67% (p=0.000 n=10)

Updates #5298
Fixes #21867
Fixes #68312

Cq-Include-Trybots: luci.golang.try:gotip-windows-amd64-longtest,gotip-windows-arm64
Change-Id: Ib7f77086d389cccb9d91cb77ea688d438a0ee5fd
Reviewed-on: https://go-review.googlesource.com/c/go/+/605135
Reviewed-by: Dmitri Shuralyov <dmitshur@google.com>
Reviewed-by: Mauri de Souza Meneguzzo <mauri870@gmail.com>
Reviewed-by: Alex Brainman <alex.brainman@gmail.com>
Reviewed-by: Cherry Mui <cherryyz@google.com>
LUCI-TryBot-Result: Go LUCI <golang-scoped@luci-project-accounts.iam.gserviceaccount.com>
gopherbot pushed a commit that referenced this issue Aug 21, 2024
Update #21867.
Update #68312.
Update #68647.

Change-Id: Ic41d6747c5a54ba28c1292258aa4d318ccb9fe40
Reviewed-on: https://go-review.googlesource.com/c/go/+/604395
Reviewed-by: Cherry Mui <cherryyz@google.com>
Reviewed-by: Dmitri Shuralyov <dmitshur@google.com>
LUCI-TryBot-Result: Go LUCI <golang-scoped@luci-project-accounts.iam.gserviceaccount.com>
Reviewed-by: Alex Brainman <alex.brainman@gmail.com>
gopherbot pushed a commit that referenced this issue Aug 27, 2024
Updates #21867.

Change-Id: I1eb923ef66aa0f338bfa0d683159edc1d8ae2a6c
Reviewed-on: https://go-review.googlesource.com/c/go/+/604415
Reviewed-by: Dmitri Shuralyov <dmitshur@google.com>
Reviewed-by: Alex Brainman <alex.brainman@gmail.com>
LUCI-TryBot-Result: Go LUCI <golang-scoped@luci-project-accounts.iam.gserviceaccount.com>
Reviewed-by: Cherry Mui <cherryyz@google.com>
@KyleSanderson
Copy link

Just hit this in 1.23. Is this going to make it back to the 1.23 series? or is this a 1.24 thing.

@qmuntal
Copy link
Contributor

qmuntal commented Sep 7, 2024

Just hit this in 1.23. Is this going to make it back to the 1.23 series? or is this a 1.24 thing.

I'm afraid that it won't backported given that the fix is non-trivial and that the issue has been around for +7 years.

@KyleSanderson
Copy link

It's all good man, I'm using the env hack for now, looking forward to 1.24.

Thank you for addressing this, let's hope no one else steps into this in 6 months. 😉

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
NeedsFix The path to resolution is known, but the work has not been done. OS-Windows
Projects
None yet
Development

Successfully merging a pull request may close this issue.