New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

net: lookup_windows.go uses APIs that are no longer in Nano Server #21867

Open
taylorb-microsoft opened this Issue Sep 13, 2017 · 14 comments

Comments

Projects
None yet
6 participants
@taylorb-microsoft

taylorb-microsoft commented Sep 13, 2017

The fall release of Nano Server removed a number of APIs with the effort of reducing the size of Nano Server. One of the DLLs removed was NetApi32.dll. Currently there are a few places that lookup_windows.go calls APIs in NetApi32.dll - specifically isDomainJoined(..); lookupFullNameDomain(..) which is in turn called by newUser(..).

It may be possible to refactor a bit to remove this dependency - I will think about it but I wanted to get this logged.

Examples:

func isDomainJoined() (bool, error) {
var domain *uint16
var status uint32
err := syscall.NetGetJoinInformation(nil, &domain, &status)
if err != nil {
return false, err
}
syscall.NetApiBufferFree((*byte)(unsafe.Pointer(domain)))
return status == syscall.NetSetupDomainName, nil
}

procGetUserNameExW = modsecur32.NewProc("GetUserNameExW")
procNetUserGetInfo = modnetapi32.NewProc("NetUserGetInfo")
procNetGetJoinInformation = modnetapi32.NewProc("NetGetJoinInformation")

EDITED by @odeke-em to use permalinks and format the target examples

@odeke-em odeke-em changed the title from lookup_windows.go uses APIs that are no longer in Nano Server to net: lookup_windows.go uses APIs that are no longer in Nano Server Sep 13, 2017

@odeke-em

This comment has been minimized.

Member

odeke-em commented Sep 13, 2017

@odeke-em odeke-em added the OS-Windows label Sep 13, 2017

@dsnet

This comment has been minimized.

Member

dsnet commented Sep 13, 2017

\cc @alexbrainman. I have almost zero knowledge of Windows.

@ianlancetaylor ianlancetaylor added this to the Go1.10 milestone Sep 13, 2017

@alexbrainman

This comment has been minimized.

Member

alexbrainman commented Sep 14, 2017

@taylorb-microsoft

It may be possible to refactor a bit to remove this dependency - I will think about it

SGTM.

We have couple of functions in syscall with name that starts with Load (for example syscall.LoadCancelIoEx). These were created because some versions of Windows did not have those functions - so we used syscall.Load... functions in standard library to program around that limitation. You can do something similar.

If you cannot replace the functionality netapi.dll provides, it is OK to just return error / skip tests. I suspect Nano Server users do not expect that functionality anyway.

Alex

@StefanScherer

This comment has been minimized.

StefanScherer commented Sep 16, 2017

This can be reproduced in a microsoft/nanoserver-insider image with this example code:

package main

import (
	"fmt"
	"os/user"
)

func main() {
	fmt.Println("Hello")
	currentuser, err := user.Current()
	if err == nil {
		fmt.Println("Current user", currentuser)
	} else {
		fmt.Println("Error", err)
	}
}

Running the binary crashes

C:\gopath>currentuser.exe
Hello
panic: Failed to load netapi32.dll: The specified module could not be found.

goroutine 1 [running]:
syscall.(*LazyProc).mustFind(0xc04203b8c0)
	C:/go/src/syscall/dll_windows.go:280 +0x5f
syscall.(*LazyProc).Addr(0xc04203b8c0, 0x1)
	C:/go/src/syscall/dll_windows.go:287 +0x32
syscall.NetGetJoinInformation(0x0, 0xc04205fca8, 0xc04205fca4, 0x0, 0x0)
	C:/go/src/syscall/zsyscall_windows.go:1784 +0x38
os/user.isDomainJoined(0x4623e4, 0x29f8600, 0x0)
	C:/go/src/os/user/lookup_windows.go:21 +0x5a
os/user.lookupFullName(0xc0420440f0, 0xc, 0xc0420440e0, 0xd, 0xc0420421c0, 0x1a, 0xd, 0xc0420421c0, 0x1a, 0x0)
	C:/go/src/os/user/lookup_windows.go:58 +0x2d
os/user.newUser(0xc042068050, 0xc0420440c0, 0xc, 0xc042042180, 0x16, 0x53f1f0, 0x0, 0x0)
	C:/go/src/os/user/lookup_windows.go:88 +0x178
os/user.current(0x0, 0x0, 0x0)
	C:/go/src/os/user/lookup_windows.go:124 +0x18c
os/user.Current.func1()
	C:/go/src/os/user/lookup.go:11 +0x2d
sync.(*Once).Do(0x544660, 0x4d36e8)
	C:/go/src/sync/once.go:44 +0xc5
os/user.Current(0xc04205ff20, 0x1, 0x1)
	C:/go/src/os/user/lookup.go:11 +0x44
main.main()
	C:/gopath/currentuser.go:10 +0x85

StefanScherer added a commit to StefanScherer/dockerfiles-windows that referenced this issue Sep 16, 2017

StefanScherer added a commit to StefanScherer/dockerfiles-windows that referenced this issue Sep 16, 2017

@alexbrainman

This comment has been minimized.

Member

alexbrainman commented Sep 17, 2017

This can be reproduced in a microsoft/nanoserver-insider image with this example code:

Thank you @StefanScherer

Alex

@StefanScherer

This comment has been minimized.

StefanScherer commented Mar 9, 2018

Any updates? I found so far class swarm.exe, Traefik reverse proxy and Prometheus that need the netapi32.dll to work correctly in NanoServer 1709 image.

@alexbrainman

This comment has been minimized.

Member

alexbrainman commented Mar 12, 2018

I found so far class swarm.exe, Traefik reverse proxy and Prometheus that need the netapi32.dll to work correctly in NanoServer 1709 image.

What exactly do these programs require? What netapi32.dll functionality do these programs use?

Alex

@StefanScherer

This comment has been minimized.

StefanScherer commented Mar 13, 2018

swarm

Swarm has vendored glog which uses user.Current() - https://github.com/docker/swarm/blob/439031691c38c0904c6f833cbb583c330e201705/vendor/github.com/golang/glog/glog_file.go#L63

C:\>swarm.exe
panic: Failed to load netapi32.dll: The specified module could not be found.

goroutine 1 [running]:
syscall.(*LazyProc).mustFind(0xc04204f950)
	C:/go/src/syscall/dll_windows.go:280 +0x5f
syscall.(*LazyProc).Addr(0xc04204f950, 0xc000000000)
	C:/go/src/syscall/dll_windows.go:287 +0x32
syscall.NetGetJoinInformation(0x0, 0xc0421fbc00, 0xc0421fbbfc, 0x0, 0x0)
	C:/go/src/syscall/zsyscall_windows.go:1803 +0x38
os/user.isDomainJoined(0x4795c3, 0xd9b60, 0x0)
	C:/go/src/os/user/lookup_windows.go:21 +0x5a
os/user.lookupFullName(0xc042058540, 0xc, 0xc042056500, 0x16, 0xc042076510, 0x23, 0x16, 0xc042076510, 0x23, 0x0)
	C:/go/src/os/user/lookup_windows.go:58 +0x2d
os/user.newUser(0xc0421020d0, 0xc042058520, 0xc, 0xc0420764b0, 0x1f, 0xc042076450, 0xc, 0xc)
	C:/go/src/os/user/lookup_windows.go:88 +0x16e
os/user.current(0x0, 0x0, 0x0)
	C:/go/src/os/user/lookup_windows.go:124 +0x17f
os/user.Current.func1()
	C:/go/src/os/user/lookup.go:11 +0x29
sync.(*Once).Do(0x11ad3a0, 0xd71120)
	C:/go/src/sync/once.go:44 +0xc5
os/user.Current(0xc042058510, 0xc, 0xc042058510)
	C:/go/src/os/user/lookup.go:11 +0x44
github.com/docker/swarm/vendor/github.com/golang/glog.init.1()
	C:/go/src/github.com/docker/swarm/vendor/github.com/golang/glog/glog_file.go:63 +0x40

traefik

The vendored Kubernetes package k8s.io/client-go also uses glog:

C:\>traefik.exe
panic: Failed to load netapi32.dll: The specified module could not be found.

goroutine 1 [running]:
syscall.(*LazyProc).mustFind(0xc04203d9b0)
	/usr/local/go/src/syscall/dll_windows.go:280 +0x5f
syscall.(*LazyProc).Addr(0xc04203d9b0, 0x1)
	/usr/local/go/src/syscall/dll_windows.go:287 +0x32
syscall.NetGetJoinInformation(0x0, 0xc0426957b8, 0xc0426957b4, 0x0, 0x0)
	/usr/local/go/src/syscall/zsyscall_windows.go:1784 +0x38
os/user.isDomainJoined(0x489354, 0xa5f30, 0x0)
	/usr/local/go/src/os/user/lookup_windows.go:21 +0x5a
os/user.lookupFullName(0xc0424c10c0, 0xc, 0xc04232ce40, 0x16, 0xc042294ff0, 0x23, 0x16, 0xc042294ff0, 0x23, 0x0)
	/usr/local/go/src/os/user/lookup_windows.go:58 +0x2d
os/user.newUser(0xc042390b10, 0xc0424c1090, 0xc, 0xc042294f90, 0x1f, 0x0, 0xc042294f00, 0xc)
	/usr/local/go/src/os/user/lookup_windows.go:88 +0x178
os/user.current(0x0, 0x0, 0x0)
	/usr/local/go/src/os/user/lookup_windows.go:124 +0x18c
os/user.Current.func1()
	/usr/local/go/src/os/user/lookup.go:11 +0x2d
sync.(*Once).Do(0x3388d60, 0x221f4e8)
	/usr/local/go/src/sync/once.go:44 +0xc5
os/user.Current(0xc0424c1070, 0xc, 0xc0424c1070)
	/usr/local/go/src/os/user/lookup.go:11 +0x44
github.com/containous/traefik/vendor/github.com/golang/glog.init.1()
	/go/src/github.com/containous/traefik/vendor/github.com/golang/glog/glog_file.go:63 +0x40
github.com/containous/traefik/vendor/github.com/golang/glog.init()
	<autogenerated>:1 +0x1d3
github.com/containous/traefik/vendor/k8s.io/client-go/pkg/labels.init()
	<autogenerated>:1 +0x6b
github.com/containous/traefik/vendor/k8s.io/client-go/pkg/api/unversioned.init()
	<autogenerated>:1 +0x93
github.com/containous/traefik/vendor/k8s.io/client-go/pkg/api.init()
	<autogenerated>:1 +0x82
github.com/containous/traefik/vendor/k8s.io/client-go/pkg/apis/extensions/v1beta1.init()
	<autogenerated>:1 +0x6e
github.com/containous/traefik/provider/kubernetes.init()
	<autogenerated>:1 +0x63
github.com/containous/traefik/configuration.init()
	<autogenerated>:1 +0xa4
main.init()
	<autogenerated>:1 +0x8b

prometheus

Also prometheus uses glog :-)

C:\bin>prometheus.exe
panic: Failed to load netapi32.dll: The specified module could not be found.

goroutine 1 [running]:
syscall.(*LazyProc).mustFind(0xc04206b950)
	/usr/local/go/src/syscall/dll_windows.go:280 +0x5f
syscall.(*LazyProc).Addr(0xc04206b950, 0xc000000000)
	/usr/local/go/src/syscall/dll_windows.go:287 +0x32
syscall.NetGetJoinInformation(0x0, 0xc042093c00, 0xc042093bfc, 0x0, 0x0)
	/usr/local/go/src/syscall/zsyscall_windows.go:1803 +0x38
os/user.isDomainJoined(0x478d33, 0x6727af0, 0x0)
	/usr/local/go/src/os/user/lookup_windows.go:21 +0x5a
os/user.lookupFullName(0xc042074540, 0xc, 0xc042072540, 0x16, 0xc042088330, 0x23, 0x16, 0xc042088330, 0x23, 0x0)
	/usr/local/go/src/os/user/lookup_windows.go:58 +0x2d
os/user.newUser(0xc04209a210, 0xc042074530, 0xc, 0xc0420882d0, 0x1f, 0xc042088270, 0xc, 0xc)
	/usr/local/go/src/os/user/lookup_windows.go:88 +0x16e
os/user.current(0x0, 0x0, 0x0)
	/usr/local/go/src/os/user/lookup_windows.go:124 +0x17f
os/user.Current.func1()
	/usr/local/go/src/os/user/lookup.go:11 +0x29
sync.(*Once).Do(0x28c6660, 0x1bb1330)
	/usr/local/go/src/sync/once.go:44 +0xc5
os/user.Current(0xc042074520, 0xc, 0xc042074520)
	/usr/local/go/src/os/user/lookup.go:11 +0x44
github.com/prometheus/prometheus/vendor/github.com/golang/glog.init.1()
	/go/src/github.com/prometheus/prometheus/vendor/github.com/golang/glog/glog_file.go:63 +0x40
@alexbrainman

This comment has been minimized.

Member

alexbrainman commented Mar 13, 2018

@StefanScherer thank you

Swarm has vendored glog which uses user.Current() - https://github.com/docker/swarm/blob/439031691c38c0904c6f833cbb583c330e201705/vendor/github.com/golang/glog/glog_file.go#L63

That uses user.Current().Username. Maybe we can get current username without netapi32.dll? @taylorb-microsoft can we do that? How?

Thank you.

Alex

@alexbrainman

This comment has been minimized.

Member

alexbrainman commented Mar 17, 2018

@StefanScherer I think the quickest and easiest way to fix swarm.exe, Traefik reverse proxy and Prometheus is to fix glog package.

I looked at the code you have pointed to:

https://github.com/docker/swarm/blob/439031691c38c0904c6f833cbb583c330e201705/vendor/github.com/golang/glog/glog_file.go#L63

and glog uses os/user package just to fetch current user name. Is there a way to get current user name on NanoServer without calling netapi32.dll ? Can we get current user name by reading %USERNAME% environment variable or something? If yes, we can change glog code to do that if os/user.Current fails.

Alex

@StefanScherer

This comment has been minimized.

StefanScherer commented Mar 22, 2018

Thanks @alexbrainman, I don't know if this would be accepted in glog package.
I did a short try calling GetUserNameExW from Golang, but inside a container I get an error No mapping between account names and security IDs was done., the same binary works on the Docker host.

@alexbrainman

This comment has been minimized.

Member

alexbrainman commented Mar 23, 2018

I don't know if this would be accepted in glog package.

Why would not they. If they want their package work on NanoServer, they will make it work somehow. I did not look at their code, but I don't see how they package could not work without knowing current user name.

I did a short try calling GetUserNameExW from Golang, but inside a container I get an error No mapping between account names and security IDs was done.,

Can you list all environment variables. Maybe user name is there. Then you could use that environment variable in glog package.

Also maybe you have some contacts from Microsoft - they would tell you how to get current user name in NanoServer.

Alex

@StefanScherer

This comment has been minimized.

StefanScherer commented Mar 23, 2018

Output of user.Current() in a Windows container:

$ docker run currentuser
Hello
Current user &{Uid:S-1-5-93-2-2 Gid:S-1-5-93-2-2 Username:User Manager\ContainerUser Name:ContainerUser HomeDir:C:\Users\ContainerUser}

So user.Current().Username maps to User Manager\ContainerUser and glog replaces the backslash to an underscore.

Listing all environments:

$ docker run currentuser cmd /c set
ALLUSERSPROFILE=C:\ProgramData
APPDATA=C:\Users\ContainerUser\AppData\Roaming
CommonProgramFiles=C:\Program Files\Common Files
CommonProgramFiles(x86)=C:\Program Files (x86)\Common Files
CommonProgramW6432=C:\Program Files\Common Files
COMPUTERNAME=86183E2D492D
ComSpec=C:\Windows\system32\cmd.exe
LOCALAPPDATA=C:\Users\ContainerUser\AppData\Local
NUMBER_OF_PROCESSORS=2
OS=Windows_NT
Path=C:\Windows\system32;C:\Windows;C:\Users\ContainerUser\AppData\Local\Microsoft\WindowsApps
PATHEXT=.COM;.EXE;.BAT;.CMD
PROCESSOR_ARCHITECTURE=AMD64
PROCESSOR_IDENTIFIER=Intel64 Family 6 Model 70 Stepping 1, GenuineIntel
PROCESSOR_LEVEL=6
PROCESSOR_REVISION=4601
ProgramData=C:\ProgramData
ProgramFiles=C:\Program Files
ProgramFiles(x86)=C:\Program Files (x86)
ProgramW6432=C:\Program Files
PROMPT=$P$G
PUBLIC=C:\Users\Public
SystemDrive=C:
SystemRoot=C:\Windows
TEMP=C:\Users\ContainerUser\AppData\Local\Temp
TMP=C:\Users\ContainerUser\AppData\Local\Temp
USERDOMAIN=User Manager
USERNAME=ContainerUser
USERPROFILE=C:\Users\ContainerUser
windir=C:\Windows

We have USERDOMAIN and USERNAME set.

So something like this?

package main

import (
	"fmt"
	"os"
	"os/user"
)

func currentUsernameFromEnv() string {
	return fmt.Sprintf("%s\\%s", os.Getenv("USERDOMAIN"), os.Getenv("USERNAME"))
}

func currentUsername() (me string) {
	defer func() {
		if r := recover(); r != nil {
			fmt.Println("Recovered!")
			me = currentUsernameFromEnv()
		}
	}()

	current, err := user.Current()
	if err == nil {
		me = current.Username
	} else {
		me = currentUsernameFromEnv()
	}
	return
}

func main() {
	fmt.Println("Hello")
	fmt.Printf("Current user %s\n", currentUsername())
}

When run on the host or in a microsoft/nanoserver:sac2016 container it works without panic/recover

Hello
Current user User Manager\ContainerUser

When run in microsoft/nanoserver:1709 container it shows

Hello
Recovered!
Current user User Manager\ContainerUser
@alexbrainman

This comment has been minimized.

Member

alexbrainman commented Mar 24, 2018

We have USERDOMAIN and USERNAME set.

Sounds good to me. I would change your code to always use USERDOMAIN and USERNAME environment variables on Windows ( if runtime.GOOS == "windows" then ... else ... ) then you won't need to recover from panic.

I looked at https://github.com/golang/glog and it says: "Send bug reports to golang-nuts@googlegroups.com.". But I would create new issue here on https://github.com/golang/go/issues explaining the problem, and suggested solution. Perhaps someone will submit your change.

If that change is submitted, I am not sure how to update vendor folder in warm.exe, Traefik reverse proxy and Prometheus. I suppose you could always create an issue there too.

Alex

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment