Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

os/user: LookupGroup does not identify all cases of UnknownGroupError #40334

Open
anonymouse64 opened this issue Jul 21, 2020 · 8 comments
Open

os/user: LookupGroup does not identify all cases of UnknownGroupError #40334

anonymouse64 opened this issue Jul 21, 2020 · 8 comments
Labels
NeedsInvestigation
Milestone

Comments

@anonymouse64
Copy link

@anonymouse64 anonymouse64 commented Jul 21, 2020

What version of Go are you using (go version)?

$ go version
go version go1.14.5 linux/amd64

Does this issue reproduce with the latest release?

I only tested Go 1.14.5, I did not test Go 1.14.6 or tip.

What operating system and processor architecture are you using (go env)?

go env Output
$ go env
GO111MODULE=""
GOARCH="amd64"
GOBIN=""
GOCACHE="/home/user/.cache/go-build"
GOENV="/home/user/.config/go/env"
GOEXE=""
GOFLAGS=""
GOHOSTARCH="amd64"
GOHOSTOS="linux"
GOINSECURE=""
GONOPROXY=""
GONOSUMDB=""
GOOS="linux"
GOPATH="/home/user/go"
GOPRIVATE=""
GOPROXY="https://proxy.golang.org,direct"
GOROOT="/snap/go/6090"
GOSUMDB="sum.golang.org"
GOTMPDIR=""
GOTOOLDIR="/snap/go/6090/pkg/tool/linux_amd64"
GCCGO="gccgo"
AR="ar"
CC="gcc"
CXX="g++"
CGO_ENABLED="1"
GOMOD=""
CGO_CFLAGS="-g -O2"
CGO_CPPFLAGS=""
CGO_CXXFLAGS="-g -O2"
CGO_FFLAGS="-g -O2"
CGO_LDFLAGS="-g -O2"
PKG_CONFIG="pkg-config"
GOGCCFLAGS="-fPIC -m64 -pthread -fmessage-length=0 -fdebug-prefix-map=/tmp/go-build704426793=/tmp/go-build -gno-record-gcc-switches"

What did you do?

https://play.golang.org/p/qEi8JbjG5-4 when built on Linux and run inside a Ubuntu 20.10 chroot inside a Ubuntu 16.04 LXD container inside an Ubuntu 16.04 VM (this setup comes from a cloud building environment) fails to identify the returned error as a user.UnknownGroupError.

I suspect there is something about the implementation that is specific to the chroot, as running the same program outside of the chroot (inside the LXD container) successfully returns. See:

# ./golang-group-lookup 
successful error handling
# chroot /tmp/tmp.evWDSKSvUt/ /golang-group-lookup
error, not matchable (type *errors.errorString), user: lookup groupname thisdoesnotexist: no such process
# uname -a
Linux first-marlin-ubuntu-bartender 4.4.0-185-generic #215-Ubuntu SMP Mon Jun 8 21:53:19 UTC 2020 x86_64 x86_64 x86_64 GNU/Linux

It's possible something about getgrpnam_r in the version of glibc in Ubuntu 20.10 has changed to return a different error code, but as per the man page of getgrpnam_r under "Notes", there are numerous possible error codes that could be used to identify user.UnknownGroupError:

The formulation given above under "RETURN VALUE" is from POSIX.1-2001. It does not call "not found" an error, hence does not specify what value errno might have in this situation. But that makes it impossible to recognize errors. One might argue that according to POSIX errno should be left unchanged if an entry is not found. Experiments on various UNIX-like systems shows that lots of different values occur in this situation: 0, ENOENT, EBADF, ESRCH, EWOULDBLOCK, EPERM and probably others.

As such, I would expect Go to do a "best effort" job at identifying errors returned as user.UnknownGroupError, including at least the documented ENOENT, ESRCH, EBADF, EPERM cases mentioned in the man page (in this case it is ESRCH that is being returned).

Finally, note that the same problem hypothetically affects user.Lookup and the associated user.UnknownUserError, but getpwnam_r is not acting up the same way getgrpnam_r is here.

What did you expect to see?

I expected the program when run inside the chroot to work the same as outside of the chroot.

What did you see instead?

Inside the chroot it fails to identify the returned error as user.UnknownGroupError.

@anonymouse64
Copy link
Author

@anonymouse64 anonymouse64 commented Jul 21, 2020

Further debugging of the issue revealed the issue to actually be related to the fact that we are in the chroot, and hence we do not have things like systemd userdb to respond, however the chroot still had this in it's /etc/nsswitch.conf:

# /etc/nsswitch.conf
#
# Example configuration of GNU Name Service Switch functionality.
# If you have the `glibc-doc-reference' and `info' packages installed, try:
# `info libc "Name Service Switch"' for information about this file.

passwd:         files systemd
group:          files systemd
shadow:         files
gshadow:        files

hosts:          files dns
networks:       files

protocols:      db files
services:       db files
ethers:         db files
rpc:            db files

netgroup:       nis

notice group: files systemd.

So it seems to be that really we have an issue with NSS in this chroot that gets bubbled up to getgrpnam_r as ESRCH, which Go then doesn't recognize as user.UnknownGroupError.

@toothrot toothrot changed the title user.LookupGroup does not identify all cases of UnknownGroupError os/user: LookupGroup does not identify all cases of UnknownGroupError Jul 22, 2020
@toothrot toothrot added the NeedsInvestigation label Jul 22, 2020
@toothrot toothrot added this to the Backlog milestone Jul 22, 2020
@toothrot
Copy link
Contributor

@toothrot toothrot commented Jul 22, 2020

/cc @bradfitz @kevinburke

@kevinburke
Copy link
Contributor

@kevinburke kevinburke commented Jul 22, 2020

Interesting... I'll try to take a look later this week. I don't think this is important enough to get through the code freeze so we'd be targeting 1.15.1 or 1.16 (another way of saying that the fix will not show up in a public release for a while).

@xnox
Copy link

@xnox xnox commented Jul 24, 2020

Checking through nss_systemd code, I do not see it returning anything unusual on the nss module level. It returns things like NSS_STATUS_UNAVAIL NSS_STATUS_NOTFOUND NSS_STATUS_TRYAGAIN. But I didn't check how that translates between libc & nss interface, and golang & libc.

@kevinburke
Copy link
Contributor

@kevinburke kevinburke commented Jul 26, 2020

I'm having trouble reproducing. I am running the same program you are:

package main

import (
	"fmt"
	"os/user"
)

func main() {
	_, err := user.LookupGroup("thisdoesnotexist")
	fmt.Printf("%#v\n", err)
	switch err.(type) {
	case user.UnknownGroupError:
		fmt.Println("successful error handling")
	default:
		fmt.Printf("error, not matchable (type %T), %v\n", err, err)
	}
}

in tmp/test.go, then compiling the Go binary in a Linux container:

$ docker run --tty -v $(PWD)/tmp:/go --interactive meterup/ubuntu-golang:latest /usr/local/go/bin/go build -ldflags '-extldflags "-static"' -o /go/test/test-program ./

then sharing the compiled binary into a Docker container:

$ docker run --tty -v $(PWD)/tmp:/go --interactive meterup/ubuntu-golang:latest /bin/bash --login

running ldd to find remaining links in the binary:

$ ldd /go/test/test-program
	linux-vdso.so.1 (0x00007ffe801af000)
	libpthread.so.0 => /lib/x86_64-linux-gnu/libpthread.so.0 (0x00007feed8a62000)
	libc.so.6 => /lib/x86_64-linux-gnu/libc.so.6 (0x00007feed8671000)
	/lib64/ld-linux-x86-64.so.2 (0x00007feed8c81000)

copying these files into the chroot:

# mkdir /go/test/lib/x86_64-linux-gnu 
# cp -r --dereference /lib/x86_64-linux-gnu/* /go/test/lib/x86_64-linux-gnu

(and similar for the other files)

then, finally, running the program:

chroot /go/test /test-program
&errors.errorString{s:"user: lookup groupname thisdoesnotexist: no such file or directory"}
error, not matchable (type *errors.errorString), user: lookup groupname thisdoesnotexist: no such file or directory

Running with strace (you have to install it into that container):

[pid   456] openat(AT_FDCWD, "/etc/group", O_RDONLY|O_CLOEXEC) = -1 ENOENT (No such file or directory)
[pid   456] futex(0x55f6b8, FUTEX_WAKE_PRIVATE, 1) = 1
[pid   457] <... futex resumed> )       = 0
[pid   457] clock_gettime(CLOCK_MONOTONIC, {tv_sec=2321, tv_nsec=911496659}) = 0
[pid   457] clock_gettime(CLOCK_MONOTONIC, {tv_sec=2321, tv_nsec=911587159}) = 0
[pid   456] write(1, "&errors.errorString{s:\"user: loo"..., 92 <unfinished ...>
[pid   457] futex(0xc00003e948, FUTEX_WAKE_PRIVATE, 1&errors.errorString{s:"user: lookup groupname thisdoesnotexist: no such file or directory"}

So it looks like my program is trying to read /etc/group and failing (a failure I would expect in the chroot). My understanding is /etc/group parse is fallback logic if cgo is not enabled, though I am compiling on Linux with a linux target, so cgo should be enabled... I don't know strace well enough to determine whether getgrnam_r is the one that is trying and failing to read /etc/group, ie. cgo is enabled but it's still failing to read the file, though I am concerned that I am getting different output than you got.

@kevinburke
Copy link
Contributor

@kevinburke kevinburke commented Jul 26, 2020

Odd... I copied the contents of os/user to a third party package so I could add print statements to it, and when I do that, it compiles and runs successfully.

$ docker run --tty -v $(PWD)/tmp:/go --interactive meterup/ubuntu-golang:latest /usr/local/go/bin/go build -ldflags '-extldflags "-static"' -o /go/test/test-program ./
# _/go
/tmp/go-link-139166518/000002.o: In function `mygetgrouplist':
/go/user2/getgrouplist_unix.go:16: warning: Using 'getgrouplist' in statically linked applications requires at runtime the shared libraries from the glibc version used for linking
/tmp/go-link-139166518/000001.o: In function `mygetgrgid_r':
/go/user2/cgo_lookup_unix.go:38: warning: Using 'getgrgid_r' in statically linked applications requires at runtime the shared libraries from the glibc version used for linking
/tmp/go-link-139166518/000001.o: In function `mygetgrnam_r':
/go/user2/cgo_lookup_unix.go:43: warning: Using 'getgrnam_r' in statically linked applications requires at runtime the shared libraries from the glibc version used for linking
/tmp/go-link-139166518/000001.o: In function `mygetpwnam_r':
/go/user2/cgo_lookup_unix.go:33: warning: Using 'getpwnam_r' in statically linked applications requires at runtime the shared libraries from the glibc version used for linking
/tmp/go-link-139166518/000001.o: In function `mygetpwuid_r':
/go/user2/cgo_lookup_unix.go:28: warning: Using 'getpwuid_r' in statically linked applications requires at runtime the shared libraries from the glibc version used for linking
# chroot /go/test /test-program
calling lookup Group using mygetgrnam_r
lookup err: <nil>
"thisdoesnotexist"
successful error handling

@anonymouse64
Copy link
Author

@anonymouse64 anonymouse64 commented Aug 25, 2020

I can share a reproducer environment when I get a chance, sorry been busy with other things... should be soon though

@mvo5
Copy link
Contributor

@mvo5 mvo5 commented Jan 19, 2022

Fwiw, I can easily reproduce this by installing the sssd package on Ubuntu which will add the sss to the nsswitch.conf. The fix for this seems fairly straightforward
0001-user-handle-ENOENT-from-getpw-uid-nam-_r.patch.gz

(tested the issue with sssd on 20.04 and 21.10 and the fix as well).

Alternatively the fmt.Errorf() return could use %w instead of the current %v to make it possible for the upper layer to reliably identify this.

mvo5 added a commit to mvo5/snappy that referenced this issue Jan 19, 2022
When the `sssd` package is installed on a Ubuntu system the
`TestUserMaybeSudoUser` test will fail because then `user.Lookup`
returns a generic error
```
user: lookup username guy: no such file or directory
```
because in the go layer the getpw{uid,nam}_r() returned ENOENT.

This is already reported upstream as
golang/go#40334

This commit works around this upstream issue by checking for
ENOENT on the error from user.Lookup(). It's not nice but the
best I could come up to to handle this case. On my 21.10 system
sssd is installed as part of `ubuntu-desktop-minimal`.
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
NeedsInvestigation
Projects
None yet
Development

No branches or pull requests

5 participants