Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

os/user: LookupUser() doesn't find users on macOS when compiled with CGO_ENABLED=0 #24383

Open
jeffreydwalter opened this issue Mar 14, 2018 · 24 comments
Labels
NeedsInvestigation OS-Darwin
Milestone

Comments

@jeffreydwalter
Copy link

@jeffreydwalter jeffreydwalter commented Mar 14, 2018

Please answer these questions before submitting your issue. Thanks!

What version of Go are you using (go version)?

go version go1.9.2 darwin/amd64

Does this issue reproduce with the latest release?

Yes.

What operating system and processor architecture are you using (go env)?

GOARCH="amd64"
GOBIN=""
GOCACHE="/home/jeff_walter/.cache/go-build"
GOEXE=""
GOHOSTARCH="amd64"
GOHOSTOS="freebsd"
GOOS="darwin"
GOPATH="/home/jeff_walter/src/agent"
GORACE=""
GOROOT="/usr/local/go"
GOTMPDIR=""
GOTOOLDIR="/usr/local/go/pkg/tool/freebsd_amd64"
GCCGO="gccgo"
CC="clang"
CXX="clang++"
CGO_ENABLED="0"
CGO_CFLAGS="-g -O2"
CGO_CPPFLAGS=""
CGO_CXXFLAGS="-g -O2"
CGO_FFLAGS="-g -O2"
CGO_LDFLAGS="-g -O2"
PKG_CONFIG="pkg-config"
GOGCCFLAGS="-fPIC -m64 -fno-caret-diagnostics -Qunused-arguments -fmessage-length=0 -fdebug-prefix-map=/tmp/go-build053729832=/tmp/go-build -gno-record-gcc-switches -fno-common"

What did you do?

If I run the following program with sudo I get a different result than I get when I run it as 'someuser' (where 'someuser' is my current logged in user).

package main

import (
    "fmt"
    "os/user"
)

func main() {
    u, err := user.Lookup("someuser")
    if err != nil {
        fmt.Printf("%s", err)
        return
    }

    fmt.Printf("%V", u)
}

What did you expect to see?

$ ./user
&{%!V(string=502) %!V(string=20) %!V(string=someuser) %!V(string=) %!V(string=/Users/someuser)}

What did you see instead?

$ sudo ./user
user: unknown user someuser

@jeffreydwalter jeffreydwalter changed the title "os/user" LookupUser() doesn't find user when run as root. os/user: LookupUser() doesn't find user when run as root. Mar 14, 2018
@odeke-em
Copy link
Member

@odeke-em odeke-em commented Mar 14, 2018

Hello @jeffreydwalter, I am not sure what you were expecting to see but when running as root, you are a different user entirely so hardcoding the username wouldn't work unless you actually used "root"

Here is an update of your repro that'll get you the information from the environment whether ran as root or as an ordinary user, it uses os.ExpandEnv("$USER")

package main

import (
    "fmt"
    "os"
    "os/user"
)

func main() {
    u, err := user.Lookup(os.ExpandEnv("$USER"))
    if err != nil {
        fmt.Printf("%s", err)
        return
    }

    fmt.Printf("%+v\n", u)
}

Ran as a normal user

e$ go run main.go 
&{Uid:501 Gid:20 Username:emmanuelodeke Name:Emmanuel Odeke HomeDir:/Users/emmanuelodeke}

Ran as root

sudo su && go run main.go
&{Uid:0 Gid:0 Username:root Name:System Administrator HomeDir:/var/root}

Hope this helps.

@jeffreydwalter
Copy link
Author

@jeffreydwalter jeffreydwalter commented Mar 14, 2018

@mattn
Copy link
Member

@mattn mattn commented Mar 14, 2018

please use user.Current()

https://golang.org/pkg/os/user/#Current

@odeke-em
Copy link
Member

@odeke-em odeke-em commented Mar 14, 2018

What expecting is to get back the user info for the username I passed in,
regardless of which user I'm running as.

Gotcha, sorry @jeffreydwalter, I misunderstood your intention. Okay, re-running your original repro but searching for "emmanuelodeke" my user, even as root gives me the correct output

&{%!V(string=501) %!V(string=20) %!V(string=emmanuelodeke) %!V(string=Emmanuel Odeke) %!V(string=/Users/emmanuelodeke)}

or with %V fixed to %+v

sh-3.2# go run main.go 
&user.User{Uid:"501", Gid:"20", Username:"emmanuelodeke", Name:"Emmanuel Odeke", HomeDir:"/Users/emmanuelodeke"}

@odeke-em
Copy link
Member

@odeke-em odeke-em commented Mar 14, 2018

And vice versa, searching for root as my ordinary user gives me the correct output
screen shot 2018-03-13 at 6 49 37 pm

@jeffreydwalter
Copy link
Author

@jeffreydwalter jeffreydwalter commented Mar 14, 2018

Thanks @odeke-em,

Are you running on macOS? Did you run as root?

@jeffreydwalter
Copy link
Author

@jeffreydwalter jeffreydwalter commented Mar 14, 2018

Interesting... I wonder if it has to do with the underscore in my username?

@jeffreydwalter
Copy link
Author

@jeffreydwalter jeffreydwalter commented Mar 14, 2018

@odeke-em, so there must be something with my user. If I lookup root, everything works as it does for you.

$ ./user 
&{%!V(string=0) %!V(string=0) %!V(string=root) %!V(string=System Administrator) %!V(string=/var/root)}
🍺  [585] jeff_walter@T2000:~/Code
$ sudo ./user 
&{%!V(string=0) %!V(string=0) %!V(string=root) %!V(string=) %!V(string=/Users/jeff_walter)}
🍺  [586] jeff_walter@T2000:~/Code

@jeffreydwalter
Copy link
Author

@jeffreydwalter jeffreydwalter commented Mar 14, 2018

I tried creating a new user, same issue as my existing user. :/

@odeke-em
Copy link
Member

@odeke-em odeke-em commented Mar 14, 2018

@jeffreydwalter interesting, no dice in a new shell? How about with other users on there? Without underscores either?

@jeffreydwalter
Copy link
Author

@jeffreydwalter jeffreydwalter commented Mar 14, 2018

@odeke-em no dice... :/ I downloaded iTerm and still no luck. The only things I haven't tried are to log out or reboot.

I created a user without underscores, named user, same stuff:

$ ./user user
Current user: &{502 20 jeff_walter  /Users/jeff_walter}
Looking up username: user
user: unknown user user

$ sudo ./user user
Current user: &{0 0 root  /Users/jeff_walter}
Looking up username: user
user: unknown user user

$ sudo su - user
T2000:~ user$ ./user user
Current user: &{503 20 user  /Users/user}
Looking up username: user
&{503 20 user  /Users/user}T2000:~ user$ exit
logout

$ sudo su -
T2000:~ root# /Users/user/user user
Current user: &{0 0 root  /var/root}
Looking up username: user

This is really baffling to me... :)

@crvv
Copy link
Contributor

@crvv crvv commented Mar 14, 2018

There are two implementations of user.Lookup.

  1. Parse /etc/passwd
  2. Use getpwnam_r in libc

I can't find my normal user in /etc/passwd on macOS, but root is in it.
With export CGO_ENABLED=0, I can reproduce this bug.

@odeke-em
Copy link
Member

@odeke-em odeke-em commented Mar 14, 2018

@crvv on mine with CGO_ENABLED=0 or with CGO_ENABLED=1, I get the same results: it works for me.

/cc @kevinburke @ianlancetaylor @bradfitz

@agnivade
Copy link
Contributor

@agnivade agnivade commented Mar 14, 2018

I am on linux/amd64 (Ubuntu 16.04). Unable to reproduce. With both CGO_ENABLED=0 and CGO_ENABLED=1.

@crvv
Copy link
Contributor

@crvv crvv commented Mar 14, 2018

@odeke-em
What is your OS version?
Can you find the normal user in /etc/passwd?

@jeffreydwalter
Copy link
Author

@jeffreydwalter jeffreydwalter commented Mar 14, 2018

I do have GCO_ENABLED=0 set. I am only able to reproduce this issue on macOS.

@odeke-em
Copy link
Member

@odeke-em odeke-em commented Mar 14, 2018

@crvv

What is your OS version?

MacOS

Darwin Emmanuels-MacBook-Pro-2.local 15.6.0 Darwin Kernel Version 15.6.0: Sun Jun  4 21:43:07 PDT 2017; root:xnu-3248.70.3~1/RELEASE_X86_64 x86_64

Can you find the normal user in /etc/passwd?

No, I can't

cat /etc/passwd | grep -i emm

produces nothing

@jeffreydwalter
Copy link
Author

@jeffreydwalter jeffreydwalter commented Mar 14, 2018

Apparently, macOS doesn't really use the /etc/passwd file anymore. Apparently, you can get the user info by parsing some plist files.

See https://apple.stackexchange.com/a/186899

$ sudo defaults read /var/db/dslocal/nodes/Default/users/user.plist uid
(
    503
)

$ sudo defaults read /var/db/dslocal/nodes/Default/users/user.plist gid
(
    20
)

$ sudo defaults read /var/db/dslocal/nodes/Default/users/user.plist passwd
(
    "********"
)

$ sudo defaults read /var/db/dslocal/nodes/Default/users/user.plist name
(
    user
)

@jeffreydwalter
Copy link
Author

@jeffreydwalter jeffreydwalter commented Mar 14, 2018

Just confirmed that everything works fine if I compile with CGO_ENABLED=1. So, seems like parsing /etc/passwd is no bueno on macOS.

@jeffreydwalter
Copy link
Author

@jeffreydwalter jeffreydwalter commented Mar 14, 2018

So, to recap. Compiling with GCO_ENABLED=0 and calling user.Lookup() on a user other than yourself or root fails.

The following commands were run as jeff_walter:

$ ./user jeff_walter
Current user: &{502 20 jeff_walter  /Users/jeff_walter}
Looking up username: jeff_walter
&{502 20 jeff_walter  /Users/jeff_walter}

$ sudo ./user jeff_walter
Current user: &{0 0 root  /Users/jeff_walter}
Looking up username: jeff_walter
user: unknown user jeff_walter

$ ./user user
Current user: &{502 20 jeff_walter  /Users/jeff_walter}
Looking up username: user
user: unknown user user

$ sudo ./user user
Current user: &{0 0 root  /Users/jeff_walter}
Looking up username: user
user: unknown user user

$ ./user root
Current user: &{502 20 jeff_walter  /Users/jeff_walter}
Looking up username: root
&{0 0 root System Administrator /var/root}

$ sudo ./user root
Current user: &{0 0 root  /Users/jeff_walter}
Looking up username: root
&{0 0 root  /Users/jeff_walter} <--- THIS ALSO LOOKS ODD, BUT I GUESS SORT OF MAKES SENSE. STILL SEEMS UNEXPECTED.

The following commands were run as root:

$ ./user root
Current user: &{0 0 root  /var/root}
Looking up username: root
&{0 0 root  /var/root}

$ ./user jeff_walter
Current user: &{0 0 root  /var/root}
Looking up username: jeff_walter
user: unknown user jeff_walter

@ALTree ALTree added the NeedsInvestigation label Mar 14, 2018
@jeffreydwalter jeffreydwalter changed the title os/user: LookupUser() doesn't find user when run as root. os/user: LookupUser() doesn't find users on macOS when compiled with GCO_ENABLED=0 Mar 14, 2018
@andybons andybons added this to the Unplanned milestone Mar 26, 2018
@andybons
Copy link
Member

@andybons andybons commented Mar 26, 2018

Compiling your program and running (both as root and as my normal username) with CGO_ENABLED=1 and CGO_ENABLED=0 has the same exact output for me on macOS High Sierra 10.13.3:

package main

import (
	"fmt"
	"os"
	"os/user"
)

func main() {
	_, err := user.Lookup(os.Args[1])
	if err != nil {
		fmt.Printf("%v\n", err)
		return
	}
	fmt.Println("found user")
}
$ whoami
andybons
$ CGO_ENABLED=0 go build -o user user.go && ./user rando
user: unknown user rando
$ CGO_ENABLED=0 go build -o user user.go && ./user andybons
found user
$ CGO_ENABLED=1 go build -o user user.go && ./user rando
user: unknown user rando
$ CGO_ENABLED=1 go build -o user user.go && ./user andybons
found user

@bradfitz bradfitz changed the title os/user: LookupUser() doesn't find users on macOS when compiled with GCO_ENABLED=0 os/user: LookupUser() doesn't find users on macOS when compiled with CGO_ENABLED=0 Mar 26, 2018
@jeffreydwalter
Copy link
Author

@jeffreydwalter jeffreydwalter commented Mar 27, 2018

@andybons yes, the code works to find the user you are running as. It does not find other users.

I see in your comment, you said, "both as root and as my normal username", but in the output you pasted, I don't see where you ran it as root.

@kazzmir
Copy link

@kazzmir kazzmir commented Feb 19, 2019

I ran into this issue recently using go 1.11.2 when trying to build a darwin binary from linux. The macos host is 10.14.

Given this code

package main

import "fmt"
import "os/user"

func main(){
    me, err := user.Lookup("jon")
    if err != nil {
        fmt.Printf("Could not lookup user 'jon': %v\n", err)
    } else {
        fmt.Printf("Found user 'jon': %+v\n", me)
    }
}

Building this on linux with this command:

# Linux host
$ GOARCH=amd64 GOOS=darwin CGO_ENABLED=0 go build x.go

Or leaving CGO_ENABLED=0 out builds the same way, results in a 64-bit mach-o program that can be executed on macos and produces different behavior depending on whether it is run as root or not.

# Macos host
$ ./x
Found user 'jon': &{Uid:503 Gid:20 Username:jon Name: HomeDir:/Users/jon}
$ sudo ./x
Could not lookup user 'jon': user: unknown user jon

Interestingly (and this is somewhat unrelated to the current issue), if I try to use CGO_ENABLED=1 on my linux host while building a darwin binary then the resulting binary cannot be executed on the macos host due to missing libraries (libc.so.6 in particular).

# Linux host
$ GOARCH=amd64 GOOS=darwin CGO_ENABLED=1 go build x.go
../../Downloads/go1.11.2/src/os/user/getgrouplist_darwin.go: In function ‘mygetgrouplist’:
../../Downloads/go1.11.2/src/os/user/getgrouplist_darwin.go:16:11: warning: implicit declaration of function ‘getgrouplist’; did you mean ‘mygetgrouplist’? [-Wimplicit-function-declaration]
  int rv = getgrouplist(user, (int) group, buf, ngroups);
           ^~~~~~~~~~~~
           mygetgrouplist

(Also note that weird warning as well, but seems harmless I guess)

# Macos host
$ ./x
dyld: Library not loaded: libc.so.6
  Referenced from: /Users/jon/tmp/./x
  Reason: image not found
Abort trap: 6
$ otool -L x
./x:
	/usr/lib/libSystem.B.dylib (compatibility version 0.0.0, current version 0.0.0)
	libc.so.6 (compatibility version 0.0.0, current version 0.0.0)
	libpthread.so.0 (compatibility version 0.0.0, current version 0.0.0)

The mach-o binary produced with CGO_ENABLED=0 has just libSystem.B.dylib as a dependency, rather than libc and libpthread.

For me what this all means is that I must produce the darwin binary on the macos host itself, since CGO_ENABLED=1 is required to get the user.Lookup function to work properly. As others have noted, when CGO_ENABLED=0 is used when building on macos then user.Lookup fails while the program is running under root.

@pingleig
Copy link

@pingleig pingleig commented Nov 11, 2020

I came across the problem and had a workaround by shell out to macOS's directory service tool dscacheutil.
The credit goes to @tweekmonster https://github.com/tweekmonster/luser. The snippet is attached at the end of the comment.

The root cause is macOS queries directory service instead of relying on /etc/passwd, you can't even find your own name (cat /etc/passwd | grep $USER). So when cgo is disabled, os/user.Lookup simply checks /etc/passwd and finds nothing.

This didn't cause a big problem because we take another path when looking for current user and then fallback to lookupUser.

However, I think we should address this problem because:

  • It has valid use cases, e.g. a process running as root spawn a new process as another user based on config, it wants to check the target user does exists, but will always fail when cgo is disabled.
  • It might even be the case for BSD distros as well (based on man page, I don't have BSD systems at hand)

Some possible solutions:

  • Document it in os/user.Lookup and stating when not using cgo or cross compile it will have problem on macOS, and user should shell out or use a third party library.
  • Create a new os/user/lookup_darwin.go to use dscacheutil (see snippet) or put it under a supplementary package, x/sys seems to be the closest one but shells out is not low level syscall...

References

Snippet

To build and run it on a mac

echo "Without CGO"
CGO_ENABLED=0 go build -o macuserlookup-gostd main.go
./macuserlookup-gostd --user $USER
sudo ./macuserlookup-gostd --user $USER

echo "With CGO"
CGO_ENABLED=1 go build -o macuserlookup-cgo main.go
./macuserlookup-cgo --user $USER
sudo ./macuserlookup-cgo --user $USER

echo "With dscacheutil"
go build -o macuserlookup-ds main.go
./macuserlookup-ds --user $USER --method ds
sudo ./macuserlookup-ds --user $USER --method ds

The output should be like NOTE: only using std without cgo failed to lookup myself from root

Without CGO
./macuserlookup-gostd 2020/11/10 19:18:32 Current user is at15 Look up user at15 using std
./macuserlookup-gostd 2020/11/10 19:18:32 Found user at15 using std: &{501 20 at15  /Users/at15}
Password:
./macuserlookup-gostd 2020/11/10 19:18:40 Current user is root Look up user at15 using std
./macuserlookup-gostd 2020/11/10 19:18:40 Look up user at15 using std failed: user: unknown user at15
With CGO
./macuserlookup-cgo 2020/11/10 19:18:40 Current user is at15 Look up user at15 using std
./macuserlookup-cgo 2020/11/10 19:18:40 Found user at15 using std: &{501 20 at15 Pinglei Guo /Users/at15}
./macuserlookup-cgo 2020/11/10 19:18:40 Current user is root Look up user at15 using std
./macuserlookup-cgo 2020/11/10 19:18:40 Found user at15 using std: &{501 20 at15 Pinglei Guo /Users/at15}
With dscacheutil
./macuserlookup-ds 2020/11/10 19:18:41 Current user is at15 Look up user at15 using ds
./macuserlookup-ds 2020/11/10 19:18:41 Found user at15 using ds: &{501 20 at15 Pinglei Guo /Users/at15}
./macuserlookup-ds 2020/11/10 19:18:41 Current user is root Look up user at15 using ds
./macuserlookup-ds 2020/11/10 19:18:41 Found user at15 using ds: &{501 20 at15 Pinglei Guo /Users/at15}
package main

/* macuserlookup shows how current os/user.Lookup does not work when not using cgo.
It also provides a workaround using dscacheutil based on https://github.com/tweekmonster/luser
The original issue is found in https://github.com/golang/go/issues/24383#issuecomment-372908869

echo "Without CGO"
CGO_ENABLED=0 go build -o macuserlookup-gostd main.go
./macuserlookup-gostd --user $USER
sudo ./macuserlookup-gostd --user $USER

echo "With CGO"
CGO_ENABLED=1 go build -o macuserlookup-cgo main.go
./macuserlookup-cgo --user $USER
sudo ./macuserlookup-cgo --user $USER

echo "With dscacheutil"
go build -o macuserlookup-ds main.go
./macuserlookup-ds --user $USER --method ds
sudo ./macuserlookup-ds --user $USER --method ds

References

- https://superuser.com/questions/191330/users-dont-appear-in-etc-passwd-on-mac-os-x
- man page for getpwnam_r `These functions obtain information from opendirectoryd(8)`
  - iOS https://developer.apple.com/library/archive/documentation/System/Conceptual/ManPages_iPhoneOS/man3/getpwnam_r.3.html
  - FreeBSD https://www.freebsd.org/cgi/man.cgi?getpwnam_r
  - Linux https://linux.die.net/man/3/getpwnam_r
*/

import (
	"flag"
	"fmt"
	"log"
	"os"
	"os/exec"
	"os/user"
	"strings"
)

var (
	fUser   string
	fMethod string
)

func init() {
	// Set binary name to prefix because we use different name in the example command
	log.SetPrefix(os.Args[0] + " ")
}

func main() {
	flag.StringVar(&fUser, "user", "at15", "User to lookup")
	flag.StringVar(&fMethod, "method", "std", "std (use cgo), dsc (shell out to macOS's directory service)")
	flag.Parse()
	lookUp(fMethod, fUser)
}

func lookUp(method string, username string) {
	var (
		u   *user.User
		err error
	)
	log.Printf("Current user is %s Look up user %s using %s", os.Getenv("USER"), username, method)
	switch method {
	case "std":
		u, err = user.Lookup(username)
	case "ds":
		u, err = dsLookup(username)
	default:
		log.Fatalf("Unsupported look up method %s", method)
	}
	if err != nil {
		log.Fatalf("Look up user %s using %s failed: %s", username, method, err)
	} else {
		log.Printf("Found user %s using %s: %s", username, method, u)
	}
}

const (
	// dsbin is a cli for querying macOS directory service.
	dsbin = "dscacheutil"
)

// dsLookup shells out to dscacheutil to get uid, gid from username.
func dsLookup(username string) (*user.User, error) {
	// dscacheutil -q user -a name at15
	// name: at15
	// password: ********
	// uid: 123456
	// gid: 123456
	// dir: /Users/at15
	// shell: /bin/zsh
	// gecos: At15
	//
	m, err := runDS("-q", "user", "-a", "name", username)
	if err != nil {
		return nil, err
	}
	u := &user.User{
		Uid:      m["uid"],
		Gid:      m["gid"],
		Username: m["name"],
		Name:     m["gecos"],
		HomeDir:  m["dir"],
	}
	if u.Username == "" || u.Username != username {
		return nil, user.UnknownUserError(username)
	}
	return u, nil
}

// runDS shells out query to dscacheutil and parse the output to key value pair.
func runDS(args ...string) (map[string]string, error) {
	b, err := exec.Command(dsbin, args...).CombinedOutput()
	if err != nil {
		cmd := strings.Join(append([]string{dsbin}, args...), " ")
		return nil, fmt.Errorf("error query directory service using %s: %w output %s", cmd, err, b)
	}
	return parseDSOutput(string(b))
}

// parseDSOutput splits dscacheutil output into key value pair.
// It returns error if no pair is found.
func parseDSOutput(s string) (map[string]string, error) {
	const sep = ": "
	lines := strings.Split(s, "\n")
	m := make(map[string]string)
	for _, line := range lines {
		keyEnd := strings.Index(line, sep)
		if keyEnd <= 0 { // the name must be longer than 1, i.e. `: value` does not exist
			continue
		}
		m[line[:keyEnd]] = line[keyEnd+len(sep):]
	}
	if len(m) == 0 {
		return m, fmt.Errorf("error parse %s output %s", dsbin, s)
	}
	return m, nil
}

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
NeedsInvestigation OS-Darwin
Projects
None yet
Development

No branches or pull requests

10 participants