os/user: query systemd’s User/Group Record Lookup API in non-cgo environments before parsing /etc/passwd?

[stapelberg]

I recently learnt about systemd’s “User/Group Record Lookup API via Varlink”.

It’s a new service introduced by systemd v245 (released March 6th 2020) which can take the role of getpwnam(3) and related calls.

We could consider this as an option for os/user, which currently integrates with Name Service Switch (NSS) only when cgo is available. In non-cgo environments, we could try querying systemd-userdbd.service(8) before falling back to the current behavior of parsing /etc/passwd.

There are two possible ways to query the service:

1. Parsing userdbctl --output=json. The upside is that userdbctl itself queries NSS if systemd-userdbd is not working. The downside is that we are relying on an external process. I’m not sure how this is regarded in the standard library, and whether overhead of os/user is of concern?

2. If we wanted to avoid the process overhead, we could integrate with systemd-userdbd directly. I implemented a <100-line proof of concept which prints just the user name. The User/Group Record Lookup API uses a subset of varlink, which boils down to sending and receiving 0-terminated JSON messages over a Unix socket.

Appendix:

userdbctl --output=json Output

{
	"userName" : "root",
	"uid" : 0,
	"gid" : 0,
	"homeDirectory" : "/root",
	"shell" : "/bin/zsh",
	"privileged" : {
		"hashedPassword" : [
			"$6$wKUquM/L7HzvV5eI$dhexy2iU1efxnYe6k0rm4qT8D1TOgAVtYYyjC5wZeClK.ETeCfPCn0xmwZKK/l8MtzJhtSPTsWEopILQXbA.40"
		]
	},
	"passwordChangeNow" : false,
	"lastPasswordChangeUSec" : 1588377600000000,
	"status" : {
		"12f5c9abd57d4914abe2d5cb4958378b" : {
			"service" : "io.systemd.NameServiceSwitch"
		}
	}
}
{
	"userName" : "nobody",
	"uid" : 65534,
	"gid" : 65534,
	"realName" : "Nobody",
	"homeDirectory" : "/",
	"shell" : "/bin/nologin",
	"passwordChangeNow" : false,
	"lastPasswordChangeUSec" : 1588377600000000,
	"status" : {
		"12f5c9abd57d4914abe2d5cb4958378b" : {
			"service" : "io.systemd.NameServiceSwitch"
		}
	}
}
{
	"userName" : "systemd-network",
	"uid" : 983,
	"gid" : 983,
	"realName" : "systemd Network Management",
	"homeDirectory" : "/",
	"shell" : "/bin/nologin",
	"passwordChangeNow" : false,
	"lastPasswordChangeUSec" : 1588377600000000,
	"status" : {
		"12f5c9abd57d4914abe2d5cb4958378b" : {
			"service" : "io.systemd.NameServiceSwitch"
		}
	}
}
{
	"userName" : "systemd-resolve",
	"uid" : 982,
	"gid" : 982,
	"realName" : "systemd Resolver",
	"homeDirectory" : "/",
	"shell" : "/bin/nologin",
	"passwordChangeNow" : false,
	"lastPasswordChangeUSec" : 1588377600000000,
	"status" : {
		"12f5c9abd57d4914abe2d5cb4958378b" : {
			"service" : "io.systemd.NameServiceSwitch"
		}
	}
}
{
	"userName" : "systemd-timesync",
	"uid" : 981,
	"gid" : 981,
	"realName" : "systemd Time Synchronization",
	"homeDirectory" : "/",
	"shell" : "/bin/nologin",
	"passwordChangeNow" : false,
	"lastPasswordChangeUSec" : 1588377600000000,
	"status" : {
		"12f5c9abd57d4914abe2d5cb4958378b" : {
			"service" : "io.systemd.NameServiceSwitch"
		}
	}
}
{
	"userName" : "systemd-coredump",
	"uid" : 980,
	"gid" : 980,
	"realName" : "systemd Core Dumper",
	"homeDirectory" : "/",
	"shell" : "/bin/nologin",
	"passwordChangeNow" : false,
	"lastPasswordChangeUSec" : 1588377600000000,
	"status" : {
		"12f5c9abd57d4914abe2d5cb4958378b" : {
			"service" : "io.systemd.NameServiceSwitch"
		}
	}
}
{
	"userName" : "systemd-network",
	"uid" : 101,
	"gid" : 101,
	"realName" : "network",
	"homeDirectory" : "/run/systemd/netif",
	"shell" : "/bin/false",
	"passwordChangeNow" : false,
	"lastPasswordChangeUSec" : 1588377600000000,
	"status" : {
		"12f5c9abd57d4914abe2d5cb4958378b" : {
			"service" : "io.systemd.NameServiceSwitch"
		}
	}
}
{
	"userName" : "systemd-resolve",
	"uid" : 105,
	"gid" : 105,
	"realName" : "resolve",
	"homeDirectory" : "/run/systemd/resolve",
	"shell" : "/bin/false",
	"passwordChangeNow" : false,
	"lastPasswordChangeUSec" : 1588377600000000,
	"status" : {
		"12f5c9abd57d4914abe2d5cb4958378b" : {
			"service" : "io.systemd.NameServiceSwitch"
		}
	}
}
{
	"userName" : "sshd",
	"uid" : 102,
	"gid" : 102,
	"homeDirectory" : "/",
	"shell" : "/bin/false",
	"status" : {
		"12f5c9abd57d4914abe2d5cb4958378b" : {
			"service" : "io.systemd.NameServiceSwitch"
		}
	}
}
{
	"userName" : "messagebus",
	"uid" : 106,
	"gid" : 106,
	"homeDirectory" : "/var/run/dbus",
	"shell" : "/bin/false",
	"status" : {
		"12f5c9abd57d4914abe2d5cb4958378b" : {
			"service" : "io.systemd.NameServiceSwitch"
		}
	}
}
{
	"userName" : "lightdm",
	"uid" : 979,
	"gid" : 979,
	"realName" : "Light Display Manager",
	"homeDirectory" : "/var/lib/lightdm",
	"shell" : "/bin/nologin",
	"passwordChangeNow" : false,
	"lastPasswordChangeUSec" : 1588377600000000,
	"status" : {
		"12f5c9abd57d4914abe2d5cb4958378b" : {
			"service" : "io.systemd.NameServiceSwitch"
		}
	}
}
{
	"userName" : "polkitd",
	"uid" : 978,
	"gid" : 978,
	"realName" : "PolicyKit Daemon",
	"homeDirectory" : "/etc/polkit-1",
	"shell" : "/bin/nologin",
	"passwordChangeNow" : false,
	"lastPasswordChangeUSec" : 1588377600000000,
	"status" : {
		"12f5c9abd57d4914abe2d5cb4958378b" : {
			"service" : "io.systemd.NameServiceSwitch"
		}
	}
}

[ianlancetaylor]

We could also implement LDAP, but we've historically decided not to. What is the use case we're addressing here? Why do programs that use os/user want to avoid using cgo? And should this functionality live in a third party package instead?

[stapelberg]

We could also implement LDAP, but we've historically decided not to.

While related, that’s a much different call IMO: here, we implement 1 reasonably small API and get access to all NSS plugins.

What is the use case we're addressing here?
Why do programs that use os/user want to avoid using cgo?

I’m not saying they do.

I’m saying that when disabling cgo for whichever reason, there is a feature disparity in os/user, and we could decide to fix it for some environments. Not sure if that scenario is something we should care about. Just wanted to mention this and see what people think.

And should this functionality live in a third party package instead?

It certainly could. The downside of course is that users then first need to discover that package. I think that instead of having each programmer find out about the os/user behavior and having them pull in a third-party package, it would be nicer if os/user just worked in that scenario.

[dmitshur]

Also /cc @bradfitz @kevinburke per owners.

[bradfitz]

While related, that’s a much different call IMO: here, we implement 1 reasonably small API and get access to all NSS plugins.

That's a pretty compelling argument, actually.

As for whether we use os/exec: I don't feel strongly either way. Both options involve encoding/json, and the do-it-ourselves way doesn't look super invasive. I'd say do it ourselves if that meant running in more environments or if it was a hot path, but I can't imagine any environment that wouldn't have userdbctl but would have systemd, and I can't imagine this being too hot.

[stapelberg]

Cool! Does someone want to send a CL, or should I give it a shot when I find a minute?

[stapelberg]

Looks like nobody wants to race me to it, so I’ll give it a shot over the weekend.

[stapelberg]

Here’s how far I have come so far: stapelberg@87fae81

I’ll try and spend a few more minutes on turning this into a Gerrit CL for proper review next week.

[stapelberg]

Sent the corresponding CL: https://go-review.googlesource.com/c/go/+/256218

[gopherbot]

Change https://go.dev/cl/459455 mentions this issue: os/user: lookup Linux users and groups via systemd userdb