proposal: os: add EnvironMap function

[lbordowitz]

Proposal Details

There are two primary use-cases for the os.Environ function:

1. Passing through environment variables to a sub-command or other executable
2. Parsing all environment variables into a map

The latter requires going through each string in os.Environ(), usually splitting it on the = sign, and then getting the last part. This is perilous and potentially bug-prone: what if there's an "=" character in the value of the environment variable? Re-joining everything ""? It is not efficient. A widespread need with so many different implementations recommends a library standard function to cut through the noise.

The parsing method in the unix syscall.Getenv could be abstracted to provide a method EnvironMap() map[string]string for the entirety of the envs, and the windows syscall GetEnvironmentStringsW can be similarly parsed. I recommend adding this method in the syscall package, and then exposing it in os as well.

[gabyhelp]

Related Issues

• proposal: os: make a Go 2 type for environment variables? #25252
• os: Getenv documentation update to point user to syscall.Getenv #9676 (closed)
• proposal: os: Splitenv #62038
• x/sys/plan9, x/sys/windows: Environment functions should just refer to syscall functions #10803 (closed)
• os/exec: add Cmd.SetEnv method #12868 (closed)

_{(Emoji vote if this was helpful or unhelpful; more detailed feedback welcome in this discussion.)}

[seankhliao]

syscall is frozen, see #62038

[ianlancetaylor]

Reopening this as a proposal for the os package.

@lbordowitz The best way to argue for this would be to show examples in third party packages that would be improved with such a function. We would only add this feature if there is reasonably widespread use.

[lbordowitz]

@ianlancetaylor Use cases in repositories with over 10,000 stars:

• In Kubernetes
• In frp, the fast reverse proxy
• In etcd (Used for getting env var names)
• In ethereum Used for getting env var names)
• In OpenTofu (Hence, also probably Terraform)
• In Istio
• In Tailscale
• In Pulumi
• In yay (for filtering env var names)
• In Open Policy Agent

While searching repositories and code, I found it interesting how Environ is used instead of Getenv, and the code then searches for a prefix in the env var names.

[apparentlymart]

I note that in several of those examples there's a filtering step where only a subset of the items from Environ get included in the result. If those cases were rewritten to use the proposed EnvironMap then they would presumably first generate a full map and then either only use a subset of it or delete elements from it before returning.

That makes me consider a different compromise:

package os

import (
    "iter"
)

// EnvironPairs returns a sequence of each of the elements that would
// be returned by [Environ], split into separate name/value pairs.
//
// Any elements of the [Environ] result that do not include "=" as a
// delimiter between name and value are skipped.
func EnvironPairs() iter.Seq2[string, string]

A caller that wants a map without any special filtering could then use maps.Collect with the result of this function to collect the pairs into a map as a separate step: maps.Collect(os.EnvironPairs()). The examples that want to construct a slice of some struct type instead of a map can also use this sequence result to do that without allocating an intermediate map. The examples that want to filter can write a for ... range loop over the os.EnvironPairs result that includes an if statement.

My intention in ignoring elements that don't represent conventional name/value pairs was that maps.Collect(os.EnvironPairs())[anyName] should be equivalent to os.Getenv(anyName) and so the resulting map should not include anything that os.Getenv would not be able to match. However, to properly create that equivalence, EnvironPairs would need to iterate Environ in reverse order so that the earlier elements would be returned last so that duplicate names would have the same priority as for os.Getenv, which is admittedly quite weird for any use other than collecting into a map. Maybe it would be better to prioritize the non-map-collecting use-cases and assume that folks who want to access just a single variable by name would continue using os.Getenv.

(I don't feel strongly about the name EnvironPairs, but I don't know what else to call it.)

[seankhliao]

What's the actual proposed functions here?
Your search only shows that it's common to loop over os.Environ, but not that any particular output format structure is shared between them.

[earthboundkid]

If you search for strings.Cut/strings.SplitN on =, you can find this lots of places including x/tools, but it's frequently working with a derivative of os.Environ, not os.Environ itself:

• https://github.com/golang/tools/blob/c2e79795ae0d753340da9a7b0060f44314f455e5/go/packages/golist.go#L1028-L1033
• https://github.com/golang/tools/blob/c2e79795ae0d753340da9a7b0060f44314f455e5/gopls/internal/cache/session.go#L212
• https://github.com/golang/tools/blob/c2e79795ae0d753340da9a7b0060f44314f455e5/gopls/internal/test/integration/options.go#L185-L190
• https://github.com/golang/tools/blob/c2e79795ae0d753340da9a7b0060f44314f455e5/internal/imports/fix_test.go#L1703

Maybe it should be os.PairFromEnviron([]string) iter.Seq2[string, string].

[earthboundkid]

Maybe a better name is os.KeysValuesFromEnviron.