Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

os: Environ doc comment does not describe keys containing "=" #49886

Open
bcmills opened this issue Nov 30, 2021 · 8 comments
Open

os: Environ doc comment does not describe keys containing "=" #49886

bcmills opened this issue Nov 30, 2021 · 8 comments

Comments

@bcmills
Copy link
Member

@bcmills bcmills commented Nov 30, 2021

In investigating #49882, I noticed some other suspicious environment variables from os.Environ(). I'll write a regression test demonstrating the bug and then update here.

@rsc
Copy link
Contributor

@rsc rsc commented Nov 30, 2021

The "=FOO" is a real environment variable on Windows and it probably needs to be preserved in os.Environ, even if it is incompatible with os.Setenv. Setenv is a helper but not necessarily capable of everything.

@bcmills
Copy link
Member Author

@bcmills bcmills commented Nov 30, 2021

At the very least, we need to update the os.Environ docs, since they currently promise the form “key=value” and do not mention the possibility of either an empty key or one containing "=".

Moreover, it is common to use os.Environ() as the basis for the Env field in an exec.Cmd, which also does not document any specific behavior for keys beginning with =. If these variables can't be set, can they also not be transferred to subprocesses? (Or are they ignored, or can they be set only for subprocesses? If parts of Cmd.Env may be silently ignored, that also seems like a serious problem )

(@bufflig, @alexbrainman: some clarification on the standard Windows semantics might be helpful here.)

@gopherbot
Copy link

@gopherbot gopherbot commented Nov 30, 2021

Change https://golang.org/cl/367850 mentions this issue: os: test that LookupEnv reports all keys found in Environ

@bcmills
Copy link
Member Author

@bcmills bcmills commented Nov 30, 2021

Empirically, it appears that if we parse through to the second =, both LookupEnv and Setenv can handle the resulting key.

It is not obvious to me how many = symbols the caller is expected to parse through in general. (For example: should a correct consumer include arbitrarily many leading = symbols, or just one?)

@bcmills bcmills changed the title os: Environ on Windows reports variables that are incompatible with Setenv os: Environ doc comment does not describe keys containing "=" Dec 1, 2021
@bufflig
Copy link
Contributor

@bufflig bufflig commented Dec 1, 2021

Microsofts own docs (like https://docs.microsoft.com/en-us/windows/win32/api/processenv/nf-processenv-getenvironmentstrings) claims that environment names can not contain '='. They do not specifically mention empty environment variable names though, so I would personally interpret =C:=C:\golang as having the key '' and the value 'C:=C:\golang' - although that is probably not what was intended...

As the windows API for creating a (sub)process takes a null terminated block of null terminated strings 'name=value\0' with possibly just the convention of "environment variable names can not contain '='", it's possible you can technically set whatever you want, completely ignoring the key/value semantics. I guess we just have to behave as closely as possible to GetEnvironmentStrings, GetEnvironmentVariable and SetEnvironmentVariable. I can write some C programs to check how they behave in these situations.

@bcmills
Copy link
Member Author

@bcmills bcmills commented Dec 1, 2021

They do not specifically mention empty environment variable names though, so I would personally interpret =C:=C:\golang as having the key '' and the value 'C:=C:\golang' - although that is probably not what was intended...

Empirically the key in that example is =C:. (I tried both approaches in CL 367850, and the “include the leading = in the key” approach is what matched the observed behavior of LookupEnv.)

Another such key noticed during testing is =ExitCode.

I guess we just have to behave as closely as possible to GetEnvironmentStrings, GetEnvironmentVariable and SetEnvironmentVariable. I can write some C programs to check how they behave in these situations.

SGTM.

gopherbot pushed a commit that referenced this issue Dec 3, 2021
For #49886

Change-Id: Ie3a7f12a0d30ec719caf375e7be30cc4a5796c3f
Reviewed-on: https://go-review.googlesource.com/c/go/+/367850
Trust: Bryan Mills <bcmills@google.com>
Run-TryBot: Bryan Mills <bcmills@google.com>
Reviewed-by: Ian Lance Taylor <iant@golang.org>
TryBot-Result: Gopher Robot <gobot@golang.org>
@bufflig
Copy link
Contributor

@bufflig bufflig commented Dec 3, 2021

Ok, it seems to behave like this in the barest MSVC program I could muster (completely empirically, I can find no docs describing this):

Environment variables beginning with exactly one '=' are allowed, you can set them and get them with SetEnvironmentVariable and GetEnvironmentVariable. No more than one single '=' in the beginning of a variable name is allowed. It's possible the interpretation should be that the first character can be whatever, as a key can not have zero characters.

When i.e. cmd.exe prints environments, these variables are not printed, i.e. they are "hidden", but they are in the environment block returned by GetEnvironmentStrings and can be retrieved by calling GetEnvironmentVariable in subprocesses (that inherit the environment).

You can add something that in the environment block looks like "===FOO=bar", either by using your own environment block when calling CreateProcess or by doing SetEnvironmentVariable("=","=FOO=bar"). But as expected, that can only be retrieved in a subprocess as GetEnvironmentVariable("=",...).

As a fun quirk/bug, you can actually do SetEnvironmentVariable("=FOO","bar=baz"); and then GetEnvironmentVariable("=FOO=bar",...) and get "baz", but only if you do it in the same process, a subprocess can only retrieve "=FOO" if it inherited the environment. I have no idea what kind of caching makes that "work" when you do it in the same process... I don't think we need to implement that.

I think the reasonable and compatible solution is to simply allow one prefix '=' in a key, and document that that is possible in Windows.

@alexbrainman
Copy link
Member

@alexbrainman alexbrainman commented Dec 9, 2021

I guess we just have to behave as closely as possible to GetEnvironmentStrings, GetEnvironmentVariable and SetEnvironmentVariable

I agree with @bufflig sentiment here. And so far I don't see any problems with our current code.

If anyone really interested to see where environment variables, like =C:, come from, see Undocumented Dynamic variables section in

https://ss64.com/nt/syntax-variables.html

and in particular

https://devblogs.microsoft.com/oldnewthing/20100506-00/?p=14133

Alex

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
Linked pull requests

Successfully merging a pull request may close this issue.

None yet
5 participants