Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

cmd/go: add a way to query for non-defaults in the env #34208

Closed
mvdan opened this issue Sep 10, 2019 · 25 comments
Closed

cmd/go: add a way to query for non-defaults in the env #34208

mvdan opened this issue Sep 10, 2019 · 25 comments

Comments

@mvdan
Copy link
Member

mvdan commented Sep 10, 2019

It's common to ask users to provide go version and go env when they report a bug. There is one problem with that, though - the output is getting huge, and most of it is generally useless. For example:

$ go version
go version devel +b38be35e4c Tue Sep 10 09:12:32 2019 +0000 linux/amd64
$ go env
GO111MODULE="on"
GOARCH="amd64"
GOBIN="/home/mvdan/go/bin"
GOCACHE="/home/mvdan/go/cache"
GOENV="/home/mvdan/.config/go/env"
GOEXE=""
GOFLAGS=""
GOHOSTARCH="amd64"
GOHOSTOS="linux"
GONOPROXY="brank.as/*"
GONOSUMDB="brank.as/*"
GOOS="linux"
GOPATH="/home/mvdan/go"
GOPRIVATE="brank.as/*"
GOPROXY="https://proxy.golang.org"
GOROOT="/home/mvdan/tip"
GOSUMDB="sum.golang.org"
GOTMPDIR=""
GOTOOLDIR="/home/mvdan/tip/pkg/tool/linux_amd64"
GCCGO="gccgo"
AR="ar"
CC="gcc"
CXX="g++"
CGO_ENABLED="1"
GOMOD="/dev/null"
CGO_CFLAGS="-g -O2"
CGO_CPPFLAGS=""
CGO_CXXFLAGS="-g -O2"
CGO_FFLAGS="-g -O2"
CGO_LDFLAGS="-g -O2"
PKG_CONFIG="pkg-config"
GOGCCFLAGS="-fPIC -m64 -pthread -fmessage-length=0 -fdebug-prefix-map=/tmp/go-build021884514=/tmp/go-build -gno-record-gcc-switches"

What's special about my environment? You can easily spot GO111MODULE or GOBIN as they are commonly set, and they are near the top. But what variables have I actually modified from the defaults on my system?

It would be useful to quickly see what Go env variables were specifically modified by a user. That is, what could be special about the user's environment, that could help in reproducing a bug they're running into.

I can kind of get that right now with a bit of shell hackery, showing the changes between my go env and the same go env with an empty environment:

$ diff <(env -i /usr/bin/go env) <(/usr/bin/go env) | grep '^>'
> GO111MODULE="on"
> GOBIN="/home/mvdan/go/bin"
> GOCACHE="/home/mvdan/go/cache"
> GOENV="/home/mvdan/.config/go/env"
> GONOPROXY="brank.as/*"
> GONOSUMDB="brank.as/*"
> GOPATH="/home/mvdan/go"
> GOPRIVATE="brank.as/*"
> GOPROXY="https://proxy.golang.org"
> GOMOD="/dev/null"
> GOGCCFLAGS="-fPIC -m64 -pthread -fmessage-length=0 -fdebug-prefix-map=/tmp/go-build040980257=/tmp/go-build -gno-record-gcc-switches"

This is much better. It even has some surprises - I botched my GOPROXY by forgetting ,direct, for example. And I'm not sure why GOGCCFLAGS is different. GONOPROXY and GONOSUMDB are also a bit redundant as they simply follow GOPRIVATE here, but that's a minor thing.

It would be useful to be able to print a summary like the above, perhaps with go env -diff or go env -changed. Note that the idea is to show what's different from the defaults on that machine, so usually GOARCH and GOOS wouldn't be included.

@mvdan mvdan added NeedsDecision Feedback is required from experts, contributors, and/or the community before a change can be made. GoCommand cmd/go labels Sep 10, 2019
@mvdan
Copy link
Member Author

mvdan commented Jan 27, 2020

This got no attention, so I'm rewriting it into a proposal.

@mvdan mvdan changed the title cmd/go: add a way to query for non-defaults in the env proposal: cmd/go: add a way to query for non-defaults in the env Jan 27, 2020
@gopherbot gopherbot added this to the Proposal milestone Jan 27, 2020
@gopherbot gopherbot added Proposal and removed NeedsDecision Feedback is required from experts, contributors, and/or the community before a change can be made. labels Jan 27, 2020
@dr2chase
Copy link
Contributor

I think this is a good idea, both because it helps us and because it might help the user with "bugs" that are really misconfigurations. Would it make sense to simply enhance the existing output with a trailing shell comment for the modified ones, that way it still copy-pastes as environment variable settings? E.g.

$ go env
GO111MODULE="on"                   # instead of ""
GOARCH="amd64"
GOBIN="/home/mvdan/go/bin"
GOCACHE="/home/mvdan/go/cache"
GOENV="/home/mvdan/.config/go/env"
GOEXE=""
GOFLAGS=""
GOHOSTARCH="amd64"
GOHOSTOS="linux"
GONOPROXY="brank.as/*"             # instead of ""
GONOSUMDB="brank.as/*"             # instead of ""
GOPROXY="https://proxy.golang.org" # instead of "https://proxy.golang.org,direct"
(etc)

@mvdan
Copy link
Member Author

mvdan commented Jan 28, 2020

That would work for me too. Though that might be a little too close to shell syntax for people's comfort :)

I also wonder if backwards compatibility would be a problem. For example, export $(go env) could break if we add that output, as we'd go from export FOO=x BAR=y to export FOO=x #instead of "" BAR=y. So we'd need a new "be helpful and show me the non-defaults" flag for humans.

@mvdan
Copy link
Member Author

mvdan commented Sep 16, 2020

We briefly discussed this in today's tools call. @jayconrod mentioned comments, but I'm still a bit worried about those in case anyone is expecting simple export word list syntax instead of the full source shell syntax supporting comments.

Another idea that came out during the call would be extending go env -json. In theory, that would allow us to add extra information without breaking any previous users. Unfortunately, the format of the output is just map[string]string with each key-value being each of the variable names and values, so we can't fit any more information there easily. Unless we go for something a bit funky like adding a second json object at the end of the output, like:

$ go env -json
{
    "FOO": "bar",
    [...]
}
{
    "nonDefaults": ["FOO"]
}

cc @bcmills

@ianlancetaylor ianlancetaylor added this to Incoming in Proposals (old) Jan 13, 2021
@seankhliao
Copy link
Member

what about explaining where each value comes from?

$ go env -src
GO111MODULE="on"                   # default
GOARCH="amd64"                     # environment, default "arm64"
GOBIN="/home/mvdan/go/bin"         # GOENV, default ""
GOCACHE="/home/mvdan/go/cache"     # GOROOT/go.env, default ""
GOENV="/home/mvdan/.config/go/env" # default

@rsc
Copy link
Contributor

rsc commented Dec 4, 2023

I see why this could possibly be useful but it's also a giant pain. I am not sure the benefit is worth the effort.

@rsc
Copy link
Contributor

rsc commented Dec 6, 2023

Does someone want to prototype this to see if there is a simple implementation?

@rsc
Copy link
Contributor

rsc commented Dec 9, 2023

This proposal has been added to the active column of the proposals project
and will now be reviewed at the weekly proposal review meetings.
— rsc for the proposal review group

@rsc
Copy link
Contributor

rsc commented Feb 8, 2024

Placed on hold.
— rsc for the proposal review group

@suntala
Copy link

suntala commented Feb 10, 2024

Does someone want to prototype this to see if there is a simple implementation?

I have been working on one. The proposal has been placed on hold though. Are there any further details that can be shared about that?

@seankhliao
Copy link
Member

We place proposals on hold if we think they may be valuable but want more information before committing to a decision. In this case we'd like to see an implementation and its effect on cmd/go.

qiulaidongfeng added a commit to qiulaidongfeng/go that referenced this issue Feb 11, 2024
DO NOT SUBMIT

Fixes golang#34208

Change-Id: Ia836aae7176aee926fe47b09fad28d265f8e0404
@qiulaidongfeng
Copy link
Member

qiulaidongfeng commented Feb 11, 2024

I wrote a prototype for this proposal that will work well on my computer.
output in my computer:

go env -changed
set GO111MODULE=auto
set GOCACHE=D:\file\go-build
set GOPATH=D:\file\gofile
set GOPROXY=https://goproxy.cn,direct
set GOSUMDB=sum.golang.org
set GOTOOLCHAIN=local
set GOAMD64=v3
set CGO_ENABLED=1

@gopherbot
Copy link

Change https://go.dev/cl/563137 mentions this issue: cmd/go: add -chaged to query for non-defaults in the env

@bcmills
Copy link
Contributor

bcmills commented Feb 14, 2024

@seankhliao, I think a flag for the source of the setting would be useful (like git config --show-origin), but probably shouldn't be the default (it complicates the parsing).

An interesting related question, though, is: what should go env -changed do for positional arguments? (Probably print a blank line for each unchanged variable named on the command line?)

@rsc
Copy link
Contributor

rsc commented Feb 14, 2024

This proposal has been added to the active column of the proposals project
and will now be reviewed at the weekly proposal review meetings.
— rsc for the proposal review group

@rsc rsc removed the Proposal-Hold label Feb 14, 2024
@qiulaidongfeng
Copy link
Member

Probably print a blank line for each unchanged variable named on the command line?

I think go env -changed only prints the non-default values of the environment variables that go env will print.
For example:

$ set GOTOOLCHAIN=local
(Other environment variables are unchanged)
$ go env -changed
$ set GOTOOLCHAIN=local

See #34208 (comment)

It's common to ask users to provide go version and go env when they report a bug. There is one problem with that, though - the output is getting huge, and most of it is generally useless.

If print blank line, Once the go env -changed is use in the future in https://github.com/golang/go/issues/new?assignees=&labels=&projects=&template=00-bug.yml&title=import%2Fpath%3A+issue+title
Empty lines take up space in the browser display, but do not help with handle bugs.

@rsc
Copy link
Contributor

rsc commented Mar 1, 2024

The question is what happens with 'go env -changed GOPATH GOROOT GOMODCACHE' for example.
Without -changed, this would print

% go env GOROOT GOPATH GOMODCACHE
/Users/rsc/go
/Users/rsc
/Users/rsc/pkg/mod
% 

In this case, GOROOT is arguably unchanged, as is GOMODCACHE; but GOPATH is changed. So is this what it prints?

% go env -changed GOROOT GOPATH GOMODCACHE
/Users/rsc
% 

If so, how do you know which of the three values got printed?

This is why @bcmills recommended the blank line, which makes sense to me:

% go env -changed GOROOT GOPATH GOMODCACHE

/Users/rsc

% 

Only programs are likely to use this anyway, and they can read the blank lines well enough.

The usage of go env in issues would be unaffected: we all agree that plain go env -changed does not print blank lines, because it is printing name=value pairs, so there is no ambiguity about what the environment variable names are.

@qiulaidongfeng
Copy link
Member

use https://go.dev/cl/563137

my machine

in windows

$ go env -changed GOPATH GOROOT GOMODCACHE
set GOPATH=D:\file\gofile

in wsl2

$ go env -changed GOPATH GOROOT GOMODCACHE
GOPATH='/mnt/d/file/gofile/'

Because the print format can always be name=value, so I don't think I need to print empty lines.

qiulaidongfeng added a commit to qiulaidongfeng/go that referenced this issue Mar 2, 2024
DO NOT SUBMIT

Fixes golang#34208

Change-Id: Ia836aae7176aee926fe47b09fad28d265f8e0404
@rsc
Copy link
Contributor

rsc commented Mar 13, 2024

OK, this seems fine.

@rsc
Copy link
Contributor

rsc commented Mar 15, 2024

Have all remaining concerns about this proposal been addressed?

The proposal is to add a new flag to go env: ‘go env -changed’.

With no other flags or arguments, ‘go env -changed’ prints all non-default settings
in the form key=value, one per line, same as ‘go env’ does (just fewer lines).

When there are command-line arguments, as in ‘go env -changed X Y Z’, the command behaves exactly like ‘go env -changed’ except it limits the consideration to X, Y, and Z. It still omits default settings. This output has a different form from ‘go env X Y Z’ in that it includes the ‘key=’ prefixes (or ‘set key=’ on Windows).

When using ‘go env -json -changed’, the JSON only includes the changed values. ‘go env -json -changed X Y Z’, similarly limits the consideration to X, Y, and Z.

@rsc
Copy link
Contributor

rsc commented Mar 27, 2024

Based on the discussion above, this proposal seems like a likely accept.
— rsc for the proposal review group

The proposal is to add a new flag to go env: ‘go env -changed’.

With no other flags or arguments, ‘go env -changed’ prints all non-default settings
in the form key=value, one per line, same as ‘go env’ does (just fewer lines).

When there are command-line arguments, as in ‘go env -changed X Y Z’, the command behaves exactly like ‘go env -changed’ except it limits the consideration to X, Y, and Z. It still omits default settings. This output has a different form from ‘go env X Y Z’ in that it includes the ‘key=’ prefixes (or ‘set key=’ on Windows).

When using ‘go env -json -changed’, the JSON only includes the changed values. ‘go env -json -changed X Y Z’, similarly limits the consideration to X, Y, and Z.

@nemith
Copy link
Contributor

nemith commented Mar 28, 2024

I think this would solve the problem I was having here as well where I didn't realize i was setting an empty string which was different than the default.

#62485

@rsc
Copy link
Contributor

rsc commented Apr 4, 2024

No change in consensus, so accepted. 🎉
This issue now tracks the work of implementing the proposal.
— rsc for the proposal review group

The proposal is to add a new flag to go env: ‘go env -changed’.

With no other flags or arguments, ‘go env -changed’ prints all non-default settings
in the form key=value, one per line, same as ‘go env’ does (just fewer lines).

When there are command-line arguments, as in ‘go env -changed X Y Z’, the command behaves exactly like ‘go env -changed’ except it limits the consideration to X, Y, and Z. It still omits default settings. This output has a different form from ‘go env X Y Z’ in that it includes the ‘key=’ prefixes (or ‘set key=’ on Windows).

When using ‘go env -json -changed’, the JSON only includes the changed values. ‘go env -json -changed X Y Z’, similarly limits the consideration to X, Y, and Z.

@rsc rsc changed the title proposal: cmd/go: add a way to query for non-defaults in the env cmd/go: add a way to query for non-defaults in the env Apr 4, 2024
@rsc rsc modified the milestones: Proposal, Backlog Apr 4, 2024
qiulaidongfeng added a commit to qiulaidongfeng/go that referenced this issue Apr 5, 2024
DO NOT SUBMIT

Fixes golang#34208

Change-Id: Ia836aae7176aee926fe47b09fad28d265f8e0404
qiulaidongfeng added a commit to qiulaidongfeng/go that referenced this issue May 3, 2024
DO NOT SUBMIT

Fixes golang#34208

Change-Id: Ia836aae7176aee926fe47b09fad28d265f8e0404
qiulaidongfeng added a commit to qiulaidongfeng/go that referenced this issue May 4, 2024
DO NOT SUBMIT

Fixes golang#34208

Change-Id: Ia836aae7176aee926fe47b09fad28d265f8e0404
@dmitshur dmitshur modified the milestones: Backlog, Go1.23 May 16, 2024
@gopherbot
Copy link

Change https://go.dev/cl/586095 mentions this issue: cmd/go/internal/envcmd: fix showing GODEBUG env value

gopherbot pushed a commit that referenced this issue May 16, 2024
It was accidently dropped in CL 563137

For #34208

Change-Id: I0aea93fda6260f07bc662dea92d1eabb26f82ca0
Reviewed-on: https://go-review.googlesource.com/c/go/+/586095
LUCI-TryBot-Result: Go LUCI <golang-scoped@luci-project-accounts.iam.gserviceaccount.com>
Reviewed-by: Sam Thanawalla <samthanawalla@google.com>
qiulaidongfeng added a commit to qiulaidongfeng/go that referenced this issue May 18, 2024
Fixes golang#67492
For golang#34208

Change-Id: I33a029f307d4a199dc338387cde41d49764f8fd7
@gopherbot
Copy link

Change https://go.dev/cl/586241 mentions this issue: cmd/go: print not-defaults arch-env

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
Status: Accepted
Development

Successfully merging a pull request may close this issue.

10 participants