cmd/gofmt: change -d to exit 1 if diffs exist

[mvdan]

First, I know that this has been asked many times before (#24230 #24427 #38551), and I was even on the other side of this argument for a long time.

The argument usually goes that, if you want to know if any files aren't gofmt-ed, you check that gofmt -l is empty, for example:

test -z $(gofmt -l .)

And this works, if one can assume a POSIX-like shell.

However, in light of #42119, I think we should make "verify that many files comply with gofmt" a trivial command, without extra machinery like checking for non-empty output.

In that issue, we suggest a number of commands that we'd encourage people to run on CI, from go test -race ./... to go mod verify and go vet ./.... You'll note that none of the commands require being wrapped with shell or extra commands to do their job, with two exceptions:

• go mod tidy, where one needs to check if files like go.mod and go.sum were modified on disk. cmd/go: add mod tidy -diff #27005 will fix that.
• gofmt, which needs testing for empty output, hence this issue.

Once go mod tidy -check is implemented, gofmt will stand alone in that list as being the only command that most people should run on their CI, but which is non-portable or just not easy to use directly.

--

So, as others have proposed before: I suggest a very simple fix, which is to make gofmt -d exit with a status code of 1 if any of the input files have a non-empty diff. Then, CI could simply run gofmt -d . and get a good result: CI would fail if any files don't comply, and the diff will show exactly which ones and why.

It's also arguably more consistent to behave this way, as the diff tool usually behaves this way too. For example, from GNU diffutils 3.7:

Exit status is 0 if inputs are the same, 1 if different, 2 if trouble.

If we think this change is too invasive, or could break too many users, we could also consider a separate flag like -exitcode, simialr to git diff --exit-code. It's really not a big deal if the command run on CI gets a bit longer.

I also don't think a solution should come from go fmt. I think gofmt is simply superior for use in CI; it allows formatting files which are not part of Go packages, formatting many modules at once, and printing diffs.

It's also worth noting I plan to expose this feature in gofumpt: mvdan/gofumpt#114

cc @griesemer
cc @bcmills @jayconrod @stevetraut for "CI best practices"
cc @ryanm101 @stevenmatthewt @belm0 as previous posters

[seebs]

I could live with an alternative of a different flag to set the exit code, to avoid breaking existing things. i am sure there's someone out there who has gofmt in a script with set -e. But yes, please make it possible to use gofmt as a checker in a sane way.

[griesemer]

@griesemer for notifications on this issue.

[bcmills]

If we guard the exit-status change with a flag, It would be nice if we could make the flag's syntax and semantics match whatever flag we add for #27005.

[bcmills]

If we think this change is too invasive, or could break too many users,

I think changing the exit status of gofmt -d probably would break some fraction of users, especially for Playground-like servers, and non-gopls integrations, and perhaps CI systems. I have a mild preference for guarding it with a flag.

[ryanm101]

You could add the flag for now and put in a deprecation warning that in X releases the flag will be inverted and the default will be exit codes.

In terms of breakage of CI systems, that is the behaviour I want, If i am enforcing gofmt on my code and you don't do it I want the CI to fall over. IMHO not doing that is the same as a compiler exiting 0 when the code failed to compile.

(i do realise my feelings may be a tad towards the hardline side of this :) ) in all honesty though I'll take whichever implementation we can get. :)

[mvdan]

I don't think we can ever make the entire tool's default be using exit codes for "was the formating changed". It's entirely reasonable to use some-generator | gofmt to easily generate gofmt-formatted Go code, for example, and all those use cases where one wants to format Go code instead of checking for valid format can't be broken.

I'm also not sure that adding a deprecation or transition period will really help. Breaking a user's script in six months or in two years isn't much different, and I imagine most users will not even see the deprecation notices and remember all the scripts they need to update.

I think changing the exit status of gofmt -d probably would break some fraction of users, especially for Playground-like servers, and non-gopls integrations, and perhaps CI systems. I have a mild preference for guarding it with a flag.

At least from what I've seen, nearly all uses of gofmt -d fall under "check that Go code is correctly formatted", so I think nearly all users would want the exit status behavior. We could have them move to something like gofmt -d -check, but it would be a bit unfortunate to nearly always use both options at the same time.

My intuition is that most scripts using gofmt -d won't break, since what I've most commonly seen is in the form of:

DIFF="$(gofmt -d .)"
if [[ -n $DIFF ]]; then
    echo "$DIFF"
    echo "please run gofmt"
    exit 1
fi

In that kind of scenario, the exit code is by default discarded via the subshell. That's how I've seen many people use gofmt -l, too. The new form, actually using the exit code, could be as follows, or in shorter form via gofmt -d . || exit 1:

if ! gofmt -d .; then
    echo "please run gofmt"
    exit 1
fi

And for the cases where a script would break, I imagine it should be fairly easy to get the old behavior back, such as replacing gofmt -d . with gofmt -d . || true.

All that said, I don't have a strong opinion on new flag versus changing -d. I slightly lean towards not adding a flag, and seeing how many users report breakages during the beta and RC cycle.

[bcmills]

In that kind of scenario, the exit code is by default discarded via the subshell.

By default, sure — but many shell style guides require explicit checks, and we probably don't want to break scripts that conform to that style either.

[bcmills]

I slightly lean towards not adding a flag, and seeing how many users report breakages during the beta and RC cycle.

FWIW, in my experience changes outside the runtime, compiler, and standard library receive comparatively little testing during the beta and RC. (I would expect most breakage to be reported after the release.)

[cespare]

I prefer an explicit flag that's orthogonal to -d for a different reason than compatibility: in CI we don't use -d, we use -l. We don't care about the diffs; only reporting which files aren't formatted.

[ryanm101]

I would agree with @cespare diffs are only really used for when a human is looking, in CI I would want ideally two returns 1) the file that is failing and 2) the line number of the fail would be nice in the case of a few lines but once you get about 10 lines or so this matters less as at that point it's likely the whole files that is wrong.

Still currently any implementation is preferable to none, so additional flag or not I'd just be happy for a non zero exit code.

[mvdan]

I personally don't mind too much whether this happens with -d, or via a separate flag, but I definitely think we should do this.

I agree that we want to be consistent with whatever #27005 ends up doing. At the same time, I don't particularly feel like waiting a few more years for a flag that's trivial to implement and very much needed for CI :)

If #27005 is expected to land on a flag design soon, like -check, then we can follow suit. If not, I think we should go ahead with something that makes sense for gofmt, which could be -check or something akin to git diff --exit-code, such as -exit or -exitcode.

Either way, I want to help us reach a decision for 1.21. If we're sure we want a decision on #27005 first, then I'll try to help push that forward.

[mvdan]

Catching up on #27005, it seems like it's landing on -diff which will print a diff, and exit with a non-zero code if the diff isn't empty. There wouldn't be any other flag or behavior change.

So, if we wanted to be consistent, we could make -d use exit codes, like I originally proposed. We could make exit codes useful for any mode that prints changes that would be made without actually making them. This would be gofmt -d and gofmt -l, but not gofmt (which writes to stdout) nor gofmt -w (which writes to disk). Presumably, this would satisfy #46289 (comment) as well, even if it's a bit of magic.

If we prefer an explicit flag, like git diff --exit-code, which can be used in combination with any gofmt mode - that's also fine by me. It might also be the right idea in terms of not breaking existing users of the -d and -l flags. In that scenario, I'd attempt to copy git diff via -exitcode.

[mvdan]

@rogpeppe suggests -check, and @findleyr seems to like it.

[mvdan]

Retitled to reflect the above, and to better line up with other issues like #27005.

[mvdan]

To be clear, here's what I would propose: add a -check flag which makes gofmt exit with a non-zero status code if any file being formatted results in different bytes. Otherwise the tool would behave exactly the same as before. I see two common use cases:

• CI users wanting a check that prints filenames and fails on badly formatted files: gofmt -l -check .
• CI users wanting a check that prints verbose diffs and fails on badly formatted files: gofmt -d -check .

The flag could be used without either of those files too, e.g. gofmt -check with stdin/stdout, or gofmt -w -check . which would fail if any files were modified. I'm not sure those combinations will be popular or even useful, but I'm not sure that it is worth intentionally restricting -check so that it requires either the -l or -d flags to be given.

For the sake of symmetry, and for those wanting to format packages like ./... rather than all files, I suggest that we also add go fmt -check. It would swap its default mode of calling gofmt -l -w with gofmt -l -check - that is, it would print filenames and fail if any files are badly formatted rather than modifying them in-place.

[mvdan]

#27005 is giving up a -check flag in favor of -diff, so the consistency point in #46289 (comment) unfortunately no longer applies.

More confusingly, I think the proposal there is for go mod tidy -diff to fail when the diff is non-empty, whereas gofmt -d does not behave like that - and we likely cannot change it at this point.

I still lean towards adding a -check flag as described above, given the existing diff flag and backwards compatibility concerns. It won't be consistent with go mod tidy, but gofmt is already pretty different from cmd/go commands in terms of flags and arguments today.

[rsc]

@mvdan says that we likely cannot change gofmt -d to exit non-zero when diffs exist.
We talked about this in proposal review and don't immediately see why we can't change the exit status.
Tool details like these are not as strictly regulated for compatibility as the main packages,
because there is no way that changing the exit status of gofmt -d breaks a Go program built with go build because of some detail in a dependency they didn't know about.

[rsc]

This proposal has been added to the active column of the proposals project
and will now be reviewed at the weekly proposal review meetings.
— rsc for the proposal review group

[rsc]

Based on the discussion above, this proposal seems like a likely accept.
— rsc for the proposal review group

The proposal is to change gofmt -d (which already exists) to exit 1 if it prints diffs. Gofmt should exit 2 for any other errors.

[mvdan]

Changing gofmt -d in-place seems fine to me, and it also makes the flag consistent with the new go mod tidy -diff, which also fails if the diff is non-empty.

There are potentially some cases where a script may break or change behavior, particularly since CI scripts like GitHub Actions tend to stop at the first failing command via e.g. set -e on Bash, but I can't say how often that might happen.

I skimmed a few results from https://github.com/search?q=%22gofmt+-d%22+path%3A**%2F.github%2Fworkflows%2F*.yml&type=code and I don't immediately see any that worries me. If anything, all the commands like diff <(echo -n) <(gofmt -d) should now just become gofmt -d, thankfully.

[rsc]

No change in consensus, so accepted. 🎉
This issue now tracks the work of implementing the proposal.
— rsc for the proposal review group

The proposal is to change gofmt -d (which already exists) to exit 1 if it prints diffs. Gofmt should exit 2 for any other errors.

[YogiLiu]

👏This is good news for users of pre-commit.

Can x/tools/cmd/goimports also implement the same behavior?