flag: PrintDefaults panics trying to ascertain whether a flag was set

[alandonovan]

(*flag.FlagSet).PrintDefaults assumes that it is safe to call .String() on the zero value of a flag type, or if that flag type is a pointer, a new instance of the variable. Consequently, a program that uses indirect data structures in its flags will panic when a user sets the -h flag.

$ go version
go version devel +7da1f7addf Thu Nov 8 08:19:48 2018 +0000 linux/amd64
$ cat a.go
package main

import "flag"

func main() {
        f := &myFlag{x: new(int)}
        flag.CommandLine.Var(f, "f", "usage")
        flag.CommandLine.PrintDefaults()
}

type myFlag struct {
        x *int
}

func (f *myFlag) Set(s string) error { return nil }
func (f *myFlag) String() string     { _ = *f.x; return "" }

$ go run a.go
panic: runtime error: invalid memory address or nil pointer dereference
[signal SIGSEGV: segmentation violation code=0x1 addr=0x0 pc=0x491878]

goroutine 1 [running]:
main.(*myFlag).String(0xc00007c030, 0x4ab080, 0xc00007c030)
        /usr/local/google/home/adonovan/got/src/golang.org/x/tools/a.go:16 +0x8
flag.isZeroValue(0xc000086000, 0x0, 0x0, 0x4c49e4)
        /usr/local/google/home/adonovan/goroot/src/flag/flag.go:457 +0xff
flag.(*FlagSet).PrintDefaults.func1(0xc000086000)
        /usr/local/google/home/adonovan/goroot/src/flag/flag.go:520 +0x202
flag.(*FlagSet).VisitAll(0xc00007e120, 0xc000092f28)
        /usr/local/google/home/adonovan/goroot/src/flag/flag.go:387 +0x61
flag.(*FlagSet).PrintDefaults(0xc00007e120)
        /usr/local/google/home/adonovan/goroot/src/flag/flag.go:503 +0x4e
main.main()
        /usr/local/google/home/adonovan/got/src/golang.org/x/tools/a.go:8 +0xbc
exit status 2

I don't think the flag package should be making assumptions about the concrete representation of the flag. That's the point of the flag.Value interface.

[ghost]

Seems like this is covered in #24439.

Note that this functionality is documented:

The flag package may call the String method with a zero-valued receiver, such as a nil pointer.

[alandonovan]

Yes, the isses are substantially similar. The documentation is incorrect though: it should say that flag may call .String() on a zero value of the flag type, or if the flag type is a pointer, on a pointer to a new variable. Not a precondition that exactly rolls off the tongue; frankly I'm surprised this wasn't considered a breaking semantic change to a published interface.

[robpike]

What is "this" that was a breaking change?

[alandonovan]

I meant this: https://go-review.googlesource.com/c/go/+/23581/3/src/flag/flag.go#382.

[robpike]

Happy to see that rolled back. I didn't like it at all at first, and only a little bit later on.

[jimmyfrasche]

The simplest thing would be to print DefValue whenever it was not the empty string. Then isZeroValue can just be deleted.

[bcmills]

@jimmyfrasche, or we could record whether the default value is the zero value of the type when we first set DefValue, rather than trying to infer it after-the-fact.

(We could even do that without changing the definition of *flag.Flag, by either storing it in a parallel map or wrapping the Flag in a larger, unexported struct.)

[jimmyfrasche]

@bcmills that would work reliably with all flags satisfying the optional Get() interface{} method.

For flag.Value without a Get method, it would have to assume that the Value is the actual value of the flag. But it could also be a nontrivial container for the value—like some nonzero parsing state but a zero default value.

It would be much superior to the status quo in that it would never explode. It would probably do the right thing most of the time. A nontrivial Value could implement Get() to provide the desired behavior, but that's an awkward enough subtlety to justify documentation, imo.

[robpike]

Why don't we just roll back the change?

[ianlancetaylor]

The change in question (https://golang.org/cl/23581) fixes #15904. Of course we can decide to not fix that, though I'll note that it was reported more than once.

[jimmyfrasche]

Checking against a whitelist of zero values is simpler to maintain, but it has false positives and negatives for user-defined flags. In addition to not recognizing "0s" there was #23543 where a string flag with "0" was being falsely recognized as a zero value.

Only checking against the empty string is even simpler. It's also uniform in that it works the same for any flag, regardless of how or where it is defined. It checks against the shell's concept of a zero value rather than Go's, so it may be of greater value to an end user unaware of Go's concept of zero value. Debatable.

Using reflect to see if it's the zero value is uniform and extensible but less simple to do right, empirically. Perhaps @bcmills' version would work if it were simplified to only run if there's a Get method and otherwise just check against the empty string.

Another option would be yet another optional method on Value like ShowDefault() bool but I don't think anyone wants to go down that road.

[robpike]

@ianlancetaylor But the fix is problematic. We can roll back this fix and handle #15904 another way, rather than continuing to hack and redesign and complicate a fundamentally simple package.

[ianlancetaylor]

If someone has a different fix that is certainly fine with me.

[jimmyfrasche]

I did a sketch of my simplification of @bcmills' check in FlagSet.Var: https://play.golang.org/p/Jsl2nSaJN8U

I'm guessing it would be roughly the same LOC as the current fix but it would be more reliable and less surprising.

[pam4]

Issue #24439 already cover the documentation being incorrect, see here:

Here's my proposed fix:
// The flag package may call the String method with a zero-valued receiver,
// or with a pointer pointing to a zero value for the base type (in case of
// a pointer receiver).

This is the same as @alandonovan said above, just from the String method perspective.
Yes, it doesn't roll off the tongue either, but unless a code change is coming soon, I think fixing the docs would be nice, especially because it is contradicted by the example (missing check for nil pointer in String method).

The mentioned breaking change broke one of my flag.Value implementations too, at that time, causing panic.
My use case was to use boolean flags to select one of many exclusive modes, so that the last one takes effect.
E.g. -slow/-normal/-fast (to the same effect of something like -mode=slow/-mode=normal/-mode=fast).

type myFlag struct {
    name string  // either "slow", "normal" or "fast"
    ptr  *string // shared pointer to the selected mode
}

func (f myFlag) String() string {
    // return strconv.FormatBool(*f.ptr == f.name) // panic
    return strconv.FormatBool(f.ptr != nil && *f.ptr == f.name) // fixed
}

I made a zero-valued myFlag (which was not supposed to happen) produce "false" to match the normally occurring falses. If I had returned something else instead, say "invalid", it wouldn't have worked.
In fact, issue #24439 is also proposing to clarify such subtlety in the docs.

I think the fix proposed here is simpler and easier to document. Users would have to make adjustments to their code again to avoid printing redundant defaults, but that's a minor issue compared to the panic (in my case I would have to add the Get method).

[gopherbot]

Change https://go.dev/cl/396079 mentions this issue: flag: use ZeroValue method to find zero values for any Value

[adg]

This bit me recently. In a closed-source project I have implemented a dynamic flag type that can be set either on the command-line (through package flag) or by changing the contents of a file. The flag.Value interface made it easy to use my dynamic flag as a regular flag.

However, my concrete type may represent many different types of flags (strings, ints, other more complex types), so the zero-value reflection trick employed by isZeroValue always generates a useless struct that doesn't know about the type of data it's supposed to hold.

I was very surprised to see my programs panic when printing help output, and we didn't even notice this was happening until about a week later. I think the approach, while clever, simply doesn't work for some concrete implementations of flag.Value and so we should do something else.

My proposed change is to test for a ZeroValue() string method on each flag.Value and use that to check whether the flag's current value matches its zero value. This should prevent future accidental panics, and also makes it possible in my case to print a better usage message.

One question is whether the zeroValuer interface should be exported or not. In my change it is not exported, but people could come to rely on it and so it might make sense to explicitly export and document it.

[ianlancetaylor]

Hi @adg. Looking for a Zerovalue method is technically an ABI change and should go through the proposal process.

[gopherbot]

Change https://go.dev/cl/396354 mentions this issue: flag: do not suppress default values in usage output for external flags

[adg]

Thanks for the quick response @iant.

That being the case, I have sent a change that takes the more conservative first step of simply using the interface for the built-in flags only. This addresses the main issues raised in this thread, with the minor downside that some redundant (default ...) clauses will be printed in the usage output.

[ianlancetaylor]

I wonder if isZeroValue should just recover a panic from the call of the String method.

[robpike]

@iant Yes. That's what the fmt package does, for example.

[ianlancetaylor]

Sent https://go.dev/cl/396517.

[gopherbot]

Change https://go.dev/cl/396517 mentions this issue: flag: recover and ignore panic when calling String on zero value

[gopherbot]

Change https://go.dev/cl/396574 mentions this issue: flag: recover panic when calling String on zero value in PrintDefaults