proposal: flag: add Strings, a default Value

[carnott-snap]

proposal

While there are certainly many custom flag.Value implementations that are required, most of the time, all I want is a []string. As such, I suggest we add a flag.Strings symbol:

package flag

var _ Getter = Strings{}

type Strings []string

func (s Strings) Get() interface{} { return s }
func (s *Strings) Set(v string) error { *s = append(*s, v); return nil }
func (Strings) String() string { return "" }

costs

It is another symbol on the flag package, so it could get lost or be confusing, however I think this will serve to improve customer code considering how much this is implemented, like sort.StringSlice. My biggest concern is if everybody needs a different implementation.

alternatives

If, as @cespare calls out, we are concerned about this custom type proliferating, we can instead expose a constructor: (see below)

func Strings(p *[]string) flag.Value

Unfortunately, this flags.Strings does not work the same as flag.String and simply mirroring that interface seems better:

func Strings(name string, value string, usage string) *[]string
func StringsVar(p *[]string, name string, value string, usage string)

extensions

Because it is easier, I have assumed that all flag.Strings will have the default of emptiness. If it is preferred, we could instead return the first element or a csv joined form of all elements:

func (s Strings) String() string { if len(s) == 0 { return "" } return s[0] }
// or
func (s Strings) String() string { return strings.Join(s, ",") }

If the use case can be justified, it may be worth adding some state and complexity to Strings, but without a clear litmus, it is hard to know where to stop:

type Strings struct{
	Data  []string
	Check func(string) (string, error)
}

func (s *Strings) Set(v string) error {
	if *s.Check != nil {
		var err error
		if v, err = s.Check(v); err != nil {
			return err
		}
	}
	*s.Data = append(*s.Data, v)
	return nil
}

That being said, there are real flexibility benefits to hiding the implementation in a struct:

type Strings struct{ v []string }

func (s Strings) Slice() []string { return *s.v }
func (s *Strings) Set(v string) error { *s.v = append(*s.v, v); return nil }

[cespare]

As it happens, in 2015 we added exactly this API (same names and everything) to our internal library at my company. It got a lot of use and it works well enough. During those five years, though, we found that there is a downside: this Strings type has a way of leaking out into the code, and types/functions that only care about a []string end up referring to this flag helper type and needing to call s.Slice() and so on.

The reason this happens is because of code like this:

type conf {
	things flagext.Strings
	// ... and lots more
}

func main() {
	var c conf
	flag.Var(&c.things, "thing", "Specify things (may be given multiple times)")
	flag.Parse()

	// ...
}

Of course you can always avoid this by declaring the Strings value separately:

type conf {
	things []string
	// ... and lots more
}

func main() {
	var c conf
	var things flagext.String
	flag.Var(&things, "thing", "Specify things (may be given multiple times)")
	flag.Parse()

	c.things = things

	// ...
}

but in practice, people usually end up doing the former because it's shorter and more direct.

A month ago I changed our API to the following:

// Strings creates a flag.Value for a []string where providing the flag multiple
// times appends to the slice. The pointer p must be non-nil and points to a
// slice containing the default value for the flag. If the user provides the
// flag, it overrides the prepopulated slice.
func Strings(p *[]string) flag.Value

(Essentially, this is taking the flag.VarX approach instead of the flag.X approach.)

This is a smaller API and lets us do what we want more directly, which is to have the flag directly manipulate a []string.

type conf {
	things []string
	// ... and lots more
}

func main() {
	var c conf
	flag.Var(flagext.Strings(&things), "thing", "Specify things (may be given multiple times)")
	flag.Parse()

	// ...
}

[carnott-snap]

While it may not be as intuitive to beginners, types have call site usage that feels like a pseudo function. As such, and I think the utility of the exported type can justify this otherwise arcane syntax: (though it would be nice if you could &flag.Strings(c.things))

var c conf
flag.Var((*flag.Strings)(&c.things), "thing", "Specify things (may be given multiple times)")
flag.Parse()

I am also unclear how your flagext.Strings plays with the style of the library. If we decide the type is not what is needed, I think these two functions would be more intuitive:

package flag

func Strings(name string, value string, usage string) *[]string
func StringsVar(p *[]string, name string, value string, usage string)

[carnott-snap]

@ianlancetaylor: can you add this to the proposals project?

[ianlancetaylor]

@carnott-snap Done.

[rsc]

As I understand it, this issue is asking for a way in the standard flag package to support flags that accumulate string values, like:

-X abc -X def

which would end up with a value []string{"abc", "def"} somehow, like the Go compiler's -I flags.

The way that would fit into the flag package would be not to define a new type but to define func Strings and func StringsVar, as in the comments above.

The three questions then are:

1. Is this common enough to add to package flag instead of having people define their own?
2. Is multiple flag invocations the right syntax, or comma-separated, or ...?
3. If we add this flag, what else will we need to add? Ints?

[carnott-snap]

1. I have done this enough times to open the issue, so imo, yes. it may be possible to search for usages of flag.Var, but I have not.
2. I tend to prefer multiple invocations over comma-separated or anything else, but I have seen comma-separation elsewhere. if possible, we could support both, but we should be pragmatic about not allowing every possibility, and I think multiple invocation is the most intuitive.
3. Strings seems to have the most utility, but I agree Ints and Float64s (based on other packages like sort in the stdlib) would probably be good, if not required. we could wait for generics or use []interface{} for func Repeated, but I am not advocating for either.

[earthboundkid]

I literally have a Github package with flagext.Strings, but I am -0 on this feature request.

First of all, my package gets around the "leaky type" problem by having another helper flagext.StringsVar(fl *flag.FlagSet, ss *[]string, name, usage string). So, I don't really worry about that issue.

Second, the default value is hard to make work with current flag package conventions.

Third, flag.Func, which I proposed in #39557 and got merged for Go 1.16 (still not shipped as of this comment), makes writing your own version of flag.Strings without a helper type much simpler:

mystrings := []string{"a", "b"}
flag.Func("s", "some help", func(s string) error {
   mystrings = append(mystrings, s)
   return nil
})

Since the semantics of flag.Strings are a little hard to specify (e.g., should it use comma splitting or multiple invocation and should it copy default flags or append in place), I think it's best left to package users, like flag.File (see my rejected proposal in #23284).

[Merovius]

FWIW, rather than a Strings type, I'd rather have a Repeated(v ...Var) or something like that (with generics, I'd probably think func (type V Var) Repeated(v *[]V) Var). I don't think there is anything necessarily special about string-kind flags being repeated.

(Also, I came here because I was confused whether this was about repeated/comma-separated strings or something like an enum-field)

[rsc]

The new flag.Func does work well for this, as @carlmjohnson points out. That's how we implement -I today in the compiler: we have an addImportDir function that gets called for each argument. And for things like -Dname=value it can be easier to parse the name=value string directly, during the function call, instead of having to get a separate []string and process that into the actual form you need.

Given that we only recently added flag.Func, it seems like we should probably slow down and not start adding even more things. flag.Func makes this use case much more convenient.

[rsc]

Based on the discussion above, this seems like a likely decline.

[icholy]

An alternative implementation would be a reflection based api.

func SliceVar(slice interface{}, name, usage string)

See: https://github.com/icholy/flagslice

[rsc]

No change in consensus, so declined.

[dolmen]

I also have (since 2018) a more generic and more powerful version that handles slices of any types as a flag.Value:

func Slice(sl interface{}, separator string, parse func(string) (interface{}, error)) Value

See: github.com/dolmen-go/flagx.Slice

The flag package is already very extensible thanks to flag.Value. The only problem is that this extensibility power is not enough known. The flag.FlagSet type is already bloated with methods while a library of types implementing flag.Value is much more powerful as you can combine them (for example my Slice above allows to reuse other types implementing flag.Value).