flag: Support flags after the first non-flag argument

[thinkerbot]

What version of Go are you using (go version)?

$ go version
go version go1.13.5 darwin/amd64

Does this issue reproduce with the latest release?

Yes, confirmed on master.

What operating system and processor architecture are you using (go env)?

go env Output

$ go env
GO111MODULE=""
GOARCH="amd64"
GOBIN=""
GOCACHE="/Users/simonchiang/Library/Caches/go-build"
GOENV="/Users/simonchiang/Library/Application Support/go/env"
GOEXE=""
GOFLAGS=""
GOHOSTARCH="amd64"
GOHOSTOS="darwin"
GONOPROXY=""
GONOSUMDB=""
GOOS="darwin"
GOPATH="/Users/simonchiang/go"
GOPRIVATE=""
GOPROXY="https://proxy.golang.org,direct"
GOROOT="/usr/local/Cellar/go/1.13.5/libexec"
GOSUMDB="sum.golang.org"
GOTMPDIR=""
GOTOOLDIR="/usr/local/Cellar/go/1.13.5/libexec/pkg/tool/darwin_amd64"
GCCGO="gccgo"
AR="ar"
CC="clang"
CXX="clang++"
CGO_ENABLED="1"
GOMOD="/Users/simonchiang/Documents/Repos/go/src/go.mod"
CGO_CFLAGS="-g -O2"
CGO_CPPFLAGS=""
CGO_CXXFLAGS="-g -O2"
CGO_FFLAGS="-g -O2"
CGO_LDFLAGS="-g -O2"
PKG_CONFIG="pkg-config"
GOGCCFLAGS="-fPIC -m64 -pthread -fno-caret-diagnostics -Qunused-arguments -fmessage-length=0 -fdebug-prefix-map=/var/folders/t_/qvvk4bys11n997vvvgcsz0lw0000gn/T/go-build789316836=/tmp/go-build -gno-record-gcc-switches -fno-common"

What did you do?

I tried to write a command line tool using flag where my users could appending flags to modify the command, or to get help. Ex:

thecmd arg -h

What did you expect to see?

I had hoped that this would be trivial to do, as it is a common pattern for option parsing.

What did you see instead?

I saw I needed to do some work to make this happen.

Per the docs and an example, flag.Parse stops at the first non-flag argument. That represents one common pattern of option parsing, which is fine, especially for subcommands. The other common pattern is what I was trying to do - recognize flags interspersed within arguments, all the way to the end.

I worked out the following loop to support this second pattern:

flag.Parse()
args := make([]string, 0)
for i := len(os.Args) - len(flag.Args()) + 1; i < len(os.Args); {
  if i > 1 && os.Args[i-2] == "--" {
    break
  }
  args = append(args, flag.Arg(0))
  flag.CommandLine.Parse(os.Args[i:])
  i += 1 + len(os.Args[i:]) - len(flag.Args())
}
args = append(args, flag.Args()...)

At the end all flags are parsed regardless of location and args is the remaining non-flag arguments (essentially flag.Args()). I looked in the issue tracker and found having this loop might be of use to others as well.

• Test binary ignores arguments after the first unrecognized one #26561 Test binary ignores arguments after the first unrecognized one
• flag: FlagSet.Parse(...) unexpected behavior with non-flag args #26104 flag: FlagSet.Parse(...) unexpected behavior with non-flag args

I propose adding that loop as a ParseToEnd function to compliment to the current Parse behavior, so that both common patterns will be readily available.

[cespare]

That would be a backward-incompatible change. Edit: I misunderstood the proposal.

[gopherbot]

Change https://golang.org/cl/216378 mentions this issue: flag: add ParseToEnd

[thinkerbot]

@ceseo I am proposing adding a new function ParseToEnd. The existing function Parse is unchanged.

[ianlancetaylor]

Dup of #4513 and #24107.

The standard library's flag package presents a particular approach to command line flags. Other approaches are possible, and https://pkg.go.dev/search?q=flag turns up quite a few packages.

[thinkerbot]

I sorta expected this to be rejected... I submit that the number of flag packages and the number of times this has come up imply that while the flag design is intentional, it misses a use case that should be supported. But you gotta draw a line somewhere.

The loop in the description above serves as an ok workaround if someone needs this in the future.

[thinkerbot]

@ianlancetaylor quick followup -- noting like you said that there are different approaches... would you have any interest in an additional example for the documentation, to demonstrate:

• how to parse flags to the end (the loop above, basically)
• how to change the formatting of the help (use VisitAll)

Just seeing if there is a way to improve the usability of flag consistent with the "A little copying is better than a little dependency" and "Documentation is for users" proverbs.

[ianlancetaylor]

Maybe? Changing the formatting of the help message could be a useful example. I'm less sure about parsing flags after arguments. But, maybe?