Add support for command-line flags

[atc0005]

Not yet sure which package, but this tool will need to be heavily command-line driven.

Standard library

• https://golang.org/pkg/flag/

  • https://gobyexample.com/command-line-flags

• https://gobyexample.com/command-line-arguments

Third-party

• https://github.com/urfave/cli
• https://github.com/peterbourgon/ff
• https://github.com/integrii/flaggy

Implementation / Examples

• https://github.com/tsenart/vegeta/blob/master/main.go
  • looks like they're using the stdlib flag package, quite a bit of validation wrapping

[atc0005]

flags-first package by Peter Bourgon supports all of these configuration file formats and command-line and environment variables.

https://github.com/peterbourgon/ff

[atc0005]

see also atc0005/golang-oracle#12

[atc0005]

Simple flaggy example: https://github.com/integrii/flaggy/blob/master/examples/simple/main.go

package main

import "github.com/integrii/flaggy"

func main() {
	// Declare variables and their defaults
	var stringFlag = "defaultValue"

	// Add a flag
	flaggy.String(&stringFlag, "f", "flag", "A test string flag")

	// Parse the flag
	flaggy.Parse()

	// Use the flag
	print(stringFlag)
}

[atc0005]

Note to self:

Whatever I opt to use, it should make it very clear to the user what the defaults are, particularly in Help or Error output.

Relying on zero/default values for the config struct is not a good idea, even during development work.

[atc0005]

Note: Not a big fan of the flags-first help output:

$ go run *.go -recurse -bob
flag provided but not defined: -bob
Usage of elbow:
  -config string
        config file (optional)
  -ext string
        file extension to match against to limit search
  -keep int
        keep specified number of matching files
  -keep-oldest
        keep oldest files instead of newer
  -path string
        path to process
  -pattern string
        file pattern to match against
  -recurse
        recurse into subdirectories
  -remove
        remove matched files
exit status 2

The defaults are specified in the flag set, but not stressed in this output.

[atc0005]

Going to test flaggy next. The Help output explicitly notes what the default values are:

https://github.com/integrii/flaggy#example-help-output

testCommand - Description goes here.  Get more information at http://flaggy.
This is a prepend for help

  Usage:
    testCommand [subcommandA|subcommandB|subcommandC] [testPositionalA] [testPositionalB]

  Positional Variables:
    testPositionalA - (Required) Test positional A does some things with a positional value. (default: defaultValue)
    testPositionalB - Test positional B does some less than serious things with a positional value.

  Subcommands:
    subcommandA (a) - Subcommand A is a command that does stuff
    subcommandB (b) - Subcommand B is a command that does other stuff
    subcommandC (c) - Subcommand C is a command that does SERIOUS stuff

  Flags:
       --version  Displays the program version string.
    -h --help  Displays help with available flag, subcommand, and positional value parameters.
    -s --stringFlag  This is a test string flag that does some stringy string stuff.
    -i --intFlg  This is a test int flag that does some interesting int stuff. (default: 5)
    -b --boolFlag  This is a test bool flag that does some booly bool stuff. (default: true)
    -d --durationFlag  This is a test duration flag that does some untimely stuff. (default: 1h23s)

This is an append for help
This is a help add-on message

[atc0005]

I don't see how to specify the default values programatically when using integrii/flaggy. Presumably their example was crafted from hard-coded strings.

[atc0005]

On a different note, this urfave/cli example displays the default value for a flag:

package main

import (
  "fmt"
  "log"
  "os"

  "github.com/urfave/cli"
)

func main() {
  app := cli.NewApp()

  app.Flags = []cli.Flag {
    cli.StringFlag{
      Name: "lang",
      Value: "english",
      Usage: "language for the greeting",
    },
  }

  app.Action = func(c *cli.Context) error {
    name := "Nefertiti"
    if c.NArg() > 0 {
      name = c.Args().Get(0)
    }
    if c.String("lang") == "spanish" {
      fmt.Println("Hola", name)
    } else {
      fmt.Println("Hello", name)
    }
    return nil
  }

  err := app.Run(os.Args)
  if err != nil {
    log.Fatal(err)
  }
}

Output:

$ go run main.go --help
NAME:
   main.exe - A new cli application

USAGE:
   main.exe [global options] command [command options] [arguments...]

VERSION:
   0.0.0

COMMANDS:
   help, h  Shows a list of commands or help for one command

GLOBAL OPTIONS:
   --lang value   language for the greeting (default: "english")
   --help, -h     show help
   --version, -v  print the version

[atc0005]

I don't see how to specify the default values programatically when using integrii/flaggy. Presumably their example was crafted from hard-coded strings.

I opened issue 46 in the integrii/flaggy repo and provided a sample program that replicates my findings.

[atc0005]

I opened issue 46 in the integrii/flaggy repo and provided a sample program that replicates my findings.

The developer confirmed the bug report and is looking into it.

[atc0005]

The developer confirmed the bug report and is looking into it.

The developer reports that the issue has been fixed in version 1.2.2. Initially the release was without a semantic 'v' prefix, but he went back and adjusted all tags to include the leading character to make them semver compliant.

Regarding the bug I ran into, the developer noted that string and integer values were intended to have their default values displayed in the help text, but not boolean flags. In their own words:

When building flaggy, I chose to hide the default value for bool flags that were set to their default value of false.

If you set your bool flag to true, you should see it appear in the help as you're expecting. I think the logic there was:

Bool flags are either supplied or they aren't. If they aren't, its false. Matter of fact, setting a bool to false, requires you to explicitly call out the false like this: -bo=false.

[atc0005]

Note: flaggy supports specifying the same config option multiple times, presumably to populate a slice:

integrii/flaggy#3 (comment)

For example, you could do -l label -l label or -v -v.

[atc0005]

I started drafting an issue for the flaggy repo, but stopped once I asked the question, "Does it matter?".

Recording the details here in case i decide to submit the issue later.

---

Issue title: Can you require that a specific flag is used?

Body:

I saw this in the README:

 Positional Variables:
    testPositionalA - (Required) Test positional A does some things with a positional value. (default: defaultValue)
    testPositionalB - Test positional B does some less than serious things with a positional value.

and when reviewing the GoDoc page for this package I saw that positional arguments can optionally be required, but didn't spot that feature when configuring flags.

Is there an easy way to determine when a flag has been provided? Would you perhaps define a "default" instance of a configuration type to compare against the configuration type that is modified by flaggy.Parse()?

For example, I've implemented a NewConfig() function to return a configuration struct for use in main() that first defines all defaults. The flaggy.X calls define flags and then (as you know) if they're not used the default settings for the config struct remain as-is.

---

We can do as proposed; if the final custom object doesn't match the default, then we can assume that the flag wasn't passed and do something about it.

[atc0005]

integrii/flaggy#48

I opted to go ahead and submit the question anyway. For now, I'll assume that I will wish to create a default Config object and then compare the two.

[atc0005]

integrii/flaggy#48

I opted to go ahead and submit the question anyway. For now, I'll assume that I will wish to create a default Config object and then compare the two.

Dev's response:

For flags, you can easily just check the value after calling flaggy.Parse(). If the value is set to something other than the default, then the caller supplied it. If it was the default value (set by you or the language), then it was not used.

Looks like I'm on the right path with comparing a default config struct against one that I expect to be modified.

Worth noting:

https://github.com/r3labs/diff#basic-example

That package is pretty useful for making the comparison. It's probably overkill, but it appears useful for prototype work.

[atc0005]

Example output from the current variation of the test file:

$ go run test.go -path="C:\Users"
2019/08/22 13:41:22 User specified command-line options
Changes to default settings: [{Type:update Path:[startpath] From: To:C:\Users}]
2019/08/22 13:41:22 Processing path: C:\Users

Ignore the last line as it is irrelevant to this example.

[atc0005]

Further work re Config object handling, support methods.

I found lots of good examples here:

• https://github.com/aws/aws-sdk-go/blob/10878ad0389c5b3069815112ce888b191c8cd325/awstesting/integration/performance/s3GetObject/config.go#L25

https://github.com/aws/aws-sdk-go/blob/10878ad0389c5b3069815112ce888b191c8cd325/awstesting/integration/performance/s3GetObject/main.go#L25

• https://github.com/aws/aws-sdk-go/blob/10878ad0389c5b3069815112ce888b191c8cd325/aws/config.go#L251

Another example here which just reaffirms that returning a pointer to a Config object is the way to go. Seems that this approach allows for method chaining in order to set multiple configuration options.

• https://github.com/go-sql-driver/mysql/blob/master/dsn.go#L67

More examples of how the aws-sdk project is using flags:

https://github.com/aws/aws-sdk-go/search?q=flag&unscoped_q=flag

[atc0005]

Will probably cleanup the prototype branch I'm working on and merge it in in the very near future. That branch is currently using flaggy for command-line flag handling.

While researching logging, I saw that the https://github.com/spf13/viper package handles environment variables, config files and command-line flags. Using this package could tick-off multiple TODO items for this project.

[atc0005]

integrii/flaggy#48
I opted to go ahead and submit the question anyway. For now, I'll assume that I will wish to create a default Config object and then compare the two.

Dev's response:

For flags, you can easily just check the value after calling flaggy.Parse(). If the value is set to something other than the default, then the caller supplied it. If it was the default value (set by you or the language), then it was not used.

Looks like I'm on the right path with comparing a default config struct against one that I expect to be modified.

Worth noting:

https://github.com/r3labs/diff#basic-example

That package is pretty useful for making the comparison. It's probably overkill, but it appears useful for prototype work.

With validation applied, the earlier call (not yet mentioned here) of reflect.DeepEqual() no longer appears to be needed.

Earlier code block:

if reflect.DeepEqual(*defaultConfig, *config) {
	// DEBUG
	log.Println("User did not provide command-line flags; current configuration matches default settings")

	// KEEP
	flaggy.ShowHelpAndExit("Required command-line options not provided.")
} else {
	// DEBUG
	log.Println("User provided command-line flags, proceeding ...")
}

Current approach:

if !pathExists(config.StartPath) {
    flaggy.ShowHelpAndExit(fmt.Sprintf("Error processing requested path: %q", config.StartPath))
}

This seems to get the point across without having to use what is purported to be a very expensive function call. While it wouldn't matter much here, perhaps it is good to not rely upon it?

[atc0005]

In addition to viper, this also looks promising:

https://github.com/spf13/pflag

[atc0005]

Yet another command-line flag package:

https://github.com/DavidGamba/go-getoptions

Many good features. Here are a couple cherry-picked items:

• Called() method indicates if the option was passed on the command line.
• Support for --long options.
• Multiple ways of managing unknown options:
  • Fail on unknown (default).
  • Warn on unknown.
  • Pass through, allows for subcommands and can be combined with Require Order.

[atc0005]

Very small CLI support package:

https://github.com/jaffee/commandeer

This package builds out the help text directly from struct tags, which is a nice annotation approach (that I've not spent enough time with yet to truly "get").

[atc0005]

Note to self: Initial command-line flag support will be in v0.1, but still keeping this issue assigned to v0.2 as I hope to make further refinements and test other "flag" packages that also provide support for environment variables, config files, etc.

[atc0005]

Just for later reference, here is what the help output looks like when using the flaggy package (v0.1.0 of this project) and when using go-flags.

flaggy help output

Elbow - Prune content matching specific patterns, either in a single directory or recursively through a directory tree.

  Flags:
       --version  Displays the program version string.
    -h --help  Displays help with available flag, subcommand, and positional value parameters.
    -p --path  Path to process
    -fp --pattern  Substring pattern to compare filenames against. Wildcards are not supported.
    -e --extension  Limit search to specified file extension. Specify as needed to match multiple required extensions.
    -k --keep  Keep specified number of matching files (default: 0)
    -r --recurse  Perform recursive search into subdirectories
    -ko --keep-old  Keep oldest files instead of newer
    -rm --remove  Remove matched files

go-flags help output

Usage:
  elbow [OPTIONS]

Application Options:
      --pattern=                                            Substring pattern to compare filenames against. Wildcards are not supported.
      --extension=                                          Limit search to specified file extension. Specify as needed to match multiple required extensions.
      --path=                                               Path to process.
      --recurse                                             Perform recursive search into subdirectories.
      --keep=                                               Keep specified number of matching files.
      --keep-old                                            Keep oldest files instead of newer.
      --remove                                              Remove matched files.
      --log-format=[text|json]                              Log formatter used by logging package. (default: text)
      --log-file=                                           Log file used to hold logged messages.
      --log-level=[panic|fatal|error|warn|info|debug|trace] Maximum log level at which messages will be logged. Log messages below this threshold will be discarded.
                                                            (default: info)
      --use-syslog                                          Log messages to syslog in addition to other ouputs. Not supported on Windows.

Help Options:
  -h, --help                                                Show this help message

go-flags doesn't support two letter "short" flags, so I had to remove short flags entirely to help prevent conflicts. I'm also not entirely done "tuning" the options, but I do like the display.

[atc0005]

Regarding go-flags, I struggled for a bit to find a project that is using it until I noticed that at the bottom of the godoc.org page that it provides a count of the number of projects using the package and a link to view them:

https://godoc.org/github.com/jessevdk/go-flags?importers

I ended up not using the list directly, but lucked out and found this StackOverflow Q/A thread:

https://stackoverflow.com/questions/38709768/understanding-subcommands-with-go-flags

That thread linked to the old repo for this project:

https://github.com/concourse/concourse

They seem to make heavy use of commands, but the general idea is there.

[atc0005]

As of this point I think I like go-flags and it seems stable enough, just not sure about the long-term maintenance efforts for their project (others have raised this as concern in one of the issues).

[atc0005]

https://github.com/alexflint/go-arg

This supports options similar to go-flags and also supports environment variables as a config source.

[atc0005]

go-flags seems to have at least one odd quirk:

$ cd /mnt/t/github/elbow; go build; cp -vf elbow /tmp/; cd /tmp/; ./elbow --path /tmp --extension ".war" --pattern "-masterdev-" --keep 2
'elbow' -> '/tmp/elbow'
expected argument for flag `--pattern', but got option `-masterdev-'

The -masterdev- literal pattern is throwing off the go-flags flag values detection.

Single or double-quotes, it doesn't seem to matter.

[atc0005]

At this point I'm strongly considering locking in the current functionality set:

• logrus for logging
• go-flags for flags
  • specifically requiring specific flags
  • spelling out the choices in help text
  • directly spelling out valid choices

I need to rework the two feature branches (likely prune one), rebase, then merge and tag.

I will likely close this issue, but reopen later if I opt to try another flag package for this project (will probably try a different package for another project).

[atc0005]

Will use the use-go-flags-and-logrus branch to complete the work for this issue.

[atc0005]

Worth noting:

As of this writing, the jessevdk/go-flags package hasn't received any attention since 2018-12-21. See issue 317 in that project for details.

The alexflint/go-arg package offers similar features and also offers environment variable support as well.

I'll stick with jessevdk/go-flags for now, but will likely give alexflint/go-arg a try with the next Go tool I work with.