Argument specification syntax proposal

Summary

Because the issues (#447 and #478) to implement this had been open for 4.5 years a simpler argparse command was implemented for the fish 2.7 release. It is unlikely this will ever be implemented.

We add a notion of a command signature, with a syntax based on docopt. The signature is a piece of metadata that provides fish with lots of information about how the command is invoked.

To give you a taste:

complete --signature '
Usage: cowsay [options] [<message>...]
Options: -e, --eye <eyestr>         Specify eye string
         -f, --cowfile <cowname>    Specify cow file name
         -help                      Display help and exit
         -l, --list                 List all cowfiles
         -n                         No word wrapping
         -T                         Specify tongue string
         -W, --width <colwidth>     Column width

Arguments: <cowname> (cowsay -l|tail -n +2|tr \  \n)

Description:
Cowsay generates an ASCII picture of a cow saying something provided by the user. If
run with no arguments, it accepts standard input, word-wraps the message given at
about 40 columns, and prints the cow saying the given message on standard output.
'

However signatures may also be very lightweight:

complete --signature 'uniq <input_file> <output_file>'

or:

complete --signature 'sort -r,--reverse'

Goals

The signature plays multiple roles:

1. It becomes the preferred way to write tab completions. complete flags like -l, -s, and -o will be deprecated in preference to signatures, which are more powerful.
2. It is used by syntax highlighting. Since it provides more information than completions, the syntax highlighting becomes more precise.
3. It parses $argv in fish functions (and builtins). fish functions can be made simpler and will no longer  have to shell out to getopt.
4. It replaces the command's usage spec. Files like doc_src/complete.txt can go away. Help text will be derived directly from the signature, either at build time or runtime.

Usage

The commands complete and function both gain a new option --signature (short option -g), whose argument is a signature. The signature is an ordinary string argument, and so is subject to the usual quoting and escaping rules.

The unescaped signature must be valid fish-flavored docopt, which is based on docopt. The differences are described here.

When specifying a signature, you can omit the command name to complete and it will be read from the signature. If you do specify a command name, it takes precedence over the one in the signature.

Syntax

Note: users and completion authors are not expected to become experts in this stuff. Instead the man page for complete and function will have a few samples, that users can copy and modify following a (hopefully obvious) syntax. As this is fish, syntax errors are reported with outstanding diagnostic messages.

A signature is processed line by line. Each line is a usage spec, an option spec, or a variable spec. Lines are disambiguated according to their first (non-whitespace) character: a dash means options, a < means variable command, and anything else means usage:

cowsay [<message>] [options]                # usage spec
-f, --cowfile <cowname>                     # option spec
<cowname> (cowsay -l|tail -n +2|tr \  \n)   # variable command spec

Indentation may be used for line continuation. For example, it is possible to break a usage spec across two lines by indenting the second line more than the first.

The usage specs show different patterns for invoking the command. docopt.org describes these in more detail, subject to fish-flavored differences.

The literal "[options]" in a usage spec is a proxy for the Options specification. Descriptions may be given for individual options, and these will appear as the descriptions in tab completions.

Variable specs are unique to fish-flavored docopt, and represents the argument list for variables. Within the section, specify the variable (including brackets), followed by an argument list which may contain subshells, identical to complete -a. This is only used during tab completion (the section is omitted when printing help.

There are some conveniences:

1. You can omit the usage spec if it is trivial. It will default to spec command [options]. For example, complete -c sort --signature '-r,--reverse' is shorthand for:

   complete -c sort --signature ' sort [options] -r, --reverse'

2. If "[options]" does not appear in any usage spec, it is as if it appears in all specs. Thus:

   complete --signature ' sort -r, --reverse'

is equivalent to:

complete --signature '
  sort <file> [options]
  -r, --reverse'

This allows you to avoid typing out [options] in the common case.

Inferred Conditions

A variable may have inferred conditions based on a suffix of its name:

- *file -> only files
- *dir or *directory -> only directories
- *path -> any valid path (the destination does not have to exist, but intermediate directories do)
- *command or *cmd -> only commands (e.g. for sudo)

For example: Usage: rmdir <directory> ... or Usage: cat [<file> ...] will only tab-complete to directories and files, respectively. Syntax highlighting will also check these conditions. This makes it very simple to achieve common tab completion behavior.

Argument parsing

When a function has a signature, $argv will be parsed according to the signature and variables automatically populated. This is based on xiaq's design. We populate variables for the options, commands, and variables in the signature:

• Options get canonicalized to the corresponding long name, if it exists. Dashes are replaced with underscores, and then opt_ is prepended, e.g. --foo => $opt_foo. The value is set to 1.
• A subcommand like checkout corresponds to the variable $cmd_checkout.
• A variable has its name used directly. (This implies that you cannot use certain variable names like <status>, but variable names do not affect the invocation syntax so this is not a burden).

Examples for this signature:

cowsay [options] [<message>...]
-e, --eye <eyestr>          Specify eye string
-f, --cow-file <cowname>    Specify cow file name

run with:

cowsay -e XX --cow-file dragon hello world [<message>...]

will result in the following function-scoped variables and values:

• $opt_eye = 1
• $opt_cow_file = 1
• $eyestr = XX
• $cowname = dragon
• $message = hello world <- list of length 2

Of course $argv will also be set to the full list of arguments.

Prefer set -q within the function to check for the presence of flags.

Authoritative vs unauthoritative

Function signatures respect the -A/--authoritative and -u/--unauthoritative options to complete, and we will add these options to function too.

When a function is invoked with an authoritative signature, its arguments are matched against the usage specifications. If the match is imperfect (unrecognized options, or missing/excessive arguments), the function does not execute, and instead help (derived from the signature) is printed. In this way, a function is saved from having to do validation.

If the function's signature is unauthoritative, then fish makes a best-effort to choose a pattern and populate variables based on that, and the function is executed.

By default, function signatures are authoritative. This behavior only applies to functions, not external commands.

Multiple signatures

A command may have multiple signatures. This is convenient for breaking up a large command like git into multiple signatures (e.g. one for each subcommand).

The behavior is as if the multiple signatures are merged into a single one. In case of ambiguity, the most-recently specified signature takes precedence.