Improve autocomplete for flag arguments

[tiagoposse]

Could autocompletions be made so that when trying to autocomplete a flag, only allowed values for it are shown?

As an example, given the following command:

commands:
- name: deploy

  args:
  - name: name
    required: true
    help: Name of the service
    repeatable: true
    allowed: [ a, b, c ]
  flags:
  - long: --stage
    short: -s
    help: Stage to manage
    arg: stage
    allowed:
      - dev
      - test
      - prod

Autocompletions will be as follows:

./cli deploy <TAB>
--help          a            test
--stage        b           prod
-h                 c
-s                 dev

Triggering autocomplete for the -s flag will result in the same autocompletions being shown:

./cli deploy -s <TAB>
--help          a            test
--stage        b           prod
-h                 c
-s                 dev

Would be clearer if:

• when autocompleting a command, show possible flags and allowed values for the positional argument
• when autocompleting a flag, show possible values for the flag only

Example:
For the command:

./cli deploy <TAB>
--help          a
--stage        b
-h                 c
-s

For the flag

./cli deploy -s <TAB>
dev
test
prod

[DannyBen]

The short answer is no.

Completions are provided by the completely gem.
What you are describing requires a more elaborate completions script that looks back at any given time, and makes a different decision as to the suggested completions.

The only way I could come up with to automatically generate a completions script, was to:

1. Look for the longest possible fixed part of the typed command
2. Present all the non fixed arguments and flags as a list of possible completions.

You can easily see what I mean by running this in your bashly folder:

$ bashly add comp yaml
$ bashly add comp script

The YAML will be:

cli:
- "--help"
- "--version"
- "-h"
- "-v"
- deploy
cli deploy:
- "--help"
- "--stage"
- "-h"
- "-s"
- a
- b
- c
- dev
- prod
- test

and the script will be (partial):

  case "$comp_line" in
    'deploy'*) COMPREPLY=($(compgen -W "--help --stage -h -s a b c dev prod test" -- "$cur")) ;;
    ''*) COMPREPLY=($(compgen -W "--help --version -h -v deploy" -- "$cur")) ;;
  esac

If you have a way of converting a simple data structure (YAML) to a more elaborate lookback-based completions script, we can discuss this (preferably in the completely gem repo), but I suspect the complexity of such a script is not going to be worth it.

If you look at completions script for complex tools (like the one for git), you will see they are extremely long and complex. This cannot be generated automatically I suspect, and if it can, it will not be easy.

[tiagoposse]

I see what you mean.

This is what I had imagined, considering how the completely yaml looks like:

cli:
- "--help"
- "--version"
- "-h"
- "-v"
- deploy
cli deploy*-s):
- dev
- prod
- test
cli deploy*--stage:
- dev
- prod
- test
cli deploy:
- "--help"
- "--stage"
- "-h"
- "-s"
- a
- b
- c

So that the completion script is generated like:

  case "$comp_line" in
    deploy*-s) COMPREPLY=($(compgen -W "dev prod test" -- "$cur")) ;;
    deploy*--stage) COMPREPLY=($(compgen -W "dev prod test" -- "$cur")) ;;
    deploy) COMPREPLY=($(compgen -W "--help --stage -h -s a b c" -- "$cur")) ;;
    ''*) COMPREPLY=($(compgen -W "--help --version -h -v deploy" -- "$cur")) ;;
  esac

This seems to adhere to the current schema of the completely gem, but I might be wrong here. Unfortunately ruby is outside my range of programming languages, otherwise I'd give it a go.
I can definitely produce a PoC utility (not ruby unfortunately) as I feel this is doable considering the current yaml schema for bashly

[DannyBen]

Yeah. Good thought. I think I experimented with this approach of having a * in the middle of the expression. Don't remember why I abandoned it - I can try it again.

[tiagoposse]

Awesome! Thanks for the support and happy to help where I can

[DannyBen]

This manually crafted script works as you expect I think. Can you confirm?

#!/usr/bin/env bash

# This bash completions script was generated by
# completely (https://github.com/dannyben/completely)
# Modifying it manually is not recommended
_cli_completions() {
  local cur=${COMP_WORDS[COMP_CWORD]}
  local comp_line="${COMP_WORDS[@]:1}"

  case "$comp_line" in
    'deploy'*'--stage'*) COMPREPLY=($(compgen -W "dev prod test" -- "$cur")) ;;
    'deploy'*'-s'*) COMPREPLY=($(compgen -W "dev prod test" -- "$cur")) ;;
    'deploy'*) COMPREPLY=($(compgen -W "--help --stage -h -s a b c" -- "$cur")) ;;
    ''*) COMPREPLY=($(compgen -W "--help --version -h -v deploy" -- "$cur")) ;;
  esac
}

complete -F _cli_completions cli

If so - I feel it is doable to implement, but will require changes both in completely and bashly - so before that, lets just confirm that it provides a good example (and hopefully also confirm that it supports more complex / realistic use cases)

Also, if there is a simpler way to do this - one that can take the YAML pattern as is and use it (without the quotes / wildcard games), it would be nice.

[tiagoposse]

I can confirm this works:

_cli_completions() {
  local cur=${COMP_WORDS[COMP_CWORD]}
  local comp_line="${COMP_WORDS[@]:1}"

  case "$comp_line" in
    deploy*--stage) COMPREPLY=($(compgen -W "dev prod test" -- "$cur")) ;;
    deploy*-s) COMPREPLY=($(compgen -W "dev prod test" -- "$cur")) ;;
    deploy*) ; COMPREPLY=($(compgen -W "--help --stage -h -s a b c" -- "$cur")) ;;
    *) COMPREPLY=($(compgen -W "--help --version -h -v deploy" -- "$cur")) ;;
  esac
}

No quote games necessary.
It's important that flag completion patterns do not end with *, in order to enforce completion just for the flag, but not for lines where the flag is already present: deploy*-s in my example instead of deploy*-s* in yours.
As such, completions follow the pattern of <command>*<flag>

You can also opt for the following when using aliases (not sure if it would be easier to implement than one option per line) and can get messy with very long commands

deploy*--stage|deploy*-s) COMPREPLY=($(compgen -W "dev prod test" -- "$cur")) ;;

Examples:

cli deploy <TAB>
--help   --stage  -h       -s       a        b        c

cli deploy -s <TAB>
dev test prod

cli deploy -s dev <TAB>
--help   --stage  -h       -s       a        b        c

In case it matters:

bash --version
GNU bash, version 5.1.16

[DannyBen]

Weird - it didn't work for me. I will investigate.

In any case - now that I see demand for this, I have more motivation to make it work - so thanks for opening the issue.

As a side note - I should mention that I always use completions differently than most people (I think). I have this in my ~/.inputrc`:

# Change up/down arrows behavior
"\e[A": history-search-backward
"\e[B": history-search-forward

# Change tab behavior and add shift tab for old behavior
TAB: menu-complete
"\e[Z": complete

which makes TAB behave differently - instead of showing a list, it completes the line.
I find it much more useful (and in addition sets up/down arrows on a partial command to do a search for a command that starts with the already typed words).

Barring unforeseeable complications, I believe this can be done by the weekend.

Plus - it occurred to me it would be nice if the completely gem also have a completely test FILE WORDS... command that will show the complete suggestions for testing and playing purposes.

[DannyBen]

So far I found a couple of issues with this approach. The main issue is that the asterisk at the end is important - without it, partial completions are not triggered:

# this works
$ script --stage <TAB>
dev test prod

# this does not
$ script --stage pr<TAB>

since the $comp_line will not match anything, and will go to the final wildcard match.

[tiagoposse]

You most likely had the quotes because of spaces in the patterns:
commandA subcommandB ) does not work, but commandA' 'subcommandB ) works.

Did some more testing today and came up with the following:

_cli_completions() {
  local cur=${COMP_WORDS[COMP_CWORD]}
  local comp_line="${COMP_WORDS[@]:1}"

  case "$comp_line" in
    deploy*--stage?([[:space:]]*([^[:space:]])) ) COMPREPLY=($(compgen -W "dev devev test" -- "$cur")) ;;
    deploy*-s?([[:space:]]*([^[:space:]])) ) COMPREPLY=($(compgen -W "dev prod test" -- "$cur")) ;;
    deploy*) ; COMPREPLY=($(compgen -W "--help --stage -h -s a b c" -- "$cur")) ;;
    *) COMPREPLY=($(compgen -W "--help --version -h -v deploy" -- "$cur")) ;;
  esac
}

the "magic" would be to append ?([[:space:]]*([^[:space:]])) after a flag, to provide partial matches the first word only, if at all provided.

To support subcommands without quoting, a string replace 's/ /[[:space:]]/g' instead of quoting in completely should get around it. Example pattern:
frontend[[:space:]]deploy*-s?([[:space:]]*([^[:space:]]))

[DannyBen]

No, this type of pattern matching is not an option. It is hard to read even in its simplest form, and I get syntax errors.

I think there are two options:

1. Set the compline variable to NOT include the value in cur (the "active" cursor)
2. Switch from case and shell matching to if-else and regex matching

Also remember that all these use cases are expected to work:

Assuming the command is cli command subcommand --protocol ssh

• cli <tab>
• cli com<tab>
• cli command <tab>
• cli command sub<tab>
• cli command subcommand <tab>
• cli command subcommand --pro<tab>
• cli command subcommand --protocol <tab>
• cli command subcommand --protocol s<tab>

Also - this is the current template that I am working by

_mycomps() {
  local cur compline
  _init_completion -s || return
  cur=${COMP_WORDS[COMP_CWORD]}
  compline="${COMP_WORDS[@]:1}"

  if [[ -n "$COMPLETELY_DEBUG" ]]; then
    echo "compline: '$compline'" > 'completely-debug.txt'
    echo "cur:      '$cur'" >> 'completely-debug.txt'
  fi

  case "$compline" in
    deploy*) COMPREPLY=($(compgen -W "--help --stage -h -s a b c" -- "$cur")) ;;
    *) COMPREPLY=($(compgen -W "--help --version -h -v deploy" -- "$cur")) ;;
  esac
} &&
complete -F <%= function_name %> <%= command %>

[tiagoposse]

No, this type of pattern matching is not an option. It is hard to read even in its simplest form, and I get syntax errors.

I do agree this would be extremely hard to read. But it's weird that you're getting syntax errors, as all proposed completion scenarios work correctly in my environment.

I did consider proposing if/else regex matching, as this would be my go-to, but I can imagine this is more work.

[DannyBen]

So - in order to streamline our collaboration on this, I propose this file as the test file:

# comp.sh
_mycomps() {
  local cur compline
  _init_completion -s || return
  cur=${COMP_WORDS[COMP_CWORD]}
  compline="${COMP_WORDS[@]:1}"

  if [[ -n "$COMPLETELY_DEBUG" ]]; then
    echo "compline: '$compline'" > 'completely-debug.txt'
    echo "cur:      '$cur'" >> 'completely-debug.txt'
  fi

  case "$compline" in
    'command subcommand'*) COMPREPLY=($(compgen -W "--protocol ssh scp" -- "$cur")) ;;
    'command'*) COMPREPLY=($(compgen -W "subcommand" -- "$cur")) ;;
    *) COMPREPLY=($(compgen -W "--help command" -- "$cur")) ;;
  esac
} &&
complete -F _mycomps cli

# TESTS
source /usr/share/bash-completion/bash_completion

# usage: check NUMBER COMMAND EXPECTED
# where NUMBER is the number of words in the command, and substract one if the
# last word is partial that needs to be completed
check() {
  COMP_CWORD=$1
  COMP_WORDS=($2)
  expected="$3"
  _mycomps
  actual="${COMPREPLY[@]}"
  echo "compline: [${COMP_WORDS[*]}]"
  echo "expected: [$expected]"
  echo "actual:   [$actual]"
  [[ "$expected" == "$actual" ]] || echo "ERROR"
  echo
}

check 1 "cli" "--help command"
check 1 "cli comm" "command"
check 2 "cli command" "subcommand"
check 3 "cli command sub" "subcommand"
check 3 "cli command subcommand" "--protocol ssh scp"
check 3 "cli command subcommand --" "--protocol"

It includes the completion script + a testing facility + built in test cases.

Running this on my Ubuntu succeeds, and running it on a clean Docker ubuntu succeeds as well.

Since we have different systems, I also suggest that if you see it succeeds on your machine, test that it also works in a clean Ubuntu. Here is how to do it (assuming you have docker and docker-compose).

# Dockerfile
FROM ubuntu
WORKDIR /app
RUN apt-get update && apt-get install bash-completion

# docker-compose.yml
services:
  test:
    build: .
    image: bash-completions-test
    volumes:
      - .:/app
    command: bash -c /app/comp.sh

command to run:

$ docker compose run --rm test