Skip to content

Shell Autocompletion

andychu edited this page May 5, 2021 · 40 revisions
Clone this wiki locally

Related: How Interactive Shells Work

Threads that started this

More Threads


Things I've Learned about Shell Auto-Completion (So Far)

Here is a draft of an Oil blog post:

  • bash and zsh "boiled the ocean" in parallel, i.e. developed completions for common and not-so-common Unix commands.
  • zsh maintains many completions in-tree. Bash is at least two different projects.
    • although there is also the zsh-completions and oh-my-zsh projects.
  • bash does not use its own parser to figure out what the completion context is.
    • readline parses the line with COMP_WORDBREAKS.
    • Not only that, but bash-completion disregards that, and reparses COMP_LINE. bash is parsed in bash.
  • bash dynamically greps the output of 'ls' and other commands for flag completion.
  • git is the biggest completion by a factor of 2 in both bash and zsh.
  • Some commands partially handle their own completion logic.
  • Some libraries handle their own completion. See the list above.
  • Grammars / pure-declarative approaches are probably good enough for 90% of completions, but not the (IMO) most important use cases like git.
  • Grammar-based approaches have been tried in the past
  • zsh has a nicer completion UI.
    • It gives you help for flags.
    • It doesn't constantly redraw the line like readline! (argh)
  • Thomas Ballinger (contributor to the BPython shell) has a great blog about Terminals.
    • It's easier to draw stuff below the prompt than above the prompt.
  • Emacs and xonsh already do something like Shellac!!! They shell out to bash.

Big questions

  • Should we "boil the ocean" 0, 1, or N times?

    • 0: emulate bash
    • 1: share autocompletion among shells
    • N: everybody develops their own
  • What would a shared completion API / protocol / grammar / data format look like?


  • Visit and sign in with GIthub! _ Advantages:
    • Write async messages like e-mail, or chat synchronously in real-time.
    • Organize the conversation into threads (the git use case, grammars, Elvish design, etc.)
    • Markdown support for code examples
    • Notify via e-mail with @, but don't spam people's inbox with every message.
    • Public, so new people can view the history (unlike e-mail / IRC)
    • Easy login with Github.

Declarative/DSL/Grammar vs. Invoking Arbitrary Code Behind a Completion Server Protocol

  • Power
    • Is the DSL Turing complete? Does it need to be?
  • Latency
    • Something in-process has better latency.
  • Version skew of completion definitions:
    • Querying a binary directly eliminates version skew. The upstream author adds a flag via argparse or golang/flags, and everything should work.
    • Otherwise there is an extra step of generating a grammar, and this can get out of date.
      • (There are many different ways to distribute binaries now. Package managers are only one. For example, the Rust toolchain is updated with a curl | sh-style shell script.)
  • Version skew of the DSL itself.
    • Is it statically linked into every library?
  • Killer app: Could you implement the completion server protocol in VimScript? Yes, as long as it can invoke an arbitrary process.
    • You probably cannot implement a DSL in VimScript, or it would be very awkward.