Tab completion flags

[bitprophet]

See e.g. https://github.com/hhatto/zsh_completions/blob/master/_invoke for a zsh completion somebody wrote for Invoke as-is. My ideas for this (on top of whatever is in Fabric #6) is to use Invoke's existing parser to print easily computer readable info, e.g.:

[12:56:44] instead, output the flags/opts in a manner that is easier to parse, ditto for subcommand list
[12:56:51] and, I assume zsh supports this, the flags for each subcommand as well
[12:57:28] So one can: inv <tab, see flags/commands> mycom<tab, completes 'mycommand'> <tab, see 'mycommand' specific flags>
[12:58:17] So as long as I can make it easy to do stuff like 'inv --opts' or 'inv --opts ' or a more computer-formatted 'inv --list'...anybody who knows their shell's completion setup can adapt that info

[jthigpen]

So in my use case, I don't need command option completion (--blah), I just wanted to be able to tab complete the task names. I came up with this: https://gist.github.com/8853914. This works on my mac and an ubuntu 12.04 box.

[colonelpanic8]

Is this issue only for the tab completion of flag (i.e. -- / part of invoke arguments) or is it also referring to the completion of commands? I just wrote a zsh completion for tasks at
https://github.com/IvanMalison/dotfiles/blob/master/dotfiles/lib/completions/_invoke

Its similar to @jthigpen's solution but it doesn't get tripped up by aliases.

I'd be happy to integrate the work done by @hhatto to provide a complete zsh completion library.

[colonelpanic8]

https://github.com/kislyuk/argcomplete might be worth looking at.

[bitprophet]

@IvanMalison It's for both, the general point is "tab completion setups need to get info out of the tool as to what valid completion values are" (true for any completion at all - core flags, task names, task flags, etc) and so this is the ticket for "have core flags to easily spit out this info so completion scripts can just ask invoke directly instead of hardcoding".

Argcomplete (at a glance) seems argparse specific, so I dunno if it'd be useful for us? I'm expecting that our implementation would be pretty small and just tie into the parser classes.

[colonelpanic8]

yeah the version at https://github.com/IvanMalison/dotfiles/blob/master/dotfiles/lib/completions/_invoke is now working for both subcommands and core flags, but it has to do a lot of nasty things to parse the output of invoke -l.

Task flags would require some way to access that information (which is not currently available as far as i know). Is there anything blocking the merge of #163 ? I'd like to make accessing the available subcommands (and their arguments) easier.

[bitprophet]

Nothing blocking it besides time/priorities :( since someone (you) is currently active & willing to poke at the shell side of things, I'll see if I can put some time aside this week to review #163 and do whatever else is necessary to get this stuff exported in a friendly manner. Thanks!

[bitprophet]

Looking over #163 raises the following more specific questions we need to answer for this to be "done well":

• What formats work best for machine-readable output? Space separated, line separated, other?
  • Guessing line-separated as this tends to work best with most arbitrary shell tools.
• Do the most common shells' completion mechanisms work best with individual 'types' of output (eg one flag for each of tasks, core flags, per-task flags, etc) or would a single "spit everything out" flag be better?
  • From what I recall, absolutely the former - we'd probably need one for "list all tasks", one for "list all core options", and then could build it out with contextualized completion lists for individual tasks, or even for namespaces (think inv <tab>do<tab>bu<tab> to complete inv docs.build)
• What actual names to use for these?
  • added --shortlist flag, primarily for driving zsh completion #163 calls the completion oriented task list --shortlist but this doesn't work as well for flags (--shortflags is confusing and has its own meaning anyways (-b vs --build).
  • --complete-<x>? eg --complete-tasks, --complete-flags, etc? maybe s/complete/completion?
  • Gets more complex if we add contextualization/args, e.g. inv --flags-for mytask?
  • So maybe --tasks, --flags, --tasks-for <namespace> and --flags-for <task>?

[colonelpanic8]

Looking over #163 raises the following more specific questions we need to answer for this to be "done well":

What formats work best for machine-readable output? Space separated, line separated, other?
Guessing line-separated as this tends to work best with most arbitrary shell tools.

Agreed, although space separated is not THAT hard to work with.

Do the most common shells' completion mechanisms work best with individual 'types' of output (eg one flag for each of tasks, core flags, per-task flags, etc) or would a single "spit everything out" flag be better?
From what I recall, absolutely the former - we'd probably need one for "list all tasks", one for "list all core options", and then could build it out with contextualized completion lists for individual tasks, or even for namespaces (think inv dobu to complete inv docs.build)
When I first read this, I was inclined to agree, but now that I've thought about it a little bit, I think it might be better to have a single command because there is fixed startup cost involved in starting a python process and then having invoke run tasks.py that could be large enough that it would make invoke completion feel laggy. In my experience, it is bad enough that it makes sense to use zsh completion caching to avoid calling it over and over again every time you are attempting completion.

What actual names to use for these?
#163 calls the completion oriented task list --shortlist but this doesn't work as well for flags (--shortflags is confusing and has its own meaning anyways (-b vs --build).
--complete-? eg --complete-tasks, --complete-flags, etc? maybe s/complete/completion?
Gets more complex if we add contextualization/args, e.g. inv --flags-for mytask?
So maybe --tasks, --flags, --tasks-for and --flags-for ?

perhaps:
--completion-info
--machine-readable

[bitprophet]

A wrinkle in task listing: some users might expect "full" listings, e.g. a deep namespace tab-completing entire task paths, e.g. a "task list" might be deploy, docs.build, docs.clean. This is what I and #163 were assuming.

But others might prefer a more "build it up" approach where the first tab is the first level of the namespace (deploy, docs) (perhaps with trailing dots denoting non-leaf paths) and subsequent tabs added depth. So e.g. inv <tab>do<tab>b<tab> would first display deploy and docs., then docs.build and docs.clean, and finally docs.build would be the last item to complete (with a space after it).

I'd expect the split between these two options to depend on the size of one's task namespace, with small ones preferring the former and anything bigger, the latter. This implies that if we pick just one, we go the latter, more 'intelligent' route.

Having both might be overdoing it, but given we need these handful of flags either way, probably not a huge deal.

[colonelpanic8]

A wrinkle in task listing: some users might expect "full" listings, e.g. a deep namespace tab-completing entire task paths, e.g. a "task list" might be deploy, docs.build, docs.clean. This is what I and #163 were assuming.

But others might prefer a more "build it up" approach where the first tab is the first level of the namespace (deploy, docs) (perhaps with trailing dots denoting non-leaf paths) and subsequent tabs added depth. So e.g. inv dob would first display deploy and docs., then docs.build and docs.clean, and finally docs.build would be the last item to complete (with a space after it).

I'd expect the split between these two options to depend on the size of one's task namespace, with small ones preferring the former and anything bigger, the latter. This implies that if we pick just one, we go the latter, more 'intelligent' route.

Having both might be overdoing it, but given we need these handful of flags either way, probably not a huge deal.

Well, zsh, for example, sort of handles this automatically in that it will complete up to the first dot if there is an ambiguity, and then cycle through the available choices that matched the original input if tab is pressed again.

[bitprophet]

Finally dug into this enough to realize the approach outlined above (flags for printing tasks and/or options) doesn't really work - any completion requires use of the Invoke cli parser to figure out what the current task context is. Having --tasks and --flags and --flags-for only makes sense if the completion script using them knows what that context is, and it can't without giving Invoke the full command string being completed.

For example, yes, a naive completion of inv <tab> could use --tasks, but inv task1 --arg1 task2 --arg2 <tab> needs to know that it's "within" task2 and that valid completions are a combination of task2 flags and other task names. The only way to know whether the current context is task1 (which it would be if --arg1 took a value) or task2 (if --arg1 was a boolean) is to run the parser.

On one hand this means fewer new flags, on the other it means I need to make sure we can execute the parser in a mode that works well for this. Suspect it won't be that hard, since we already have options for running the parser in a "ignore everything you can't understand" mode - just need a way for it to 'export' its current state.

[bitprophet]

Of course, this also means we need a way of getting a possibly-complex Invoke command line into an Invoke parameter argument itself, but that should be doable either by using STDIN or something like invoke --complete -- <command line here>.

[bitprophet]

Also need to clarify behavior re: 'bare' completion - should inv <tab> complete task names only, or task names and core options?

One of the tab completion scripts submitted for Fabric only completes core options when the partial word starts with -, and this also appears to be how e.g. my Git completion module (for zsh) behaves. Feels like a reasonable compromise.