pass the namespace to completion function

[teto]

I would like to set a choices_function that depends on the previous arguments, i.e., I believe cmd2 should pass the current parser namespace when calling the choices_function
Like I have this first argument that loads a csv file, and I would like the 2nd argument to choose between availables ids in the previously loaded csv file.
Right now the choices_function looks a bit useless and can only return static results.

[anselor]

@kmvanbrunt is working on an update to the autocompleter that will give the completer function that will give a dict containing all prior arguments.

[teto]

oh cool, seems like it's https://github.com/python-cmd2/cmd2/commits/completion_state .
I will try it locally if the API is stable ?

[tleonhardt]

@kmvanbrunt Can you comment as to the likely API stability of the code on the completion_state branch?

[anselor]

@teto I think it'd probably be worthwhile for you to try it and give some feedback. From my conversation with @kmvanbrunt about it I don't think it's going to change very much.

Also, I think it's actually giving a namespace of string tokens, not a dict.

[kmvanbrunt]

@teto
choices_function and choices_method serve two purposes

1. argparse's choices parameter only allows Iterables (e.g. lists, sets, tuples). If a choices list needs to be generated from a dynamic source like a database or other application state data, then choices_function and choices_method allow for this.

2. If you have a lot of choices and don't want help text that looks like the following, then providing this list with one of our parameters solves the problem.

Usage: fake [-h]
            {choice_01, choice_02, choice_03, choice_04, choice_05, choice_06,
            choice_07, choice_08, choice_09, choice_10, choice_11, choice_12,
            choice_13, choice_14, choice_15, choice_16, choice_17}

The documentation in argparse_custom.py says this about these parameters:

Pass a function that returns choices. This is good in cases where the choice list is dynamically generated when the user hits tab.

[kmvanbrunt]

@teto @anselor @tleonhardt
The completion_state branch is mostly done, but I'd like feedback on my design choices.
Once the design is finalized, then I need to update documentation and unit tests.

I've added a parameter called pass_parsed_args to:

• add_argument()
• set_choices_function()
• set_choices_method()
• set_completer_function()
• set_completer_method()

If this parameter is True, then choices/completer functions is passed an argparse.Namespace by AutoCompleter. The namespace contains tokens prior to the one being tab completed organized under the argparse argument they belong to.

Ex:

parser = Cmd2ArgumentParser()
parser.add_argument('arg1')
parser.add_argument('arg2')
parser.add_argument('arg3')

If you are completing arg3, then the Namespace will have arg1 and arg2 members. All members of the Namespace are lists of strings that contain the tokens which belong to that argument.

If arg3 was changed to:

parser.add_argument('arg3', nargs=3)

If you were tab completing the third token of arg3, then the Namespace will now have an arg3 member whose value is a list of the first two tokens of arg3.

So I'd like opinions on these design choices

1. Is pass_parsed_args a good name?
2. All members of the Namespace are lists, even if a particular argument expects only 1 token
3. Since AutoCompleter is for tab completion, it does not convert the tokens to their actual argument types or validate their values. All tokens are stored in the Namespace as the raw strings provided on the command line. It is up to the developer to determine if the user entered the correct argument type (e.g. int) and validate their values.

[teto]

I wonder if pass_parsed_args makes sense ? Isn't it simpler to always pass arguments ? it breaks backwards compatibility but the feature is very recent (~a month ?) and anyway recent cmd2 versions haven't been scared of breaking backwards compatibility.

the functions set_choices_function/set_choices_method have been introduced following one of my issues I think, yet they don't seem to do much, they seem more like a burden to the devs so might be best to get rid of them ?

I wonder if having a cmd2.Action that derives from argparse.Action could not help with implementation ? just a random thought I am not familiar with the code enough to make that call.

  File "/nix/store/k85cwpsdamsznlk2yk1rzjbym3pjhwb3-python3.7-cmd2-0.9.17/lib/python3.7/site-packages/cmd2/cmd2.py", line 1569, in complete
    self._completion_for_command(text, line, begidx, endidx, shortcut_to_restore)
  File "/nix/store/k85cwpsdamsznlk2yk1rzjbym3pjhwb3-python3.7-cmd2-0.9.17/lib/python3.7/site-packages/cmd2/cmd2.py", line 1454, in _completion_for_command
    self.completion_matches = self._redirect_complete(text, line, begidx, endidx, compfunc)
  File "/nix/store/k85cwpsdamsznlk2yk1rzjbym3pjhwb3-python3.7-cmd2-0.9.17/lib/python3.7/site-packages/cmd2/cmd2.py", line 1249, in _redirect_complete
    return compfunc(text, line, begidx, endidx)
  File "/nix/store/k85cwpsdamsznlk2yk1rzjbym3pjhwb3-python3.7-cmd2-0.9.17/lib/python3.7/site-packages/cmd2/cmd2.py", line 1607, in _autocomplete_default
    return completer.complete_command(tokens_to_parse, text, line, begidx, endidx)
  File "/nix/store/k85cwpsdamsznlk2yk1rzjbym3pjhwb3-python3.7-cmd2-0.9.17/lib/python3.7/site-packages/cmd2/argparse_completer.py", line 255, in complete_command
    return completer.complete_command(tokens[token_index:], text, line, begidx, endidx)
  File "/nix/store/k85cwpsdamsznlk2yk1rzjbym3pjhwb3-python3.7-cmd2-0.9.17/lib/python3.7/site-packages/cmd2/argparse_completer.py", line 324, in complete_command
    begidx, endidx, consumed_arg_values)
  File "/nix/store/k85cwpsdamsznlk2yk1rzjbym3pjhwb3-python3.7-cmd2-0.9.17/lib/python3.7/site-packages/cmd2/argparse_completer.py", line 473, in _complete_for_arg
    arg_choices = arg_choices.to_call(*args)
TypeError: 'ChoicesCallable' object is not callable

in
teto/mptcpanalyzer@370fb52#diff-32b8d0cac9da815b67726256f094d4aeR555

[kmvanbrunt]

@teto
pass_parsed_args is needed because completer functions aren't required to take a Namespace. For instance, path_complete doesn't take one. This is the same reason we have the choice_function/choice_method and completer_function/completer_method distinctions. They basically tell the AutoCompleter whether or not the functions expect self to be passed.

Not all cmd2 software uses the argparse capabilities, but they still can use the tab completion functions without worrying about passing a Namespace.

The set_choices_*/set_completer_* functions I added are for custom action types which enable tab completion in their initializer. These functions do exactly what add_argument() would do without the developer needing to know the underlying details of ChoicesCallable.

I'm not sure how having a cmd2.Action would make life easier. Passing parameters to add_argument already accounts for actions of any type. What did you have in mind with this suggestion?

Assuming the params dictionary defines parameters passed to add_argument(), your code needs to be changed to:

params = {
            'action': "store",
            'help': proto_str + '.stream wireshark id',
            'choices_function': show_range,
            'pass_parsed_args': True
        }

[teto]

pass_parsed_args is needed because completer functions aren't required to take a Namespace. For instance, path_complete doesn't take one

I think it's best to always pass on the namespace (better more info than none + it reduces complexitiy since we can remove pass_parsed_args), even if it means ignoring it in the completer.
Btw with your change, I can make it work. Very cool thanks.

[teto]

I would like to run functions that may take some time to complete (like loading a csv on the fly). zsh/bash can interrupt autocompletion via Ctrl-C, would something like that be doable with cmd2 ? Like interrupt the autocompletion function via Ctrl-C.

[kmvanbrunt]

cmd2 already has a Ctrl-C handler called sigint_handler() that will interrupt your function. You won't have to make any changes.

[teto]

ok I will try. I have another question: my understanding of cmd2 is that the Namespace is filled by each Action when the action is called. Nevertheless, your branch manages to pass a Namespace that is filled even though actions have not been called yet. Do you do some magic first ? or do I get the argparse working wrong.

One of my issue is that I would like to trigger the Action of the previous argument upon calling completion. I wonder if it wouldn't be worth to pass the parser along with the namespace.