Subparser help does not respect SUPPRESS argument

[BrettHannigan]

BPO	22848
Nosy	@warsaw, @gaul, @bitdancer, @mitar
Files	argparse_test.py: Quick python script illustrating the issue. argparse.py: Proposed patch to argparse.py (python2.7) argparse.patch issue22484.patch: help=SUPPRESS hides a subparser issue22848.py

^{Note: these values reflect the state of the issue at the time it was migrated and might not reflect the current state.}

Show more details

GitHub fields:

assignee = None
closed_at = None
created_at = <Date 2014-11-11.17:51:37.710>
labels = ['type-bug', 'library']
title = 'Subparser help does not respect SUPPRESS argument'
updated_at = <Date 2020-05-03.05:31:01.339>
user = 'https://bugs.python.org/BrettHannigan'

bugs.python.org fields:

activity = <Date 2020-05-03.05:31:01.339>
actor = 'mitar'
assignee = 'none'
closed = False
closed_date = None
closer = None
components = ['Library (Lib)']
creation = <Date 2014-11-11.17:51:37.710>
creator = 'Brett.Hannigan'
dependencies = []
files = ['37177', '37178', '37187', '39040', '47574']
hgrepos = []
issue_num = 22848
keywords = ['patch']
message_count = 14.0
messages = ['231035', '231098', '232400', '236253', '240883', '241111', '241124', '241133', '241203', '241310', '241841', '316070', '316137', '316249']
nosy_count = 11.0
nosy_names = ['barry', 'gaul', 'bethard', 'r.david.murray', 'mitar', 'derks', 'paul.j3', 'mattlong', 'Brett.Hannigan', 'Drake Bridgewater', 'floreal']
pr_nums = []
priority = 'normal'
resolution = None
stage = None
status = 'open'
superseder = None
type = 'behavior'
url = 'https://bugs.python.org/issue22848'
versions = ['Python 3.5']

[BrettHannigan]

When adding an argument to a subparser and passing help=argparse.SUPPRESS, I would expect this argument to not show up when running help. Instead, I find that the argument is listed and the help given is ==SUPPRESS==. For example (also in attached python script):

import argparse

parser = argparse.ArgumentParser('test')
subparsers = parser.add_subparsers()
parser_foo = subparsers.add_parser('foo', help='This is help for foo')
parser_bar = subparsers.add_parser('bar', help=argparse.SUPPRESS)

parser.parse_args(['-h'])

usage: test [-h] {foo,bar} ...

positional arguments:
  {foo,bar}
    foo       This is help for foo
    bar       ==SUPPRESS==

optional arguments:
  -h, --help  show this help message and exit

I believe I have found the proper fix in argparse.py
In the class _SubParsersAction look at the method add_parser().
There is the following block of code:

if 'help' in kwargs:
            help = kwargs.pop('help')
            choice_action = self._ChoicesPseudoAction(name, help)
            self._choices_actions.append(choice_action)

This should instead check to see if help is SUPPRESS or not like so:

if 'help' in kwargs:
            help = kwargs.pop('help')
            if help != SUPPRESS:
                choice_action = self._ChoicesPseudoAction(name, help)
                self._choices_actions.append(choice_action)

If I make this change locally, then the above code does in fact suppress printing the bar option.

[paulj3]

A notational point - you are adding a subparser, not an argument, to the subparsers action.

Why would a user want to use help=argparse.SUPPRESS, as opposed to simply omitting the help parameter? The effect would be the same as your patch.

Another possibility is to interpret this SUPPRESS as meaning, omit the subparser (and it's aliases?) from the choices list(s) as well. In other words, make this an entirely stealth choice.

usage: test [-h] {foo} ...
positional arguments:
  {foo}
    foo       This is help for foo
    ...

'test bar -h' would still display a help for that subparser (unless given a add_help=False parameter).

I don't know how much work this stronger SUPPRESS would required, or whether such an interpretation would be intuitive to most users. There isn't a mechanism to omit a regular argument from the usage, so why should there be a mechanism for omitting a subparsers choice from usage?

With the weaker interpretation, this patch fills in a hole in the logic, but doesn't add any functionality (that I can think of).

[warsaw]

Targetting to 3.5 and nosying myself. It would be nice if it were possible to suppress the help of an entire subparser, but I took a quick look at the code and it seems tricky.

[derks]

I would like to add to this regarding the following:

Why would a user want to use help=argparse.SUPPRESS, as opposed to simply omitting the help parameter? The effect would be the same as your patch.

This actually isn't the case, if you omit the 'help' option entirely, the sub-parser is still listed in the {short_list}. For example:

    import argparse

    root_parser = argparse.ArgumentParser(prog='myapp')
    root_parser.add_argument('--foo', action=CustomAction)
    sub_parsers = root_parser.add_subparsers(dest='commands', title='subcommands')
    sub_parser = sub_parsers.add_parser('sub-command-1', help='useless help txt')
    sub_parser = sub_parsers.add_parser('sub-command-2', help=argparse.SUPPRESS)
    sub_parser = sub_parsers.add_parser('sub-command-3')

    args = root_parser.parse_args()

You end up with:

$ python argtest.py --help
usage: myapp [-h] [--foo FOO] {sub-command-1,sub-command-2,sub-command-3} ...

optional arguments:
  -h, --help            show this help message and exit
  --foo FOO

subcommands:
  {sub-command-1,sub-command-2,sub-command-3}
    sub-command-1       useless help txt
    sub-command-2       ==SUPPRESS==

The 'sub-command-3' is still listed in the {sub-parsers} .... where, if I want to hide a sub-parser... I don't want to see it anywhere. Would be ideal to have a 'hide=True' option for sub-parsers and arguments alike. All the same functionality, you just wouldn't see it in '--help'.

[bitdancer]

I believe BJ is saying the same thing that Paul is, that SUPPRESS could omit the subparser from the list of subparsers. I haven't checked, but I believe this means that the proposed patch doesn't do anything other than what omitting the help would do, so a different patch is required to remove the subparser from the list of subparsers in the help.

[mattlong]

I prefer the idea of help=SUPPRESSED resulting in a "hidden" subcommand. That is, one that does not show up at all in the usage/help output:

    import argparse

    parser = argparse.ArgumentParser(prog='myapp')
    parser.add_argument('--foo', action=CustomAction)
    sub_parsers = parser.add_subparsers(dest='commands', title='subcommands')
    sub_parser = sub_parsers.add_parser('sub-command-1', help='sub-command-1 help')
    sub_parser = sub_parsers.add_parser('sub-command-2', help=argparse.SUPPRESS)
    sub_parser = sub_parsers.add_parser('sub-command-3')

    parser.parse_args(['-h'])

Would result in:

usage: myapp [-h] [--foo FOO] {sub-command-1,sub-command-3} ...

optional arguments:
  -h, --help            show this help message and exit
  --foo FOO

subcommands:
  {sub-command-1,sub-command-3}
    sub-command-1       normal subcommand help

Assuming this behavior, what should happen if you request help for a subparser with help=SUPPRESSED? That is, you know the subcommand exists even though it's not mentioned in the usage. I would assume it would show usage as normal (note the description kwarg for sub-command-2):

    import argparse

    parser = argparse.ArgumentParser(prog='myapp')
    parser.add_argument('--foo', action='store_true')
    sub_parsers = parser.add_subparsers(dest='commands', title='subcommands')
    sub_parser = sub_parsers.add_parser('sub-command-1', help='sub-command-1 help')
    sub_parser = sub_parsers.add_parser('sub-command-2',
                                        help=argparse.SUPPRESS,
                                        description='description of suppressed sub-command-2')
    sub_parser = sub_parsers.add_parser('sub-command-3')

    parser.parse_args(['sub-command-2', '-h'])

Would result in:

usage: myapp sub-command-2 [-h]

description of suppressed sub-command-2

optional arguments:
  -h, --help  show this help message and exit

An edge case to consider: what should top-level help look like if ALL subparsers are suppressed? It seems like a degenerate scenario, but complete is important, right? The one that feels most correct to me is to follow the current behavior when you call add_subparsers but never call add_parser:

    import argparse

    parser = argparse.ArgumentParser(prog='myapp')
    parser.add_argument('--foo', action='store_true')
    sub_parsers = parser.add_subparsers(dest='commands', title='subcommands')
    parser.parse_args(['-h'])

Currently results in:

usage: myapp [-h] [--foo] {} ...

optional arguments:
  -h, --help  show this help message and exit
  --foo

subcommands:
  {}

Another possibility would be to follow the current behavior when you never even call add_subparsers which would of course remove all notion that subcommands are accepted, but that doesn't feel right to me.

[mattlong]

Here's a patch for the proposal in my previous comment.

As Barry mentioned, it wasn't as straightforward as I had hoped due to parts of the usage text being generated by the state of both self._name_parser_map and self._choices_actions.

[bitdancer]

This patch looks good to me, but I did not go through the argparse logic fully. Paul, can you review this and make sure itliteral_eval makes sense? There is a small concern about user code that provides a Mapping for choices having an attribute collision on _is_help_suppressed, but I think it is probably small enough we can ignore it. Or consider it an "undocumented feature".

[paulj3]

It'll take me a while to work through this proposed patch. My first reaction is that I am uncomfortable with adding an attribute to all parsers just to implement this help feature. For one thing it seems to cross functionality boundaries.

I need to review how subparser help is implemented. A lot of the work is done in the subparsers action itself, not the HelpFormatter. I have a vague memory of another bug issue involving this subparser help. I'll have to look it up.

There are several bug issues related to formatting choices. In some cases they are too long to display properly. And this formatting adds an iterablity requirement. I worked on consolidating choices formatting into a separate method. In one version it was module level function, in another refactoring I made it an Action method. If it was an Action method, then the subparsers_action class could implement its own version.

Another thought - the formatting of choices is closely linked to the metavar.

Anyways, at this point I have a bunch of random thoughts that need research and organizing.

[paulj3]

Giving the subparsers Action a metavar might achieve everything the proposed patch does - without any code changes.

In the 1st test case:

    subparsers = parser.add_subparsers(title='subcommands',
                                           description='subcommands description',
                                           help='subcommands help',
                                           metavar='{1,2}')

and for the 2nd

    parser = argparse.ArgumentParser(prog='PROG_ALL',
                                            description='main description')
    subparsers = parser.add_subparsers(title='subcommands',
                                           description='subcommands description',
                                           help='subcommands help',
                                           metavar='{}')
    parser1 = subparsers.add_parser('1')  # no help
    parser2 = subparsers.add_parser('2', aliases=('2alias',))

Note that neither the patch nor the metavar affects how the choices are displayed in an error message, e.g.

usage: PROG [-h] {} ...
PROG: error: argument {}: invalid choice: 'foo' (choose from '1', '2', '2alias')

For more discussion on formatting of choices see http://bugs.python.org/issue16468

--------------------

While I'll continue to think about this issue, my tentative suggestion is to make this a documentation fix, rather than a code change. Just let users know that add_subparsers takes a metavar parameter, just like add_argument. It can hide or otherwise customize the listing of commands.

[paulj3]

A similar Stackoverflow question

http://stackoverflow.com/questions/21692395/hiding-selected-subcommands-using-argparse/21712058#21712058

[floreal]

Will this patch be integrated in future days / weeks / years? I still want to add a hidden subparser, but doing this by editing manually the metavar of the subparsers is not a good solution...

[paulj3]

I've reviewed the comments and proposed patch, and still think that the custom metavar is still the best fix.

subparses.metavar can be changed after subparsers has been created. The programmer could, for example, write a simple helper function that calls add_parser, and also appends names to a list. Then at the end, turn that list into a properly formatted metavar string.

subparsers.metavar = '(%s}'%','.join(['cmd1','foo','cmd3'])

In fact, if I were to write a patch, I'd take this approach, trying to confine all changes to the _SubParsersAction.add_parser method, and out of the HelpFormatter.