Issue with spacing in argparse module while using help

[falu2010]

BPO	29626
Nosy	@bitdancer, @serhiy-storchaka, @falu2010, @csabella
PRs	bpo-29626: spacing issue when using help in argparse #1835

^{Note: these values reflect the state of the issue at the time it was migrated and might not reflect the current state.}

Show more details

GitHub fields:

assignee = None
closed_at = None
created_at = <Date 2017-02-22.23:00:55.432>
labels = ['library']
title = 'Issue with spacing in argparse module while using help'
updated_at = <Date 2017-05-28.23:25:22.809>
user = 'https://github.com/falu2010'

bugs.python.org fields:

activity = <Date 2017-05-28.23:25:22.809>
actor = 'paul.j3'
assignee = 'none'
closed = False
closed_date = None
closer = None
components = ['Library (Lib)']
creation = <Date 2017-02-22.23:00:55.432>
creator = 'falu2010'
dependencies = []
files = []
hgrepos = []
issue_num = 29626
keywords = []
message_count = 11.0
messages = ['288389', '288403', '288674', '288676', '288679', '288681', '288683', '294615', '294619', '294655', '294663']
nosy_count = 5.0
nosy_names = ['r.david.murray', 'paul.j3', 'serhiy.storchaka', 'falu2010', 'cheryl.sabella']
pr_nums = ['1835']
priority = 'normal'
resolution = None
stage = 'resolved'
status = 'open'
superseder = None
type = None
url = 'https://bugs.python.org/issue29626'
versions = ['Python 2.7']

[falu2010]

When I use argparse the optional arguments with character "-p" and "-s" has improper spacing while using help/usage command.

Sample out put from argparse module:

usage: cli node list [-h] [-p] [-o]

Lists nodes in your current project.

optional arguments:
-h, --help show this help message and exit
// extra space after p
-p , --projectid
-o, --org (For administrators only) Lists all the nodes in

[paulj3]

We need to see the parser setup as well. I've never seen a bug like this before. The usage line suggests that you are using subparsers.

It might be better if you asked this on StackOverFlow with a repeatable code example. That's a better place to get debugging help of your own code. Come back here if others think this is a problem with argparse itself rather than your own setup.

[paulj3]

With this setup

import argparse
parser=argparse.ArgumentParser(prog='cli')
parser.add_argument('nodes')
sp=parser.add_subparsers()
p1 = sp.add_parser('list', description='Lists nodes in your current project')
p1.add_argument('-p','--projectid', action='store_true')
p1.add_argument('-o', '--org', action='store_true', help=' (For administrators only) Lists all the nodes in')
parser.parse_args()

this help looks normal

1354:~/mypy/argdev$ python3 bpo-29626.py nodes list -h
usage: cli nodes list [-h] [-p] [-o]

Lists nodes in your current project

optional arguments:
-h, --help show this help message and exit
-p, --projectid
-o, --org (For administrators only) Lists all the nodes in

Without further feedback this issue should be closed

[falu2010]

Sample code to repro this:

import argparse

    parser = argparse.ArgumentParser(prog='cli')
    subparsers = parser.add_subparsers(dest="command")

    subparsers_user_delete = subparsers.add_parser("delete", help="Deletes a user in your organization.",
                                                        description="Deletes a user in your organization.")
    subparsers_user_delete.add_argument("userid", help="The userid of user.", default=None, type=int)
    subparsers_user_delete.add_argument("-p", "--projectid", metavar="",
                                        help="Specify the project ID of project from where you want to delete"
                                             " the user or else user will be deleted from organization.", type=int,
                                        default=None)

    parser_nodes = subparsers.add_parser('nodes')
    sp = parser_nodes.add_subparsers()
    p1 = sp.add_parser('list', description='Lists nodes in your current project')
    p1.add_argument('-p', '--projectid', action='store_true')
    p1.add_argument('-o', '--org', action='store_true', help=' (For administrators only) Lists all the nodes in')
    parser.parse_args()

Please run following command:
$issue.py delete -h

[paulj3]

This help looks normal:

1427:~/mypy/argdev$ python3 bpo-29626.py delete -h
usage: cli delete [-h] [-p] userid

Deletes a user in your organization.

positional arguments:
userid The userid of user.

optional arguments:
-h, --help show this help message and exit
-p , --projectid Specify the project ID of project from where you want to
delete the user or else user will be deleted from
organization.

So does this help for the nested subparser:

1430:~/mypy/argdev$ python3 bpo-29626.py nodes list -h
usage: cli nodes list [-h] [-p] [-o]

Lists nodes in your current project

optional arguments:
-h, --help show this help message and exit
-p, --projectid
-o, --org (For administrators only) Lists all the nodes in

This double layered subparsers is not common, and might not even be included in the unittest file. But provided you don't try anything too tricky it does work. I've seen a few questions along this line on StackOverflow.

Note that the help line for '-p' in the second case is empty because you did not specify any help string (as you did for 'delete').

[falu2010]

There is an extra spacing here if you look carefully. I have commented
above the code. Similarly it happens for "-o" too but not for other
alphabets.

1427:~/mypy/argdev$ python3 bpo-29626.py delete -h
usage: cli delete [-h] [-p] userid

Deletes a user in your organization.

positional arguments:
userid The userid of user.

optional arguments:
-h, --help show this help message and exit
// extra space after p and ",".
-p , --projectid Specify the project ID of project from where you want
to
delete the user or else user will be deleted from
organization.

I saw this issue as I use argparse for my CLI usecase which has nested
commands.

On Mon, Feb 27, 2017 at 2:39 PM, paul j3 <report@bugs.python.org> wrote:

paul j3 added the comment:

This help looks normal:

1427:~/mypy/argdev$ python3 bpo-29626.py delete -h
usage: cli delete [-h] [-p] userid

Deletes a user in your organization.

positional arguments:
userid The userid of user.

optional arguments:
-h, --help show this help message and exit
-p , --projectid Specify the project ID of project from where you want
to
delete the user or else user will be deleted from
organization.

So does this help for the nested subparser:

1430:~/mypy/argdev$ python3 bpo-29626.py nodes list -h
usage: cli nodes list [-h] [-p] [-o]

Lists nodes in your current project

optional arguments:
-h, --help show this help message and exit
-p, --projectid
-o, --org (For administrators only) Lists all the nodes in

This double layered subparsers is not common, and might not even be
included in the unittest file. But provided you don't try anything too
tricky it does work. I've seen a few questions along this line on
StackOverflow.

Note that the help line for '-p' in the second case is empty because you
did not specify any help string (as you did for 'delete').

----------

---

Python tracker <report@bugs.python.org>
<http://bugs.python.org/issue29626\>

---

[paulj3]

Sorry, I missed that. For some reason I looking something bigger.

That's coming from the `metavar=""'.

If I specify `metavar="xxx" that help line will have

-p xxx, --projectid xxx

Replace the 'xxx` with '', and you still have space between '-p' and ','.

Now that I see it, it looks familiar. I noted it in passing in StackOverflow answer, http://stackoverflow.com/a/40497623/901925

I can't find a related bug/issue.

It's a natural consequence of the formatting in HelpFormatter._format_action_invocation

            # if the Optional takes a value, format is:
            #    -s ARGS, --long ARGS
            parts.append('%s %s' % (option_string, args_string))

There's no special handling for the case where ARGS is blank.

That formatter method could be customized as suggested in

http://stackoverflow.com/a/23941599/901925

Often people want a more compact invocation like:

-s, --long ARG help

Usage gets that space between option_string and args_string, but it gets striped out later.

So the fix (not tested) would something like:

      def _format_action_invocation(self, action):
           ....
           for option_string in action.option_strings:
               if len(args_string)>0:
                    parts.append('%s %s' % (option_string, args_string))
               else:
                    parts.append('%s' % option_string)
           ....

[serhiy-storchaka]

Does an empty metavar make sense? This makes the option with argument non-distinguising from the option without argument.

[csabella]

I submitted a PR for this based on paul.j3's suggested change:

      def _format_action_invocation(self, action):
           ....
           for option_string in action.option_strings:
               if len(args_string)>0:
                    parts.append('%s %s' % (option_string, args_string))
               else:
                    parts.append('%s' % option_string)
           ....

I created a test that would have duplicated the original issue and, with the change, no longer prints the extra blank.

To Serhiy's point, I'm not sure if an empty metavar makes sense since it's intention is to give the parser a way to refer to the expected arguments within the help messages.

However, with no metavar at all, the help output is:
-f FOO, --foo FOO

With metavar = '' (and this change), the output is:
-f, --foo

With metavar = '' (and without this change), the output is:
-f , --foo

[bitdancer]

If I'm interpreted what the OP wrote correctly, he wanted the help text to not show that the option takes an argument, but instead rely on the help text to show that. That works for the option text, but it doesn't work for the synopsis (the synopsis is just wrong in that case).

Probably a blank metavar should just be rejected because of the issue with they synopsis. That should only be done in 3.7 if we do it, though.

Perhaps the OP will respond to explain the blank-metavar use case in more detail.

[paulj3]

I don't anticipate any backward compatibility issues with the proposed patch. But at the same time it feels almost too trivial of an issue to merit a patch.

A short metavar such as metavar="ID"' or even metavar="_"' would be nearly as compact, and still leave an indicator of the required argument (in usage as well as the help).

I'm not even sure if disallowing a blank metavar is worth the change.

As indicated before I've only seen this extra space problem when I or others suggest the blank metavar as a way making a more compact help invocation. That has come up frequently on SO, but I can't find a relevant bug/issue.

https://stackoverflow.com/a/18280876/901925 - here I suggest suppressing the help line and putting custom info in the group's description.

https://stackoverflow.com/questions/9642692/argparse-help-without-duplicate-allcaps - suggests a help formatter subclass

So maybe the bigger issue is, can we make it easier to customize the the help invocation formatting?

Things to consider:

• metavar affects both the help line and usage

• setting help width and indentation is a messy part of the formatter

• there are unittests for long and long-long-longer option names to test these indentation issues.

As with some other help formatter issues, we are treading a fine line between making trivial patches, and substantive ones that make it cleaner and more robust.