argparse does not honor prefix_chars when adding default options

[dhellmann]

BPO	9444
Nosy	@dhellmann, @bitdancer
Files	argparse_prefix_chars_bug.py argparse_char_fix+doc.patch: patch against py3k to fix bug and make docs more clear argparse_test.patch: unit tests

^{Note: these values reflect the state of the issue at the time it was migrated and might not reflect the current state.}

Show more details

GitHub fields:

assignee = None
closed_at = <Date 2010-08-03.18:14:34.418>
created_at = <Date 2010-07-31.22:20:26.321>
labels = ['type-bug', 'library']
title = 'argparse does not honor prefix_chars when adding default options'
updated_at = <Date 2010-08-03.19:44:41.717>
user = 'https://github.com/dhellmann'

bugs.python.org fields:

activity = <Date 2010-08-03.19:44:41.717>
actor = 'doughellmann'
assignee = 'none'
closed = True
closed_date = <Date 2010-08-03.18:14:34.418>
closer = 'r.david.murray'
components = ['Library (Lib)']
creation = <Date 2010-07-31.22:20:26.321>
creator = 'doughellmann'
dependencies = []
files = ['18292', '18317', '18320']
hgrepos = []
issue_num = 9444
keywords = ['patch']
message_count = 20.0
messages = ['112219', '112220', '112257', '112258', '112308', '112316', '112317', '112326', '112330', '112353', '112355', '112356', '112357', '112406', '112445', '112472', '112555', '112603', '112644', '112657']
nosy_count = 5.0
nosy_names = ['bethard', 'doughellmann', 'r.david.murray', 'catherine', 'ted.turocy']
pr_nums = []
priority = 'normal'
resolution = None
stage = 'resolved'
status = 'closed'
superseder = None
type = 'behavior'
url = 'https://bugs.python.org/issue9444'
versions = ['Python 2.7', 'Python 3.2']

[dhellmann]

If an ArgumentParser is created with a prefix_chars string that does not include '-', the default options created for showing help (-h and --help) and the version (-v and --version) are invalid and generate an exception.

[dhellmann]

One solution would be to use the first character of prefix_chars when building those default options.

[tedturocy]

What is the appropriate behavior for long options when '-' is not one of the accepted prefix_chars? All of '-h', '--help', '-v', and '--version' are hardcoded in the ctor. If, for instance, prefix_chars='+', should the long options be '++help', or should long options simply be disabled?

Also, for backwards compatibility, to implement Doug Hellmann's suggestion, '-' should be used for '-h' and '--help' whenever '-' is listed in prefix_chars (so, e.g., prefix_chars='+-' will result in the same behavior as previous versions).

I have a working patch for this but wouldn't mind someone else's thoughts first.

[tedturocy]

Looking at the test fixtures that exercise argparse, it appears that the intended behavior when '-' is not a prefix_char is to accept a doubling of any of the prefix_chars for long arguments. That is, if '-' is not present in prefix_chars but ':' is, then an argument like '::help' would be a valid long argument.

I'm attaching a patch which fixes the originally reported problem as follows:

• If '-' is in prefix_chars, then the help and version arguments are '-h', '--help', '-v', '--version'
• If '-' is not in prefix_chars, then the first char in prefix_chars is used to lead the option, e.g., '+h', '++help', '+v', '++version' if prefix_chars starts with '+'.

Catherine Devlin is also sprinting here at PyOhio and will have a test fixture separately.

[bethard]

Yes, this looks fine, assuming a test is also added.

[dhellmann]

I haven't read the existing tests, but I am not seeing the behavior described by Ted in msg112258. If I specify the prefix_chars as '+/' and define a long option '//myopt' then using ++myopt on the command line gives an error that the option is unrecognized. I don't know if that's a bug or the desired behavior.

[dhellmann]

Oh, I should point out that last comment is describing what I see when using the unpatched 2.7 version of the module.

[tedturocy]

I was less than clear in what I wrote last night, Doug. What I mean is that the idiom "::foo" for a long argument, instead of the more standard-looking "--foo", appears in the test suite. This suggests to me that the intended behavior for the default help option should be to use a doubled prefix character in front of the long-form option.

For instance, if prefix_chars='+' and add_help=True, then the automatically provided help arguments will be '+h' and '++help'.

Certainly, if one explicitly adds an option '//myopt', we would not expect '::myopt' to also be accepted. I'm talking here only about the options that argparse adds as a convenience.

[dhellmann]

I was actually surprised that prefix_chars didn't allow *any* of those characters to indicate an option. For example, a program on Unix might use options that start with '-', but also support '/' as a prefix under Windows. If that's the intended behavior, that's OK, and maybe the docs can be made more specific.

It does, however, open the question of how to pick the prefix to use for automatically-generated options. Maybe the current behavior is best after all, so the programmer disabling the '-' prefix character realizes they need to handle the help and version options explicitly.

[bethard]

It is intentional that you have to specify both "/foo" and "+foo" if you want them to be aliases for the same argument. A common use case for prefix_chars is to define "+x" and "-x" options, which usually mean different things.

As far as the current issue, I believe the current behavior is buggy - an argument that you can never use is added to the parser automatically:

>>> parser = argparse.ArgumentParser(prefix_chars='+')
Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
  File "/Users/bethard/Documents/projects/python/2.X/Lib/argparse.py", line 1569, in __init__
    help=_('show this help message and exit'))
  File "/Users/bethard/Documents/projects/python/2.X/Lib/argparse.py", line 1246, in add_argument
    kwargs = self._get_optional_kwargs(*args, **kwargs)
  File "/Users/bethard/Documents/projects/python/2.X/Lib/argparse.py", line 1369, in _get_optional_kwargs
    raise ValueError(msg % tup)
ValueError: invalid option string '-h': must start with a character '+'

Yes you could explicitly specify add_help=True, but I think using an appropriate prefix character is actually the right behavior.

[dhellmann]

Explicitly specifying aliases makes sense, it just wasn't clear that was the intent from the existing documentation. So, I don't think the behavior needs to change, but a doc update might help.

[bethard]

A doc patch would also be welcome, but I do think it's a bug that "ArgumentParser(prefix_chars='+')" throws an exception, and I think Ted's patch looks fine to me.

[dhellmann]

Sorry I'm not being clear: I do like the patch, I think the exception should not be raised.

[tedturocy]

I'm uploading a new version of my patch which includes a proposed clarification to the documentation about the behavior in this case. Doug, does this make the documentation clearer to you? It is now explicit about the behavior for formulating the help option if '-' is not in prefix_chars.

[dhellmann]

Yes, that doc change is clear. Thanks!

[catherine]

Attached a unit test patch corresponding to Ted's patch. http://bugs.python.org/file18320/argparse_test.patch

[bitdancer]

The combined patches look good to me, and I tested the tests and patch. (I accidentally deleted the earlier fix patch, sorry). It would be nice to add one more test case that does prefix="+-/" to make sure - is used when it isn't first.

Steven, I'd be happy to commit this if you are OK with it, since I was mentoring the people who wrote it.

[bethard]

Yep, I'm fine with you committing this (after adding the prefix="+-/" you suggested). I don't have time right now to test the patches, but the code looks about right, and the tests ran fine for you, so I'm fine with it.

[bitdancer]

Committed (with the additional test) to py3k in r83657, and 2.7 in r83676.

[dhellmann]

Thanks, everyone!