argparse: subparsers, argument abbreviations and ambiguous option

[jakub]

BPO	14365
Nosy	@merwok
Files	argparse_subparsers_ambiguous_bug.py: Bug reproduction argparse_dirty_hack.py subparser_optionals.diff subparser_patch.diff

^{Note: these values reflect the state of the issue at the time it was migrated and might not reflect the current state.}

Show more details

GitHub fields:

assignee = None
closed_at = None
created_at = <Date 2012-03-18.20:27:59.700>
labels = ['type-bug', 'library']
title = 'argparse: subparsers, argument abbreviations and ambiguous option'
updated_at = <Date 2014-08-07.22:47:31.748>
user = 'https://bugs.python.org/jakub'

bugs.python.org fields:

activity = <Date 2014-08-07.22:47:31.748>
actor = 'paul.j3'
assignee = 'none'
closed = False
closed_date = None
closer = None
components = ['Library (Lib)']
creation = <Date 2012-03-18.20:27:59.700>
creator = 'jakub'
dependencies = []
files = ['24928', '24945', '26466', '31901']
hgrepos = []
issue_num = 14365
keywords = ['patch']
message_count = 14.0
messages = ['156272', '156279', '156280', '156290', '156302', '156304', '156306', '156352', '166059', '198456', '198507', '198508', '198563', '225044']
nosy_count = 5.0
nosy_names = ['bethard', 'eric.araujo', 'tshepang', 'paul.j3', 'jakub']
pr_nums = []
priority = 'normal'
resolution = None
stage = 'test needed'
status = 'open'
superseder = None
type = 'behavior'
url = 'https://bugs.python.org/issue14365'
versions = ['Python 2.7', 'Python 3.2', 'Python 3.3']

[jakub]

Assuming following:

1. optional argument, say "--foo", in a subparser;
2. at least two different optional arguments in the main parser prefixed with "--foo", say "--foo1" and "--foo2";

parsing fails with "error: ambiguous option: --foo could match --foo1, --foo2".

It seems like argument abbreviation mechanism described at http://docs.python.org/library/argparse.html#argument-abbreviations is not working properly when in use with subparsers...

[tshepang]

More or less a duplicate of 12713?

[tshepang]

Sorry, I meant bpo-12713.

[bethard]

Yep. Closing as duplicate.

[jakub]

I don't understand how both bugs are related. Surely, patch provided for bpo-12713 does not fix the issue I described.

[bethard]

My mistake. I see that the error you're getting is a bad interaction between the option in the main parser and an ambiguous option in the subparser.

[bethard]

The problem is basically that _parse_known_args calls _parse_optional to determine whether something is an optional or a positional. But _parse_optional only checks "if arg_string in self._option_string_actions", that is, if the string can be found in the main parser actions.

So either this method needs to check the subparser actions, or the "if arg_string in self._option_string_actions" stuff needs to be delayed until the subparser. Neither of these seems like a simple change, but I agree it's a bug.

[jakub]

Attached quick&dirty fix I'm currently using in my project. Maybe this will help to create a valid patch.

Instead of changing _parse_optional, I've overridden _get_option_tuples to scan subparsers for matching optional arguments if option is ambiguous in the current parser.

I've only skimmed through the code, so forgive me if that fix is useless.

[bethard]

I think the real fix needs to search recursively though all subparsers. Here's a first draft of a patch that does this. I believe your sample code works after this patch.

I don't have the time to turn your bug report into proper test (and add a test for the recursive bit), but hopefully this is enough to let someone else finish up fixing this bug.

And yes, your "dirty hack" was still helpful in putting this together. ;-)

[paulj3]

Steven's patch (subparse_optionals.diff) run with jakub's test case (argparse_subparses_ambiguous_bug.py) works. But if the input string is

    print(parser.parse_args('--foo baz'.split()))

produces

    Namespace(cmd=None, foo='baz', foo1=None, foo2=None)

(I added the 'cmd' subparse 'dest').

Two things seem to be going on now:

1. '--foo' is being parsed even though its subparser is not invoked,

2. and the subparser is not required.

The issue of whether subparsers are required or not is another issue. They used to be required, but the testing for 'required' was changed, and subparsers fell through the crack.

I suspect that if the missing subparser error were raised, the first issue wouldn't be apparent.

[paulj3]

I think the correction to the problem that I noted in the previous post is to return 'None, arg_string, None', rather than 'action, arg_string, None' in the case where the action is found in a subparser.

This is what '_parse_optional' does at the end:

        # it was meant to be an optional but there is no such option
        # in this parser (though it might be a valid option in a subparser)
        return None, arg_string, None

An input like '--foo baz' would then produce an 'invalid choice' error. Since '--foo' is an optional that the primary parser does not recognize, 'baz' in interpreted as a positional, in this case an invalid subparser choice.

I'm working on cleaning up a test script.

[paulj3]

In the last patch, 'parser.scan = True' is needed to activate this fix. I left that switch in for testing convenience.

[paulj3]

This the argparse patch as described in the previous post, along with test_argparse tests.

For now the tests handle both subparser required cases and not required ones ( http://bugs.python.org/issue9253 ). While error messages can differ based on this requirement, it does not affect the 'ambiguity' that underlies the current issue.

[paulj3]

Another issue dealing with abbreviations is close to being committed.

http://bugs.python.org/issue14910
argparse: disable abbreviation

[Conchylicultor]

It's 2022 and I'm still affected by this in Python 3.10.

import argparse

parser = argparse.ArgumentParser()
parser.add_argument("--flagA", type=str)
parser.add_argument("--flagB", type=str)
subparser = parser.add_subparsers(title="command")
build_parser = subparser.add_parser("build")
build_parser.add_argument("--flag", type=str)

parser.parse_args(["build", "--flag=asd"])

Raises

debug_flags.py: error: ambiguous option: --flag=asd could match --flagA, --flagB

The fix is to set allow_abbrev=False:

parser = argparse.ArgumentParser(allow_abbrev=False)