Add disable_interspersed_args() to argparse.ArgumentParser

[LA-Toth]

BPO	13966
Nosy	@rhettinger, @LA-Toth
PRs	bpo-13966: Add enable/disable_interspersed_args to ArgumentParser #30071
Files	argparse.patch: Adding disable_interspersed_args() argparse-disable_interspersed.patch: Adding disable_interspersed_args() with unit tests argparse.disable_interspersed_args.python35.diff: Documented and tested version of disable_interspersed_args()

^{Note: these values reflect the state of the issue at the time it was migrated and might not reflect the current state.}

Show more details

GitHub fields:

assignee = None
closed_at = None
created_at = <Date 2012-02-08.07:49:27.591>
labels = ['type-feature', 'library', '3.10']
title = 'Add disable_interspersed_args() to argparse.ArgumentParser'
updated_at = <Date 2021-12-12.10:36:45.938>
user = 'https://github.com/LA-Toth'

bugs.python.org fields:

activity = <Date 2021-12-12.10:36:45.938>
actor = 'python-dev'
assignee = 'none'
closed = False
closed_date = None
closer = None
components = ['Library (Lib)']
creation = <Date 2012-02-08.07:49:27.591>
creator = 'Laszlo.Attila.Toth'
dependencies = []
files = ['24449', '24546', '38275']
hgrepos = []
issue_num = 13966
keywords = ['patch']
message_count = 11.0
messages = ['152839', '153044', '153087', '153552', '166079', '181164', '185993', '186000', '186046', '216882', '236882']
nosy_count = 6.0
nosy_names = ['rhettinger', 'bethard', 'kalt', 'python-dev', 'paul.j3', 'Laszlo.Attila.Toth']
pr_nums = ['30071']
priority = 'normal'
resolution = None
stage = 'patch review'
status = 'open'
superseder = None
type = 'enhancement'
url = 'https://bugs.python.org/issue13966'
versions = ['Python 3.10']

[LA-Toth]

If someone ports his code from optparse to argparse, there is a limit, that options and non-options can be mixed by default, and this behaviour cannot be disabled easily, an extra '--' argument is required in the command line.

In some cases it is much prettier to explicitly disable this, as was available in the deprecated optparse module.

I attach a patch that does this, adds disable_interspersed_args() to argparse.ArgumentParser.

[kalt]

nice, thank you!

[bethard]

The idea and patch seem okay to me. Needs tests though.

[LA-Toth]

I added unit test, which revealed some bugs. These are fixe now.
The attached file contains both the unit tests and the updated patch.

[bethard]

The argparse changes and tests look good. The new method needs to be documented. You can see some other things (e.g. Misc/NEWS) that also need to be updated by running "make patchcheck" as described here:

http://docs.python.org/devguide/patch.html

[LA-Toth]

Unfortunatelly the implementation bugous as of now. I wrote additional tests.

self.parser = ErrorRaisingArgumentParser()
self.parser.add_argument('-a', action='store_true')
self.parser.add_argument('-b')
self.parser.add_argument('rem', nargs=argparse.REMAINDER)
self.assertEquals(self.parser.parse_args('-b 4 -a  -b 5'.split()), NS(a=True, b='5', rem=[]))

This part is OK. But with the new option:
self.parser.disable_interspersed_args()
self.assertEquals(self.parser.parse_args('-b 4 -a -b 5'.split()), NS(a=False, b='4', rem=['-a', '-b', '5']))

This assertation also passes because it contains the actual result, which is unexpected. This is because the code doesn't handle properly the arguments that are non-options, such as '-b' in this case.

I can't see a good solution for this.

[paulj3]

The optparse page gives a reason for disable_interspersed_args():

"Use this if you have a command processor which runs another command which has options of its own and you want to make sure these options don’t get confused. For example, each command might have a different set of options."

In argparse:
"argparse.REMAINDER. All the remaining command-line arguments are gathered into a list. This is commonly useful for command line utilities that dispatch to other command line utilities:"

If you have only one positional argument, and its nargs is REMAINDER, you have effectively disabled interspersed.

Argparse doesn't prohibit all interspersed positionals. You could, for example, have one or more positionals with other nargs that could be interspersed. But the REMAINDER one has to be last.

In the library, profile.py uses 'optparse.disable_interspersed'. I just replaced optparse with argparse using the REMAINDER, and got the same behavior. (Actually I used argparse.PARSER which effectively requires at least one argument.)

[paulj3]

Oops, I was wrong about this:
"Argparse doesn't prohibit all interspersed positionals. You could, for example, have one or more positionals with other nargs that could be interspersed. But the REMAINDER one has to be last."

    parser.add_argument('a')
    parser.add_argument('--foo')
    parser.add_argument('rest', nargs='...')
    parser.parse_args('a --foo b c'.split(' ')

produces:

    Namespace(a='a', foo=None, rest=['--foo', 'b', 'c'])

That is because, 'rest' matches an empty list of arguments. With an nargs='*' or '?', the same thing happens, both 'a' and 'rest' are used up when processing the first positional argument string.

nargs=argparse.PARSER (= 'A...') gives the expected

    Namespace(a='a', foo='b', rest=['c'])

In this case, 'rest' has to wait till the second set of positionals.

Documentation warns "Note that it generally doesn’t make much sense to have more than one positional argument with nargs='*'". Maybe it should warn against combining any of the 'zero or more' positionals with other positionals.

[paulj3]

Looking further at test_argparse.py, I should say that the behavior of multiple positionals when there is one cluster of positional argstrings is well illustrated in the tests. It's the behavior when there are multiple clusters (interspersed positionals) that can go against some intuitions.

[paulj3]

http://bugs.python.org/issue14191
implements the other side of optparse behavior - allowing a complete intermixing of optionals and positionals.

It does that with a new 'ArgumentParser.parse_intermixed_args()' method.

[LA-Toth]

It seems I found the solution in the attached file argparse.disable_interspersed_args.python35.diff, and it's much-much easier than I thought. I assume that this patch can cleanly applied to earlier versions (python 3.2-3.4), but I didn't check it.