-
-
Notifications
You must be signed in to change notification settings - Fork 29.5k
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Add "necessarily inclusive" groups to argparse #55797
Comments
Just as some options are mutually exclusive, there are others that are "necessarily inclusive," i.e. all or nothing. I propose the addition of ArgumentParser.add_necessarily_inclusive_group(required=True). This also means that argparse will need to support nested groups. For example, if I want to set up options such that the user has to provide an output file OR (an output directory AND (an output file pattern OR an output file extension)): output_group = parser.add_mutually_exclusive_group(required=True)
output_group.add_argument("-o", "--outfile")
outdir_group = output_group.add_necessarily_inclusive_group()
outdir_group.add_argument("-O", "--outdir")
outfile_group = outdir_group.add_mutually_exclusive_group(required=True)
outfile_group.add_argument("-p", "--outpattern")
outfile_group.add_argument("-s", "--outsuffix") The usage should then look like: (-o FILE | (-O DIR & (-p PATTERN | -s SUFFIX)) |
I think this is a great suggestion. Care to work on a patch? |
I am subscribing to this idea as I've just fall into such use case where I need it. I would like to submit a patch, but I still have difficulties to understand argparse code not much spare time to spent on this. |
The suggestion in this issue is to add a 'mutually_inclusive_group' mechanism, one that would let users specify that certain sets of arguments must occur together. Furthermore there was mention of allowing some sort of nesting. Modeling it on the mutually_exclusive_group would be straight forward. But should it affect the usage and help display?mutually_exclusive_groups add a messy layer to the usage formatting. The only place such a group would act would be at the end of '_parse_known_args', where the current code checks for things like required actions (and mxgroups). A test at this point could use 'namespace', 'seen_actions' and 'seen_non_default_actions' to check whether the required group actions were seen. But the only thing that the argument_group contributes to this test is a list of argument names ('dest'?). Why not provide this list directly? And what if the user wants A to occur together with either B or C, but not both? Or make the inclusivity conditional on the value of A? Currently users can define argument interactions in a couple of ways. They can define custom Actions. In test_argparse.py there's a custom Actions test that does something like this (using '--spam' and 'badger'). But tests in Actions depend on the order in which arguments are given. An alternative is to test for interactions of arguments after I am proposing 'cross_test' mechanism that would give the user access to the 'seen_actions' and 'seen_non_default_actions' sets that 'mutually_exclusive_groups' use. Specifically an optional function can be called at the end of '_parse_known_args' that has access to these sets as well as the parser and the namespace. The core of the change would be adding cross_test = getattr(self, 'cross_test', None)
if cross_test:
cross_test(self, namespace, extras, seen_actions, seen_non_default_actions) at the end of 'parser._parse_known_args'. In addition 'cross_test' (or some other name) could be added to the 'ArgumentParser.__init__' arguments. The feature could be used by defining such a 'cross_test' function and adding it to the parser (either instance or subclass) def foobar(self, namespace, extras, seen_actions, seen_non_default_actions):
...
(raise self.error(...))
parser.cross_test = foobar The patch proposed http://bugs.python.org/issue18943 should be included I am working on tests and examples of such functionality. |
Regarding a usage line like:
The simplest option is to just a custom written 'usage' parameter. With the existing HelpFormatter, a nested grouping like this is next to impossible. It formats the arguments (e.g.'-O DIR'), interleaves the group symbols, and then trims out the excess spaces and symbols. http://bugs.python.org/issue10984 is a request to allow overlapping mutually_exclusive_groups. It loops on the groups, formatting each. It would be easier with that to format several different types of groups, and to handle nested ones. What would it take to convert a usage string like that into a logical expression that tests for the proper occurrence (or non-occurrence) of the various arguments. It might, for example be converted to exc(file, inc(dir, exc(pattern, suffix))) where 'exc' and 'inc' are exclusive and inclusive tests, and 'file','dir' etc are booleans. And what would be the error message(s) if this expression fails? I can imagine a factory function that would take usage line (or other expression of groupings), and produce a function that would implement a cross_test (as outlined in my previous post). It would be, in effect, a micro-language compiler. |
This patch uses: tests = self._registries['cross_tests'].values() to get a list of functions to run at the end of '_parse_known_args'. I replaced all of the mutually_exclusive_group tests (3 places) in the ArgumentParser with a static function defined in class _MutuallyExclusiveGroup, and registered this function. This refactoring should make it easier to add other specialized groups (e.g. inclusive) in the future. I'm using the _registries because they are already being shared among groups. A user can also register a custom testing function. For example: def inclusive_test(parser, seen, *args):
# require an argument from one of the 2 groups, g0 and g1
g0seen = seen.intersection(g0._group_actions)
g1seen = seen.intersection(g1._group_actions)
if len(g0seen.union(g1seen))==0:
parser.error('one of the 2 groups is required')
parser.register('cross_tests','inclusive', inclusive_test) This patched 'argparse.py' runs 'test_argparse.py' without error. This patch does not include the bpo-18943 changes, which make setting 'seen_non_default_actions' more reliable. |
This is an example of using 'patch_w_mxg2.diff' to handle the inclusive group case proposed by the OP. Since 'seen_non_default_actions' (and 'seen_actions') is a set of 'Actions', it is convenient to use 'set' methods with pointers to the actions that a collected during setup. Tests could also be done with the 'dest' or other action attributes. In this example I wrote 3 simple tests corresponding to the 3 proposed groups, but they could also have been written as one test. a_file= parser.add_argument("-o", "--outfile", metavar='FILE')
a_dir = parser.add_argument("-O", "--outdir", metavar='DIR')
a_pat = parser.add_argument("-p", "--outpattern", metavar='PATTERN')
a_suf = parser.add_argument("-s", "--outsuffix", metavar='SUFFIX')
...
def dir_inclusive(parser, seen_actions, *args):
if a_dir in seen_actions:
if 0==len(seen_actions.intersection([a_pat, a_suf])):
parser.error('DIR requires PATTERN or SUFFIX')
parser.register('cross_tests', 'dir_inclusive', dir_inclusive)
... In theory tests like this could be generated from groups as proposed by the OP. There is one case in 'test_argparse.py' where a mutually_exclusive_group is nested in an argument_group. But the current groups do not implement nesting. A (plain) argument_group does not share its '_group_actions' list with its 'container'. A mutually_exclusive_group shares its '_group_actions' but the result is a flat list (no nesting). For now I think it is more useful to give users tools to write custom 'cross_tests' than to generalize the 'group' classes. |
http://stackoverflow.com/questions/11455218 $ python myScript.py --parameter1 value1
$ python myScript.py --parameter1 value1 --parameter2 value2
$ python myScript.py --parameter2 value2 # error This is an example where a 'mutually inclusive group' wouldn't quite do the job. The proposed answers mostly use a custom Action. I'd lean toward an after-the-parse test of the namespace. With the patch I proposed this could be implemented with: a1 = parser.add_argument("--parameter1")
a2 = parser.add_argument("--parameter2")
def test(parser, seen_actions, *args):
if a2 in seen_actions and a1 not in seen_actions:
parser.error('parameter2 requires parameter1')
parser.register('cross_tests', 'test', test) One poster on that thread claimed that the use of 'a1 = parser.add_argument...' is using an undocumented feature. The fact that |
The addition of a simple decorator to the 'ArgumentParser' class, would simplify registering the tests: def crosstest(self, func):
# decorator to facilitate adding these functions
name = func.__name__
self.register('cross_tests', name, func) which would be used as: @parser.crosstest
def pat_or_suf(parser, seen_actions, *args):
if 2==len(seen_actions.intersection([a_pat, a_suf])):
parser.error('only one of PATTERN and SUFFIX allowed') |
A couple more thoughts on an expanded argument testing mechanism:
'seen_actions' is used only to test whether all required actions have been seen. These 2 sets differ in how positionals with '?*' are categorized. Positionals like this are always 'seen', even if they just get the default value. But they are not required (the case of a '*' positional without default needs to be revisited.)
For example the decorator could wrap the 'seen_non_default_actions' argument in a 'seen' function. Such a function could accept either an Action or a 'dest' string, it could accept a single Action, or a list of them, etc. There could be other functions like 'count', 'unique', 'mutually_exclusive', 'inclusive', etc. def testwfnc(func):
# decorator to register function and provide 'seen'
name = func.__name__
def wrapped(parser, seen_actions, *args):
def seen(*args):
actions = seen_actions
if isinstance(args[0], str):
actions = [a.dest for a in actions]
if len(args)>1:
return [a in actions for a in args]
else:
return args[0] in actions
return func(parser, seen)
parser.register('cross_tests', name, wrapped)
return wrapped
#@testwfnc
def test(parser, seen, *args):
if seen(a_file):
print(seen(a_dir, a_pat, a_suf))
cnt = sum(seen(a_dir, a_pat, a_suf))
if cnt>0:
parser.error('FILE cannot have DIR, PATTERN or SUFFIX')
... The attached script experiments with several versions of decorators. Some sort of testing Class is probably the way to go if we want to provide many convenience methods. |
http://stackoverflow.com/questions/22929087 In a subparser:
The |
I have developed a UsageGroup class that can implement nested 'inclusive' tests. Using this, the original example in this issue could be coded as 3 groups, 2 mutually_exclusive and inclusive one. parser = ArgumentParser(prog='PROG', formatter_class=UsageGroupHelpFormatter)
g1 = parser.add_usage_group(dest='FILE or DIR', kind='mxg', required=True)
a_file= g1.add_argument("-o", "--outfile", metavar='FILE')
g2 = g1.add_usage_group(dest='DIR and PS', kind='inc')
a_dir = g2.add_argument("-O", "--outdir", metavar='DIR')
g3 = g2.add_usage_group(dest='P or S', kind='mxg')
a_pat = g3.add_argument("-p", "--outpattern", metavar='PATTERN')
a_suf = g3.add_argument("-s", "--outsuffix", metavar='SUFFIX')
# usage: PROG [-h] (-o FILE | (-O DIR & (-p PATTERN | -s SUFFIX))) UsageGroup is like MutuallyExclusiveGroup, except that:
The attached 'usagegroup.py' file has the core code for this change. The full working code is at
It incorporates too many other changes (from other bug issues) to post |
http://stackoverflow.com/questions/25626109/python-argparse-conditionally-required-arguments asks about implementing a 'conditionally-required-arguments' case in if args.argument and (args.a is None or args.b is None):
# raise argparse error here I believe the clearest and shortest expression using Groups is: p = ArgumentParser(formatter_class=UsageGroupHelpFormatter)
g1 = p.add_usage_group(kind='nand', dest='nand1')
g1.add_argument('--arg', metavar='C')
g11 = g1.add_usage_group(kind='nand', dest='nand2')
g11.add_argument('-a')
g11.add_argument('-b') The usage is (using !() to mark a 'nand' test):
This uses a 'nand' group, with a 'not-all' test (False if all its actions are present, True otherwise). |
Attached is a patch for 3.5.0 dev that adds UsageGroups. Now different 'kinds' are implemented as subclasses, which are accessed via the registry:
Open issues:
|
So far I've proposed adding a 'hook' at the end of '_parse_known_args', that would give the user access to the 'seen_non_default_actions' variable. This function could perform an almost arbitrarily complex set of logical co-occurrence tests on this set (or list) of Actions. The rest of my proposed patches (nested groups, etc) are user interface components that attempt make this testing more user-friendly, both in specification and usage display. It just occurred to me that an alternate stop-gap fix is to make 'seen_non_default_actions' available to the user for his own testing after parsing. Adding it to the method return is not backward compatible. But it could be added as an attribute to parser. self._seen_actions = seen_non_default_actions It would be the first case of giving the parser a memory of past parsing actions, but I don't think that's a problem. Another possibility is to conditionally add it to the 'namespace'. if hasattr(namespace, 'seen_actions'):
setattr(namespace, 'seen_actions', seen_non_default_actions) The user could initial this attribute with a custom 'Namespace' object or with a 'set_defaults' call. (I'm proposing to save 'seen_non_default_actions' because in my earlier tests that seemed to be more useful than 'seen_actions'. It's the one used by mutually_exclusive_group testing.) |
Another issue requesting a 'mutually dependent group' |
Just voicing my support for this. I was also looking for a solution on StackOverflow and ended up here. |
+1 It would be nice to have such feature. |
Expressing support for this option. Would be a major improvement. |
I would like this as an option |
Would love to have this feature! |
+1 would have used now if available |
+1 useful to have |
+1 would be interested in using this! |
+1 would have used now if available |
I also got here by a post in Stackoverflow, when looking for this exact feature. Are there any plans to implement this? |
ditto (although the fact that this issue has been open since 2011 doesn't give me much hope ...) |
I would love to see this feature added! |
+1 would have used now if available |
+1! I also believe it would be great to have it! |
+1 |
1 similar comment
+1 |
+1 would be very useful now |
I would also appreciate this feature, doing it manually for now. |
Note: these values reflect the state of the issue at the time it was migrated and might not reflect the current state.
Show more details
GitHub fields:
bugs.python.org fields:
The text was updated successfully, but these errors were encountered: