Nested option groups with constraints

[havok2063]

• cloup version: 0.7 (or latest)
• Python version: 3.7
• Operating System: OS X

Description

I am trying to create a group of options with a seemingly complex, but perhaps not, constraint system. I basically have 5 parameters that I want to house in one group, that are a combination of required and mutually exclusive, and at least x. For example, options: A, B, C, D, and E. D and E are an "all_or_none" group. C and D+E are "mutually_exclusive" group, but at least one is required. B is required with one of (C,D+E). And either A or group (B + (C or D+E)) is required. Is this possible? It's not clear to me if nested option groups are possible, i.e. if I can create one option_group with a constraint and apply a constraint on top of it as part of another option_group.

I just discovered this code and am digging into the docs, so perhaps the answer is buried in there.

something like the following...

OptionGroup(
  A,
  OptionGroup(
    B, required=True,
    OptionGroup(
      C,
      OptionGroup(
        D,
        E
        constraint=all_or_none),
    constraint=mutually_exclusive
    ),
  constraint=RequireAtLeast(1)
  ),
constraint=RequireAtLeast(1)
)

or would I do something like

OptionGroup(
  A,
  B,
  C,
  D,
  E,
  constraint="some defined complex conditional constraint"
)

[janluke]

I just discovered this code and am digging into the docs, so perhaps the answer is buried in there.

You should read the docs before opening an issue. It's not nice to do otherwise. I'll answer you this time...

Option groups are primarily a way to organize your options in multiple help sections, not a way to define constraints. The constraint argument of @option_group is just a nice-to-have.

Option groups are intentionally not nestable since the resulting help text would be a mess.

You can define constraints on subsets of option groups using the @constraint decorator: https://cloup.readthedocs.io/en/stable/pages/constraints.html#usage-with-constraint

If you still want to describe the constraints defined on subsets of an option groups you can either do it "manually" passing the help parameter of @option_group or you can let Cloup generate a "Constraints" help section as explained in the link above (passing show_constraints=True to @command).

[janluke]

Constraints like A or (B and (C or D+E)) are not definable with a single constraint. This is a limitation. I'll think if I can do something about it. Meanwhile, I'd suggest to do "nested validation" inside the function rather than trying to accomplish the same thing with multiple constraints. For simple constraints like the first two, you can use @constraint instead.

[havok2063]

You should read the docs before opening an issue. It's not nice to do otherwise. I'll answer you this time...

That comment was only meant to convey I'm new to your package, not that I'm literally new to the docs at the moment I'm writing the issue. I've been reading your docs, along with https://github.com/click-contrib/click-option-group, for the past few hours, testing each package to see if I can get it working for what I need. Once it became clear that it wasn't clear from the docs, I filed the issue. Sorry about that, next time I'll wait even longer before asking for help.

Thanks for the clarification on Option groups that they cannot be nested. Although if you can combine constraints through operators and work out the help message, couldn't you do the same with multiple Option groups that each have a constraint?

You can define constraints on subsets of option groups using the @constraint decorator: https://cloup.readthedocs.io/en/stable/pages/constraints.html#usage-with-constraint

If you still want to describe the constraints defined on subsets of an option groups you can either do it "manually" passing the help parameter of @option_group or you can let Cloup generate a "Constraints" help section as explained in the link above (passing show_constraints=True to @command).

Constraints like A or (B and (C or D+E)) are not definable with a single constraint. This is a limitation. I'll think if I can do something about it. Meanwhile, I'd suggest to do "nested validation" inside the function rather than trying to accomplish the same thing with multiple constraints. For simple constraints like the first two, you can use @constraint instead.

Yeah maybe I can get something working with @constraints or calling them inside the function. I quickly tried

@cloup.constraint(require_all, ['D', 'E'])
@cloup.constraint(mutually_exclusive, ['C', 'D'])

but that didn't quite seem to work. It might be nice to be able to assign labels to a constraint that you can reference in other constraints, e.g.

@cloup.constraint(require_all, ['D', 'E'], reference='constraint_1')
@cloup.constraint(mutually_exclusive, ['C', 'constraint_1'], reference='constraint_2')
@cloup.constraint(require_all, ['B', 'constraint_2'], reference='constraint_3')
@cloup.constraint(RequireAtLeast(1), ['A', 'constraint_3'], reference='constraint_4')

I image it should be possible to compile each constraint down into a series of Predicates combined with logical operators.

I'll keep playing around with it to see if I can get something working.

[janluke]

Option groups are not going to be nestable, ever.

It might be nice to be able to assign labels to a constraint

I don't like it. Writing custom Python code is much better. I have something better in mind but it requires a complete redesign, I need to evaluate if it's doable and worth it.

Keep in mind that you can always validate your arguments inside your function with simple Python code. I'm not talking about calling the constraints, I mean simple Python code. You can raise click.UsageError when input data doesn't pass validation. Cloup constraints are just an experiment to save some typing in simple cases and automate the documentation. In complex cases, you often obtain better result with custom code and custom error messages. It's more typing, sure, but not so much.

Anyway, you can obtain something decent with the following:

@command(show_constraints=True)
@option_group(
    'Option group',                    
    option('-a', is_flag=True),       # using flags for easier testing
    option('-b', is_flag=True),
    option('-c', is_flag=True),
    option('-d', is_flag=True),
    option('-e', is_flag=True),
)
@constraint(all_or_none, ['d', 'e'])
@constraint(mutually_exclusive, ['c', 'd'])
@constraint(RequireExactly(1), ['a', 'b'])
@constraint(If('b', RequireExactly(1)), ['c', 'd'])
def cmd(a, b, c, d, e):
    pass

This generates the following help section:

Constraints:
  {-d, -e}  provide all or none
  {-c, -d}  mutually exclusive
  {-a, -b}  exactly 1 required
  {-c, -d}  exactly 1 required if -b is set

Error messages will probably be suboptimal but you can use constraint.rephrased(), I guess. But again, at that point, you are probably better just writing simple Python code inside the function.

[janluke]

Note that if you use a tuple option for D and E, you don't need the all_or_none constraint:

@option_group(
    'Option group',
    option('-a', is_flag=True),
    option('-b', is_flag=True),
    option('-c', is_flag=True),
    option('-d', nargs=2),
)
@constraint(mutually_exclusive, ['c', 'd'])
@constraint(RequireExactly(1), ['a', 'b'])
@constraint(If('b', RequireExactly(1)), ['c', 'd'])

[havok2063]

Yeah I agree for complex items it's probably easier to do it in the code. I do that now actually in my old code. I'm switching my code over from argparse to click and wanted to see what options were available for click for moving that logic into the cli. I am liking your code. It's more complete and flexible compared to the other package I found. It's a good idea as this functionality is sorely lacking in default click.

Thanks for the example. This goes a long way and gives me a good start. I'll play around with things a bit more. Thanks for your help! I appreciate it!

[janluke]

In Cloup v0.9.0, a limited form of nesting will be possible. Specifically, one can constrain one or multiple subgroups of an @option_group:

@option_group(
    'Number options', 
    RequireAtLeast(1)(
        option('--one'),
        option('--two')
    ),
    option('--three'),
)

Intentionally, only one level of nesting is allowed. Again, for very complex items, one should use custom code.

This change is part of a bigger feature (#8) which is about using constraints as decorators to avoid rewriting parameter names.

EDIT: I had forgotten the title in @option_group.

[havok2063]

This looks pretty promising and helps the readable of the relationship between options. I like it. So, e.g. does the equivalent of this code

num_group = OptionGroup('Number Options', help='blah')

@num_group.option('--one')
@num_group.option('--two')
@num_group.option('--three')
@cloup.constraint(RequireAtLeast(1), ['one', 'two'])
def hello(one, two, three):
    ...

turns into

num_group = OptionGroup('Number Options', help='blah')

@num_group(RequireAtLeast(1)(option('--one'), option('--two')), option('--three'))
def hello(one, two, three):
    ...

And in #8 you mention the incompatibility below Python 3.9. So for users of Python <3.8 they'll just need to pin cloup to <0.9?

[janluke]

@havok2063 No, since OptionGroup is not callable and can't be used as a decorator. The equivalent would be:

num_group = OptionGroup('Number Options', help='blah')

@RequireAtLeast(1)(
    num_group.option('--one'), 
    num_group.option('--two'),
)
@num_group.option('--three')
def hello(one, two, three):
    ...

I prefer the @option_group version I wrote in the previous message.

And in #8 you mention the incompatibility below Python 3.9. So for users of Python <3.8 they'll just need to pin cloup to <0.9?

No, Cloup will still be compatible with Python >= 3.6. It's just that in Python < 3.9 you can't use parametric/compound/conditional constraints with @ directly. You have to either do this:

require_any = RequireAtLeast(1)

@require_any(    # no more double call on the right of @
    option('--one'),
    ...
)

or this:

@constrained_params(
    RequireAtLeast(1),
    option('--one'),
    ...
)

Notice that there's not such a problem if you nest a parametric constraint inside @option_group (as in my previous comment) because in that case @ is not required.

[havok2063]

Ahh yeah that makes sense, and looks pretty readable to me. Oh yeah I didn't notice the group name in @option_group. That's a much nicer way of writing it. Good stuff! Thanks for the update.