Support Argument groups

[aarcangeli]

Hi, it is possible to use argument groups in argparse?

This is useful when you want to group arguments together.
For example, you can group arguments by their functionality.

import argparse

parser = argparse.ArgumentParser()

group = parser.add_argument_group('Training options')
group.add_argument('--epochs', type=int, default=10, help='Number of epochs')
group.add_argument('--batch_size', type=int, default=32, help='Batch size')

group = parser.add_argument_group('Optimizer options')
group.add_argument('--lr', type=float, default=0.01, help='Learning rate')

args = parser.parse_args()

usage: train.py [-h] [--epochs EPOCHS] [--batch_size BATCH_SIZE] [--lr LR]
                                                                          
options:                                                                  
  -h, --help            show this help message and exit

Training options:
  --epochs EPOCHS       Number of epochs
  --batch_size BATCH_SIZE
                        Batch size

Optimizer options:
  --lr LR               Learning rate

Docs: https://docs.python.org/3/library/argparse.html#argument-groups

[mivade]

My initial thought on how to implement something like this would be to include a group field in the field metadata.

Pull requests are welcome!

[vsoch]

We are interested in this use case too - specifically being able to add a group (derived from dataclass) onto a traditional ArgumentParser. The use case is that we have a main library that supports plugins, and the plugins can add a group (for their special case) to the argument parser. Right now we are doing a set of args verbatim, but this is problematic in that we have to carry around the args throughout the various functions to get to the plugin logic. The issue is that it wouldn't work well for strictly Python use cases. A solution with a dataclass would be much more ideal!

[vsoch]

In case this is helpful to someone else, the use case we have is wanting to be able to have this custom plugin add arguments on the fly. Our original strategy was using a custom group, but then we had this group that needed to be passed via the argparse.Namespace, which is not ideal if someone wants to use your classes external to an argparser. What we would up doing is:

1. Allow the custom plugin to define a vanilla dataclasses.dataclass, and with arguments that have a prefix specific to the plugin (to namespace them in the argument space)
2. We can then use the (private) function here _add_dataclass_options(ExecutorParameters, parser) that will ensure our custom dataclass options are added (and this is why the namespacing is important, otherwise you'd get a conflicting name error)
3. Once we get the args, we allow the plugin to be handed back the args, either to customize any directly, but most importantly, to derive the dataclass back from them.
4. And then we are able to pass forward the dataclass to the plugin instead of the args object!

We have an early writeup here: https://github.com/snakemake/snakemake-executor-flux#how-does-it-work. @mivade I am wondering why that function is considered private, and if a public variant (for this use case of quickly modifying an argument parser) might be exposed?

[mivade]

Closed by #57.