Support for shared arguments?

[mahmoudimus]

What are your thoughts on shared argument support for click? Sometimes it is useful to have simple inheritance hierarchies in options for complex applications. This might be frowned upon since click's raison d'etre is dynamic subcommands, however, I wanted to explore possible solutions.

A very simple and trivial example is the verbose example. Assume you have more than one subcommand in a CLI app. An ideal user experience on the CLI would be:

python script.py subcmd -vvv

However, this wouldn't be the case with click, since subcmd doesn't define a verbose option. You'd have to invoke it as follows:

python script.py -vvv subcmd

This example is very simple, but when there are many subcommands, sometimes a root support option would go a long way to make something simple and easy to use. Let me know if you'd like further clarification.

[mitsuhiko]

This is already really simple to implement through decorators. As an example:

import click

class State(object):

    def __init__(self):
        self.verbosity = 0
        self.debug = False

pass_state = click.make_pass_decorator(State, ensure=True)

def verbosity_option(f):
    def callback(ctx, param, value):
        state = ctx.ensure_object(State)
        state.verbosity = value
        return value
    return click.option('-v', '--verbose', count=True,
                        expose_value=False,
                        help='Enables verbosity.',
                        callback=callback)(f)

def debug_option(f):
    def callback(ctx, param, value):
        state = ctx.ensure_object(State)
        state.debug = value
        return value
    return click.option('--debug/--no-debug',
                        expose_value=False,
                        help='Enables or disables debug mode.',
                        callback=callback)(f)

def common_options(f):
    f = verbosity_option(f)
    f = debug_option(f)
    return f

@click.group()
def cli():
    pass

@cli.command()
@common_options
@pass_state
def cmd1(state):
    click.echo('Verbosity: %s' % state.verbosity)
    click.echo('Debug: %s' % state.debug)

[mahmoudimus]

@mitsuhiko yep, I built something similar as well.

From your example, is the intention that users build their own common options and annotate all relevant commands?

Maybe a better user experience is to register these common options to the group and have the group transitively propagate them to its subcommands? I could see arguments for either or.

[mitsuhiko]

If the option is available on all commands it really does not belong to the option but instead to the group that encloses it. It makes no sense that an option conceptually belongs to the group but is attached to an option in my mind.

[mahmoudimus]

Right, and I'm on board with your logic. However, this translates to position dependence for options which causes cognitive load on the user of the cli app, unless the approach above is used to get, what I would consider, the desirable and expected UX.

For example, assume a CLI exists to manage some queues:

python script.py queues {create,delete,sync,list}

Let's say I want to enable logging for this, a natural inclination is:

python script.py queues create --log-level=debug

However, if the option belongs to the group, the user is forced to configure the command as they write it:

python script.py queues --log-level=debug create

That's why I'm wondering if it makes sense to have a parameter on the group class which propagates options down to commands to get the desired behavior without surprising the user.

If my use case is the outlier in terms of what is desired and expected, then I guess the option of using a similar idiom to what you've demonstrated above is the right way to go.

[jkeifer]

I have to agree with mahmoudimus on this. I understand the logic that the option does not belong to the command if it is available on all commands, however, the positional dependence I too see to be a problem.

I have a similar circumstance to the one above where I want to have an option to enable logging for all commands, which would only make sense to be an option at the group level. Yet, I can imagine two scenarios: (1) users don't understand the group->command hierarchy, and therefore do not understand they need to place that option switch after the group but before the command, or (2) that users will have spent a significant amount of time trying to get all the arguments filled in for the command (my commands have MANY options and arguments), only to realize they want to have logging, and then they will have to move their cursor way back in the command to get it into a position where it will actually work. I believe figuring out a way to allow group options without regard to position is key for a better user experience.

[mitsuhiko]

Doing this by magic will not happen, that stands against one of the core principles of Click which is to never break backwards compatibility with scripts by adding new parameters/options later. The correct solution is to use a custom decorator for this. :)

[untitaker]

How about adding such a decorator to click or a click-contrib package?

[apollux]

I have a similar, but yet slightly different use case. In my situation I want to create a group with 3 sub commands. Two of these commands can need the same password for a remote the server. The third command invokes a local operation.

From a user perspective any combination of these commands can make sense. I would like to be able to only ask for the password if necessary and only ask it once. So the password option is only shared between two of three commands.

Is it possible to implement this with a similar solution using decorators?

[Stiivi]

I'm giving my vote for this feature, as it makes the CLI experience less complex. Even though technically cmd -a subcmd -b subsubcmd -c is correct, cmd subcmd subsubcmd -a -b -c is analogous to have cmd_subcmd_subsubcmd -a -b -c.

Think of PostgreSQL tools pg_dump, pg_restore and pg_ctl which might be implemented using Click as a single command pg with subcommands dump, restore and ctl and they both share options such as --host, --port or -D which come after the whole logical command.

It would be nice if there was a Group argument tail_options = True, share_options or something similar, meaning that all the options will be put at the "tail" of the command chain:

@click.group(tail_options=True)
@click.pass_context
@click.option('--host', nargs=1)
@click.option('--port', nargs=1)
def pg(ctx, host, port):
    # initialize context here

@pg.command()
@click.pass_context
def dump(ctx, ...):
    # ...

@pg.command()
@click.pass_context
def restore(ctx, ...):
    # ...

Another problem with current behavior is that pg restore --help will not display the pg options, which might be confusing to the users. In this case, user will see all the options, including the shared ones, available to the pg restore.

Caveat: group defines namespace and in this case all options would share a single "command tail" namespace. There should be either a well described behavior in a case of two options (such as "the latest definition overrides the previous one") with the same name or to raise an error.

[stephenmm]

The noted resolution is to complex for such a common/feature/request.

[untitaker]

Then please create a separate package that contains this functionality. This is possible in a clean way AFAIK

[mikenerone]

I think this can be done even more trivially than the given example. Here's a snippet from a helper command I have for running unit tests and/or coverage. Note that several of the options are shared between the test and cover subcommands:

_global_test_options = [
    click.option('--verbose', '-v', 'verbosity', flag_value=2, default=1, help='Verbose output'),
    click.option('--quiet', '-q', 'verbosity', flag_value=0, help='Minimal output'),
    click.option('--fail-fast', '--failfast', '-f', 'fail_fast', is_flag=True, default=False, help='Stop on failure'),
]

def global_test_options(func):
    for option in reversed(_global_test_options):
        func = option(func)
    return func

@click.command()
@global_test_options
@click.option('--start-directory', '-s', default='test', help='Directory (or module path) to start discovery ("test" default)')
def test(verbosity, fail_fast, start_directory):
    # Run tests here

@click.command()
@click.option(
    '--format', '-f', type=click.Choice(['html', 'xml', 'text']), default='html', show_default=True,
    help='Coverage report output format',
)
@global_test_options
@click.pass_context
def cover(ctx, format, verbosity, fail_fast):
    # Setup coverage, ctx.invoke() the test command above, generate report

[stevekuznetsov]

@mikenerone very cool example! I just wanted to post a little bit more complex code that I wrote today since it took a little bit of work to get everything straightened out -- decorators can get a little confusing!

In my code, I had two requirements that were unique from the previous examples:

• I wanted to share a set of options that were tightly coupled, but not identical. Specifically, I wanted to parameterize the help-text to make it local to the CLI endpoint
• I wanted to have another CLI endpoint that shared one of the options included in the above set

In order to accomplish this, I had to use decorators with parameters to achieve the first point, and break out the option decorator construction and parameterization for the second point.

In the end, my code looks something like this:

import click

def raw_shared_option(help, callback):
    """
    Get an eager shared option.

    :param help: the helptext for the option
    :param callback: the callback for the option
    :return: the option
    """
    return click.option(
        '--flagname',
        type=click.Choice([
            an_option,
            another_option,
            the_last_option
        ]),
        help=help,
        callback=callback,
        is_eager=True
    )


def shared_option(help, callback):
    """
    Get a decorator for an eager shared option.

    :param help: the helptext for the option
    :param callback: the callback for the option
    :return: the option decorator
    """

    def shared_option_decorator(func):
        return raw_shared_option(help, callback)(func)

    return shared_option_decorator


def coupled_options(helptext_param, eager_callback):
    """
    Get a decorator for coupled options.

    :param helptext_param: a parameter for the helptext
    :param eager_callback: the callback for the eager option
    :return: the decorator for the coupled options
    """

    def coupled_options_decorator(func):
        options = [
            click.option(
                '--something',
                help='Helptext for something ' + helptext_param + '.'
            ),
            raw_shared_option(
                help='Helptext for eager option' + helptext_param + '.',
                callback=eager_callback
            )
        ]

        for option in reversed(options):
            func = option(func)

        return func

    return coupled_options_decorator

@click.group()
def groupname:
    pass

def eager_option_callback(ctx, param, value):
    """
    Handles the eager option.
    """
    if not value or ctx.resilient_parsing:
        return

    click.echo(value)
    ctx.exit()

@groupname.command()
@coupled_options('some parameter', eager_option_callback)
def command_with_coupled_options(something, flagname):
    pass


def different_eager_option_callback(ctx, param, value):
    """
    Handles the eager option for other command.
    """
    if not value or ctx.resilient_parsing:
        return

    click.echo(value)
    ctx.exit()


@groupname.command()
@coupled_options('some different parameter', different_eager_option_callback)
def other_command_with_coupled_options(something, flagname):
    pass

@groupname.command()
@shared_option('simple parameter', eager_option_callback)
def command_with_only_shared_command(flagname):
    pass

Hopefully someone is helped by this! It definitely is nice to have shared options for commands that do similar things, without the positional problems that people have mentioned before.

[jerowe]

@mikenerone, when I used your example as is I wasn't able to generate any help. I expanded on the repo example and got it going.

import click
import os
import sys
import posixpath

_global_test_options = [
    click.option('--force_rebuild', '-f',  is_flag = True, default=False, help='Force rebuild'),
]

def global_test_options(func):
    for option in reversed(_global_test_options):
        func = option(func)
    return func

class Repo(object):

    def __init__(self, home):
        self.home = home
        self.config = {}
        self.verbose = False

    def set_config(self, key, value):
        self.config[key] = value
        if self.verbose:
            click.echo('  config[%s] = %s' % (key, value), file=sys.stderr)

    def __repr__(self):
        return '<Repo %r>' % self.home


pass_repo = click.make_pass_decorator(Repo)


@click.group()
@click.option('--repo-home', envvar='REPO_HOME', default='.repo',
              metavar='PATH', help='Changes the repository folder location.')
@click.option('--config', nargs=2, multiple=True,
              metavar='KEY VALUE', help='Overrides a config key/value pair.')
@click.option('--verbose', '-v', is_flag=True,
              help='Enables verbose mode.')
@click.version_option('1.0')
@click.pass_context
def cli(ctx, repo_home, config, verbose):
    """Repo is a command line tool that showcases how to build complex
    command line interfaces with Click.

    This tool is supposed to look like a distributed version control
    system to show how something like this can be structured.
    """
    # Create a repo object and remember it as as the context object.  From
    # this point onwards other commands can refer to it by using the
    # @pass_repo decorator.
    ctx.obj = Repo(os.path.abspath(repo_home))
    ctx.obj.verbose = verbose
    for key, value in config:
        ctx.obj.set_config(key, value)

@cli.command()
@pass_repo
@global_test_options
def clone(repo, force_rebuild):
    """Clones a repository.

    This will clone the repository at SRC into the folder DEST.  If DEST
    is not provided this will automatically use the last path component
    of SRC and create that folder.
    """
    click.echo("Force rebuild is {}".format(force_rebuild))

Thanks so much for pointing me in the right direction! Something about the

command --opts subcommand

Syntax was just really bugging me. ;-)

Also, I am quite new to click and to the decorators and callbacks in python. Would someone mind explaining what

def global_test_options(func):
    for option in reversed(_global_test_options):
        func = option(func)
    return func

this is doing? Or point me towards some resources?

[mikenerone]

@jerowe When you apply the functions as decorators, the Python interpreter applies them in reverse order (because its working from the inside out). Click's design takes the into account so that the order in which help is generated is the same as the order of your decorators. In the case of the global_test_options decorator, I'm just doing the same thing Python does and applying them in reverse order. It's purely a dev convenience that allows you to order your entries in _global_test_options in the same order you want help generated.

[jerowe]

@mikenerone, thanks for explaining. I still need to look more into decorators. They seem very cool, but I just don't have my head wrapped around them. Learning click has been good for that.

I'm glad there is such a good command line application framework in python. I've been hesitant to switch for awhile because I love the framework I use in perl, but I think I will be quite happy with click. ;-)

[gorgitko]

@mikenerone Thanks for this simple solution! I have slightly edited your function to allow to pass options list in parameter:

import click

cmd1_options = [
    click.option('--cmd1-opt1'),
    click.option('--cmd1-opt2')
]

def add_options(options):
    def _add_options(func):
        for option in reversed(options):
            func = option(func)
        return func
    return _add_options

@click.group()
def main(**kwargs):
    pass

@main.command()
@add_options(cmd1_options)
def cmd1(**kwargs):
    print(kwargs)

if __name__ == '__main__':
    main()

[p7k]

it's also possible to do this using another decorator, albeit with some boilerplate function wrapping. nice if you only need one such common parameter group.

def common_params(func):
    @click.option('--foo')
    @click.option('--bar')
    @functools.wraps(func)
    def wrapper(*args, **kwargs):
        return func(*args, **kwargs)
    return wrapper


@click.command()
@common_params
@click.option('--baz')
def cli(foo, bar, baz):
    print(foo, bar, baz)

[MinchinWeb]

@p7k : thank you for your example! To get it to work, I had to use @functools.wraps() as '@itertools.wraps()` doesn't seem to exist.

[gronka]

edit: removing my recommendation since I was probably importing an import

Anyways, @p7k thanks from here as well!

[p7k]

whoops - i definitely meant functools - i've edited my comment.

[NiklasRosenstein]

The problem with all these solutions is that the options are passed to the sub-command as well. I would like the Group to handle these options, but still have them accessible when specified after the sub-command. Something like this would be very handy:

@click.group()
@click.option('-v', count=True, shared=True)
def main(v):
  ... # do something with v

@main.command()
def subc():
  ... # subcommand does not receive v, main already handled it

Also so far, all suggested solutions where the options are simply added to all subcommands are very convoluted and clutter the code. There could still be an option so that the argument/option is received in subcommands, if one so desires.

Edit: For the sake of completeness, the above should work as main -v subc and main subc -v.

[mikenerone]

@NiklasRosenstein That's not a problem with "these solutions". The recent solutions you're referring to are just a shorthand for normal click decorators. You just don't like the way click works regarding the "scoping" of parameters such that they have to come after the command that specifies them but before any of its subcommands. It's a valid opinion - I've been annoyed by the same thing at times (and I mean with direct click decorators that don't employ any of the tricks proposed here). On the other hand, I can certainly understand the motivation behind click's design - it eliminates some potential ambiguity. I find myself just about in the middle, which doesn't justify advocating for a change in behavior.

[183amir]

This might have been bikeshedding in 2014 but I think it is an important enough issue now. Also none of the workarounds posted here really work as explained in #108 (comment)
I encourage the click developers to take a look at this again.

[davidism]

This is a deliberate design decision in Click.