Adding group to CommandCollection loses group options

[rcoup]

Goal:

• lots of commands that require and use a config file.
• init command that creates a valid config file.

Attempt

import os
import click

@click.group()
def cli_noconfig():
  # need this because you can't add Commands to a CommandCollection, only MultiCommands
  pass

@cli_noconfig.command("init")
@click.option("--config", type=click.Path(exists=False), default=os.path.join(click.get_app_dir('foo', force_posix=True), "foo.conf"))
def init(config):
  # creates the config file
  print "init! creating config: %s" % config

@click.group()
@click.option("--config", type=click.File('r'), default=os.path.join(click.get_app_dir('foo', force_posix=True), "foo.conf"))
@click.pass_context
def cli_main(ctx, config):
  print "main! config=%s" % config
  ctx.obj['config'] = config

@cli_main.command('cmd1')
@click.pass_context
def cmd1(ctx):
  print "cmd1! config=%s" % ctx.obj.get('config')

# ... lots more commands that use & require the config file

# promote both groups to the "top" level
cli = click.CommandCollection(sources=[cli_noconfig, cli_main])

if __name__ == '__main__':
  cli(obj={})

Problem

The file parameter (and any other parameters/options on the cli_main group) get dropped/ignored when the commands are accessed via the CommandCollection.

Not sure if this is a bug (it's certainly unobvious), and I'm curious as to what other approaches could solve my problem? Looks like I can't override a group parameter for a specific command either, right? And validation callbacks don't appear to get passed context or anything else I can use to figure out which command is being run...

[cluelessperson]

Same here, completely broken.
Now I have to spell it out in every, single, command.

import click

@click.group()
def testcli():
    print("test")

@testcli.command()
def test():
    print("test2")

cli = click.CommandCollection(sources=[testcli])
if __name__ == '__main__':
    cli()

the above code ONLY outputs "test"

[ziazon]

the two referenced issues don't help with the above situation where the options are on one of the command groups being specified in the collection. For example:

@click.group()
@click.option(
    '--profile',
    required=True,
    type=str,
)
@click.pass_context
def with_profile(ctx, profile):
    ctx.obj = {"profile": profile}

@click.group()
def without_profile():
    pass

@click.command(cls=click.CommandCollection, sources=[with_profile, without_profile])
def cli():
    pass

 if __name__ == '__main__':
    cli()

Doesn't work.

Neither does doing cli = click.CommandCollection(sources=[with_profile, without_profile])

[schollii]

I don't know if I'm rationalizing but to me this is actually expected behaviour: you are taking commands from one or more groups, and making the cli group pretend that they are in its group, thus the original groups are no longer in the picture... so why would their options still be used? You could say that the options should be used but that would mean the original groups are still present which is not the case I think, as it would imply each command of the original groups are getting a second parent.

Is it not possible to put those options on the command collection itself? If not, or if not all groups added to the command collection use the special options, you could also just refactor the group option code into your own decorator like @config_option, and command that has this decorator will support that option.

[jcrotts]

I'm a little unclear on the exact use case that this bug pertains to. I think @scholli has some good recommendations for anyone hitting this issue. Also a similar issue with a workaround #1179 .

Was this previously possible in click at some point?

[ghost]

@schollii It depends a bit on how you frame the usecase of the command collection. I think of it as a way to merge two groups. You could put the two groups side by side as sub commands, but maybe for usability reasons you'd rather have all the commands at the same level, so you merge the groups instead.
If you are thinking of it as merging groups then it's quite unobvious that the other features of the groups get lost in the process.
For me the ideal behaviour would be something like "behaves exactly like the sub command approach except without the sub commands"

[BNMetrics]

I have found a workaround for this, you will need to extend click.CommandCollection, like so:

class CommandGroupCollection(click.CommandCollection):
    @property
    def sources_map(self) -> dict:
        """
        A dictionary representation of {command_name: source_group_object}
        """
        r = {}
        for source in self.sources:
            if not isinstance(source, click.Group):
                continue
            for command in source.commands:
                r[command] = source

        return r

    def invoke(self, ctx):
        if ctx.protected_args:
            group = self.sources_map.get(ctx.protected_args[0])
            if group:
                group.invoke(ctx)
        else:
            super().invoke(ctx)

[shmolyneaux]

I hit this issue today. Mentally, click.CommandCollection seems like it merges groups of commands. It's surprising that the callback of each group is lost in this process. This is with click 7.1.2.

In my case, the purpose is to merge built-in commands with custom subcommands (similar to git). All the built-in commands need the same setup.