Documentation doesn't describe "lazy loading of subcommands at runtime" #945
Labels
Comments
ncraike
changed the title
|
I think so. Another way do do lazy loading would be something like this: https://github.com/indico/indico/blob/master/indico/cli/util.py#L95 / https://github.com/indico/indico/blob/master/indico/cli/core.py#L68 |
2 tasks
|
Yes this would be a good thing. |
|
I also found this quite misleading, can this third point be removed from the docs? There doesn't seem to be anything in Click that does this out of the box |
|
I've implemented lazy sub-commands in my way, FYI. import sys
from importlib import import_module
from my_project.db import alembic
import click
def import_from_string(spec):
"""
Thanks to https://github.com/encode/django-rest-framework/blob/master/rest_framework/settings.py#L170
Example:
import_from_string('django_filters.rest_framework.DjangoFilterBackend')
engine = conf['ENGINE']
engine = import_from_string(engine) if isinstance(engine, six.string_types) else engine
"""
try:
# Nod to tastypie's use of importlib.
parts = spec.split('.')
module_path, class_name = '.'.join(parts[:-1]), parts[-1]
module = import_module(module_path)
return getattr(module, class_name)
except (ImportError, AttributeError) as e:
msg = "Could not import '%s'. %s: %s." % (spec, e.__class__.__name__, e)
raise ImportError(msg)
@click.group()
def manager():
"""MyProject Management Tools."""
def get_db_cmd():
db_cmd = alembic.get_click_cli("db")
db_cmd.help = 'Design database schema with confidence.'
return db_cmd
cmd_factories = dict((
('db', get_db_cmd),
('dev', lambda: import_from_string('my_project.commands.dev.dev')),
('shell', lambda: import_from_string('my_project.commands.shell.shell')),
('psql', lambda: import_from_string('my_project.commands.psql.psql')),
('protoc', lambda: import_from_string('my_project.commands.protoc.protoc')),
('cache', lambda: import_from_string('my_project.commands.cache.cache')),
('test', lambda: import_from_string('my_project.commands.testing.test')),
...
))
if __name__ == '__main__':
if len(sys.argv) > 1 and (cmd_name := sys.argv[1]) in cmd_factories:
# Construct sub-command only as needed.
manager.add_command(cmd_factories[cmd_name](), name=cmd_name)
else:
# For user can list all sub-commands.
for cmd_name, fct in cmd_factories.items():
manager.add_command(fct(), name=cmd_name)
manager() |
sirosen
added a commit
to sirosen/click
that referenced
this issue
Sep 1, 2022
Resolves pallets#945 The goal of this example is to show how one might subclass Group, overriding `get_command` and `list_commands` with some importlib usage and a dict, and produce a viable lazy-loading solution. One significant benefit of offering an example is that it provides space to explain how and when `get_command` is invoked. i.e. Users might be surprised to realize that subcommands are resolved during completion scenarios. At the risk of diverging a small amount into what some would consider "general deferred import semantics in python", the section closes out with an example of a `click.command` decorated function which contains a deferred import. For codebases which have significant at-import-time work (e.g. importing `requests` or `urllib3`), this strategy is probably *more* useful than lazy loading of subcommands. Other variations on this same solution are possible, e.g. using `importlib.load_module(module_name, self.callback.__module__)` to handle imports relative to the definition of the callback function. However, any further work in this space is left as an exercise to readers of the doc. Importantly, unlike some solutions already discussed in pallets#945, nothing about this is application-specific. The example LazyGroup implementation does not encode any knowledge about the structure of the codebase in which it is used, which means that users can copy-paste the example and expect it to work (within reason). Because this also introduces the risk of reckless copy-paste by novice users, a warning is included in the doc that is meant to point at a certain level of application maturity which should be reached before this strategy can be safely applied.
sirosen
added a commit
to sirosen/click
that referenced
this issue
Sep 1, 2022
Resolves pallets#945 The goal of this example is to show how one might subclass Group, overriding `get_command` and `list_commands` with some importlib usage and a dict, and produce a viable lazy-loading solution. One significant benefit of offering an example is that it provides space to explain how and when `get_command` is invoked. i.e. Users might be surprised to realize that subcommands are resolved during completion scenarios. At the risk of diverging a small amount into what some would consider "general deferred import semantics in python", the section closes out with an example of a `click.command` decorated function which contains a deferred import. For codebases which have significant at-import-time work (e.g. importing `requests` or `urllib3`), this strategy is probably *more* useful than lazy loading of subcommands. Other variations on this same solution are possible, e.g. using `importlib.load_module(module_name, self.callback.__module__)` to handle imports relative to the definition of the callback function. However, any further work in this space is left as an exercise to readers of the doc. Importantly, unlike some solutions already discussed in pallets#945, nothing about this is application-specific. The example LazyGroup implementation does not encode any knowledge about the structure of the codebase in which it is used, which means that users can copy-paste the example and expect it to work (within reason). Because this also introduces the risk of reckless copy-paste by novice users, a warning is included in the doc that is meant to point at a certain level of application maturity which should be reached before this strategy can be safely applied.
sirosen
added a commit
to sirosen/click
that referenced
this issue
Sep 1, 2022
Resolves pallets#945 The goal of this example is to show how one might subclass Group, overriding `get_command` and `list_commands` with some importlib usage and a dict, and produce a viable lazy-loading solution. One significant benefit of offering an example is that it provides space to explain how and when `get_command` is invoked. i.e. Users might be surprised to realize that subcommands are resolved during completion scenarios. At the risk of diverging a small amount into what some would consider "general deferred import semantics in python", the section closes out with an example of a `click.command` decorated function which contains a deferred import. For codebases which have significant at-import-time work (e.g. importing `requests` or `urllib3`), this strategy is probably *more* useful than lazy loading of subcommands. Other variations on this same solution are possible, e.g. using `importlib.load_module(module_name, self.callback.__module__)` to handle imports relative to the definition of the callback function. However, any further work in this space is left as an exercise to readers of the doc. Importantly, unlike some solutions already discussed in pallets#945, nothing about this is application-specific. The example LazyGroup implementation does not encode any knowledge about the structure of the codebase in which it is used, which means that users can copy-paste the example and expect it to work (within reason). Because this also introduces the risk of reckless copy-paste by novice users, a warning is included in the doc that is meant to point at a certain level of application maturity which should be reached before this strategy can be safely applied.
davidism
pushed a commit
to sirosen/click
that referenced
this issue
Jun 30, 2023
Resolves pallets#945 The goal of this example is to show how one might subclass Group, overriding `get_command` and `list_commands` with some importlib usage and a dict, and produce a viable lazy-loading solution. One significant benefit of offering an example is that it provides space to explain how and when `get_command` is invoked. i.e. Users might be surprised to realize that subcommands are resolved during completion scenarios. At the risk of diverging a small amount into what some would consider "general deferred import semantics in python", the section closes out with an example of a `click.command` decorated function which contains a deferred import. For codebases which have significant at-import-time work (e.g. importing `requests` or `urllib3`), this strategy is probably *more* useful than lazy loading of subcommands. Other variations on this same solution are possible, e.g. using `importlib.load_module(module_name, self.callback.__module__)` to handle imports relative to the definition of the callback function. However, any further work in this space is left as an exercise to readers of the doc. Importantly, unlike some solutions already discussed in pallets#945, nothing about this is application-specific. The example LazyGroup implementation does not encode any knowledge about the structure of the codebase in which it is used, which means that users can copy-paste the example and expect it to work (within reason). Because this also introduces the risk of reckless copy-paste by novice users, a warning is included in the doc that is meant to point at a certain level of application maturity which should be reached before this strategy can be safely applied.
|
Fixed by #2348, can be found at https://click.palletsprojects.com/complex/#lazily-loading-subcommands |
Sign up for free
to subscribe to this conversation on GitHub.
Already have an account?
Sign in.
The documentation home page includes:
How is "lazy-loading of subcommands at runtime" supported? The various methods of creating command groups and attaching sub-commands described in the quickstart guide all require importing the sub-command first.
The closest I can find is a demonstration of how lazy-loading of commands from plugins might be implemented with a custom multi-command class. Is this the "lazy loading of subcommands at runtime" which the documentation describes?
The text was updated successfully, but these errors were encountered: