register_commands() plugin hook to register extra CLI commands

[simonw]

The datasette CLI tool currently has 7 subcommands: serve, inspect, install, package, plugins, publish and uninstall.

A plugin hook could allow plugins to register extra subcommands.

I've avoided this for quite a while because I didn't have good use-cases for it - but the existence of the datasette install xxx command for installing packages into the correct virtual environment means that actually there's a good reason to do this: it would allow plugins to provide additional command-line mechanisms without the user having to understand how virtual environments work in order to install those commands into the same environment as the rest of Datasette.

[simonw]

The closest plugin hook to this right now is publish_subcommand - which looks like this:

@hookimpl
def publish_subcommand(publish):
    @publish.command()
    @add_common_publish_arguments_and_options
    @click.option(
        "-k",
        "--api_key",
        help="API key for talking to my hosting provider",
    )
    def my_hosting_provider(...):

But there are also several plugin hooks with register_ prefixes, which may be a good naming convention to stick to here: register_output_renderer(datasette), register_routes(datasette), register_facet_classes(), register_magic_parameters(datasette).

[simonw]

There's also the option for plugins to muck around with existing registered commands - this could get a bit untidy if multiple plugins try to do it, but being able to replace serve with a fresh implementation that adds an additional command-line option before calling back to the original might open up some interesting possibilities.

[simonw]

Terminology question: is it correct to call these subcommands or should they be commands? publish_subcommand() adds subcommands of the format datasette publish X - but are we instead adding commands with this new one?

[simonw]

Here's the answer to that:

~ % datasette --help
Usage: datasette [OPTIONS] COMMAND [ARGS]...

  Datasette!

Options:
  --version  Show the version and exit.
  --help     Show this message and exit.

Commands:
  serve*     Serve up specified SQLite database files with a web UI
  inspect
  install    Install Python packages - e.g.
  package    Package specified SQLite files into a new datasette Docker...
  plugins    List currently available plugins
  publish    Publish specified SQLite database files to the internet...
  uninstall  Uninstall Python packages (e.g.

Since it's adding extra things that show up in --help under the "Commands:" heading, I should call them commands.

[simonw]

I'm going to call the new hook register_commands - since it will allow ambitious plugins to register more than one command if they want to. That's also pleasingly similar to register_routes.

[simonw]

The first example plugin I'm going to build for this will be datasette verify file.db file2.db - it will take one or more paths to SQLite files and verify if they can be opened by Datasette or not.

[simonw]

Considering this piece of code:

datasette/datasette/cli.py

Lines 122 to 142 in a1a33bb

	@click.group(cls=DefaultGroup, default="serve", default_if_no_args=True)
	@click.version_option(version=__version__)
	def cli():
	"""
	Datasette!
	"""
	@cli.command()
	@click.argument("files", type=click.Path(exists=True), nargs=-1)
	@click.option("--inspect-file", default="-")
	@sqlite_extensions
	def inspect(files, inspect_file, sqlite_extensions):
	app = Datasette([], immutables=files, sqlite_extensions=sqlite_extensions)
	loop = asyncio.get_event_loop()
	inspect_data = loop.run_until_complete(inspect_(files, sqlite_extensions))
	if inspect_file == "-":
	sys.stdout.write(json.dumps(inspect_data, indent=2))
	else:
	with open(inspect_file, "w") as fp:
	fp.write(json.dumps(inspect_data, indent=2))

I think the hook itself gets called with a single argument, cli, which it can then use in the standard Click way to register extra stuff.

I can't pass it datasette (like I do with register_routes()) because the Datasette object itself is instantiated by the serve command, which will not have been called.

[simonw]

I don't think I can get this new hook to support the handy --plugins-dir= mechanism for loading plugins from Python files as opposed to registering them with setuptools - that mechanism is itself implemented inside of code called by datasette serve so I don't have a way of taking advantage of it from outside that command.

[simonw]

I'll probably have to use this mechanism for the tests then: https://til.simonwillison.net/pytest/registering-plugins-in-tests

[simonw]

Partial prototype of datasette-verify:

from datasette import hookimpl
import click


@hookimpl
def register_commands(cli):
    from datasette.cli import sqlite_extensions
    @cli.command()
    @click.argument("files", type=click.Path(exists=True), nargs=-1)
    @sqlite_extensions
    def verify(files, sqlite_extensions):
        "Verify that files can be opened by Datasette"
        for file in files:
            print(file)

I had to move the from datasette.cli import sqlite_extensions inside the hook function to avoid a circular import.

[simonw]

Writing the test for this is proving difficult, because the cli module has already been imported when I attempt to register a new plugin - so it doesn't pick up on the additional command registrations.

Trying to work around that with importlib.reload(cli).

[simonw]

Documentation: https://docs.datasette.io/en/latest/plugin_hooks.html#register-commands-cli

[simonw]

I need to push this out as an alpha so I can release a demo plugin that uses it.