[QUESTION] How to avoid explicit call to subcommands command

[Andrew-Sheridan]

First check

• I used the GitHub search to find a similar issue and didn't find it.
• I searched the Typer documentation, with the integrated search.
• I already searched in Google "How to X in Typer" and didn't find any information.
• I already searched in Google "How to X in Click" and didn't find any information.

Description

I'm not sure how to articulate this question without an example. Because of that I had some trouble googling around.

I have two installed packages, foo and bar.

• bar can be used independently of foo. Ex: bar PATH # bar does stuff with PATH
• foo accepts bar as a plugin. Now foo can use the logic from bar. Ex: foo bar PATH. # foo does bar stuff with PATH
• foo can also do other stuff that is built into itself. Ex: foo fizz PATH. # foo does fizz stuff with PATH
• I have this working with argparse but can't get foo bar PATH working quite right with typer

So here goes with an example.

In one file bar/src/bar/cli.py I have

import typer

app = typer.Typer()


@app.command()
def main(path: str = typer.Argument(...)):
    typer.echo(f"Doing bar things to {path}")

If I call bar --help I get

Usage: bar [OPTIONS] PATH

Options:
  --install-completion [bash|zsh|fish|powershell|pwsh]
                                  Install completion for the specified shell.
  --show-completion [bash|zsh|fish|powershell|pwsh]
                                  Show completion for the specified shell, to
                                  copy it or customize the installation.

  --help                          Show this message and exit.

which is great.

This is installed with this setup.py from bar/setup.py

"""Setup."""

from setuptools import find_packages
from setuptools import setup

PACKAGE_NAME = "bar"

setup(
    name=PACKAGE_NAME,
    version="0.0.1",
    package_dir={"": "src"},
    packages=find_packages(where="src"),
    zip_safe=False,
    include_package_data=True,
    python_requires=">=3.7",
    entry_points={
        "console_scripts": [f"{PACKAGE_NAME} = {PACKAGE_NAME}.cli:app"],
        "foo.cli_plugins": [f"{PACKAGE_NAME} = {PACKAGE_NAME}.cli"],
    },
    install_requires=["typer"],
)

• there is also an empty __init__.py in bar/src/bar/

In another file foo/src/foo/cli.py I have

import typer
import pkg_resources

app = typer.Typer()

# real program has lots of plugins
CLI_PLUGINS = {
            entry_point.name: entry_point.load()
            for entry_point in pkg_resources.iter_entry_points("foo.cli_plugins")
        }

for name, entry_point in CLI_PLUGINS.items():
    app.add_typer(entry_point.app, name=name)


@app.command()
def fizz(path: str = typer.Argument(...)):
    typer.echo(f"Fizzy {path}")

This is installed with this setup.py from foo/setup.py

"""Setup."""

from pathlib import Path

from setuptools import find_packages
from setuptools import setup

PACKAGE_NAME = "foo"

setup(
    name=PACKAGE_NAME,
    version="0.0.1",
    package_dir={"": "src"},
    packages=find_packages(where="src"),
    zip_safe=False,
    include_package_data=True,
    python_requires=">=3.7",
    entry_points={
        "console_scripts": [f"{PACKAGE_NAME} = {PACKAGE_NAME}.cli:app"],
    },
    install_requires=["typer"],
)

• there is also an empty __init__.py in foo/src/foo/

When I call foo --help I get

Usage: ex-primary [OPTIONS] COMMAND [ARGS]...

Options:
  --install-completion [bash|zsh|fish|powershell|pwsh]
                                  Install completion for the specified shell.
  --show-completion [bash|zsh|fish|powershell|pwsh]
                                  Show completion for the specified shell, to
                                  copy it or customize the installation.

  --help                          Show this message and exit.

Commands:
  fizz
  bar

which is also great. This is exactly what I want.

However..

When I call foo bar --help I get

Usage: foo bar [OPTIONS] COMMAND [ARGS]...

Options:
  --help  Show this message and exit.

Commands:
  main

Which is not what I want. I don't want to have to call main out by name here.

Question

• Is there a way to not have main as a command here? I want to be able to call foo bar PATH instead of foo bar main PATH.

What I want for for bar --help is

Usage: foo bar [OPTIONS] PATH

Options:
  --install-completion [bash|zsh|fish|powershell|pwsh]
                                  Install completion for the specified shell.
  --show-completion [bash|zsh|fish|powershell|pwsh]
                                  Show completion for the specified shell, to
                                  copy it or customize the installation.

  --help                          Show this message and exit.

Additional context

I have tried adding a callback to the add_typer and it almost does what I want.
Ex:
changing

app.add_typer(entry_point.app, name=name)

to

app.add_typer(entry_point.app, name=name, callback=entry_point.main) 

• Doing this adds the argument of bar to the usage of foo bar --help (yay) but it still leaves the COMMAND [ARGS] and the Commands section. Also I don't think this is the right way to go about this.

I feel like there is a nice way to get this working and I just cannot see it.

I looked at the documentation (which is beautiful) and the example with subcommands doesn't quite match this use case. There the subcommands are designed to be called explicitly.

If there is any more information I can provide please let me know.

Thank you for your great work on typer! Cheers

[Andrew-Sheridan]

I played around with this for a while and came up with a potential solution:

Change this:

for name, entry_point in CLI_PLUGINS.items():
    app.add_typer(entry_point.app, name=name)

To this:

for name, entry_point in CLI_PLUGINS.items():
    app.registered_commands.append(CommandInfo(name=name, callback=entry_point.main))

Which appears to work.

Is there a better way to do this?

Are there any unintended consequences of this approach? I don't see any documentation for registered_commands at https://typer.tiangolo.com/, and there are no docstrings in the source.

Thanks again for your time, apologies for the wall of text.

[tiangolo]

Hey @Andrew-Sheridan , here's the thing, in the first case, when you do @app.command(), as there's only one function and nothing else, it is used as a single command, not a group of commands.

On the second case, as it is included as a sub-command of the main application and it is a complete Typer application, then it can no longer assume that it's just a simple command directly and will treat it as a group, no matter if it has one or several subcommands.

In this case, I think you can make it explicitly a group from the beginning, by not using @app.command() but @app.callback().

The callback is the main function of a group (instead of a command), but it is taken as the main function for that group that is always run no matter if there are sub-commands.

That way, it would run directly when you call the first version alone, and should work fine when you use app.add_typer() (without creating the second sub-command).

Maybe the docs here can help: https://typer.tiangolo.com/tutorial/commands/callback/

[Andrew-Sheridan]

@tiangolo Thank you answering!

Your explanation seems to make sense but I can't actually seem to implement it correctly. I've gone through the docs and the examples but something just must not be clicking.

If I set the main in bar.py (the plugin) as a callback then bar is unusable.

bar.py

import typer

app = typer.Typer()

@app.callback()
def main(path: str = typer.Argument(...)):
    typer.echo(f"Doing bar things to {path}")

help

$ bar --help
Usage: bar [OPTIONS] PATH COMMAND [ARGS]...

Options:
  --install-completion [bash|zsh|fish|powershell|pwsh]
                                  Install completion for the specified shell.
  --show-completion [bash|zsh|fish|powershell|pwsh]
                                  Show completion for the specified shell, to
                                  copy it or customize the installation.

  --help  

usage

$ bar some_path
Usage: bar [OPTIONS] PATH COMMAND [ARGS]...
Try 'bar --help' for help.

Error: Missing command.

You can see it asking for a command when there isn't one. The same issue occurs when calling it with foo bar some_path.

Is there an aspect to this I'm missing or should I fall back to the undocumented app.registered_commands which appears to work?

Thank you again

[cataerogong]

@Andrew-Sheridan
I have a solution, maybe not so beautiful 😺

@tiangolo
Maybe the behavior of Typer.add_typer() method could change to this: when the sub typer has only 1 command, add it as a command, not a sub-commands group.

---

code

sub.py

import typer

app = typer.Typer()

@app.command()
def main(arg: str):
    typer.echo(f'sub(arg={arg})')

if __name__ == "__main__":
    app()

sub2.py

import typer

app = typer.Typer()

@app.command()
def cmd1(arg: str):
    typer.echo(f'sub2.cmd1(arg={arg})')

@app.command()
def cmd2(arg: str):
    typer.echo(f'sub2.cmd2(arg={arg})')

if __name__ == "__main__":
    app()

main.py

import typer
import sub, sub2

app = typer.Typer()

@app.command()
def main(arg: str):
    typer.echo(f'main(arg={arg})')

if __name__ == "__main__":
    if len(sub.app.registered_commands) == 1:
        app.command('sub')(sub.app.registered_commands[0].callback)
    else:
        app.add_typer(sub.app, name='sub')
    if len(sub2.app.registered_commands) == 1:
        app.command('sub2')(sub2.app.registered_commands[0].callback)
    else:
        app.add_typer(sub2.app, name='sub2')
    app()

---

run

main.py

$ python main.py --help
Usage: main.py [OPTIONS] COMMAND [ARGS]...

Options:
  --install-completion [bash|zsh|fish|powershell|pwsh]
                                  Install completion for the specified shell.
  --show-completion [bash|zsh|fish|powershell|pwsh]
                                  Show completion for the specified shell, to
                                  copy it or customize the installation.

  --help                          Show this message and exit.

Commands:
  main
  sub
  sub2

$ python main.py main --help
Usage: main.py main [OPTIONS] ARG

Arguments:
  ARG  [required]

Options:
  --help  Show this message and exit.

$ python main.py sub --help
Usage: main.py sub [OPTIONS] ARG

Arguments:
  ARG  [required]

Options:
  --help  Show this message and exit.

$ python main.py sub2 --help
Usage: main.py sub2 [OPTIONS] COMMAND [ARGS]...

Options:
  --help  Show this message and exit.

Commands:
  cmd1
  cmd2

$ python main.py sub2 cmd1 --help
Usage: main.py sub2 cmd1 [OPTIONS] ARG

Arguments:
  ARG  [required]

Options:
  --help  Show this message and exit.

$ python main.py sub2 cmd2 --help
Usage: main.py sub2 cmd2 [OPTIONS] ARG

Arguments:
  ARG  [required]

Options:
  --help  Show this message and exit.

$ python main.py main sth
main(arg=sth)

$ python main.py sub sth
sub(arg=sth)

$ python main.py sub2 cmd1 sth
sub2.cmd1(arg=sth)

$ python main.py sub2 cmd2 sth
sub2.cmd2(arg=sth)

sub.py

$ python sub.py --help
Usage: sub.py [OPTIONS] ARG

Arguments:
  ARG  [required]

Options:
  --install-completion [bash|zsh|fish|powershell|pwsh]
                                  Install completion for the specified shell.
  --show-completion [bash|zsh|fish|powershell|pwsh]
                                  Show completion for the specified shell, to
                                  copy it or customize the installation.

  --help                          Show this message and exit.

$ python sub.py sth
sub(arg=sth)

sub2.py

$ python sub2.py --help
Usage: sub2.py [OPTIONS] COMMAND [ARGS]...

Options:
  --install-completion [bash|zsh|fish|powershell|pwsh]
                                  Install completion for the specified shell.
  --show-completion [bash|zsh|fish|powershell|pwsh]
                                  Show completion for the specified shell, to
                                  copy it or customize the installation.

  --help                          Show this message and exit.

Commands:
  cmd1
  cmd2

$ python sub2.py cmd1 --help
Usage: sub2.py cmd1 [OPTIONS] ARG

Arguments:
  ARG  [required]

Options:
  --help  Show this message and exit.

$ python sub2.py cmd2 --help
Usage: sub2.py cmd2 [OPTIONS] ARG

Arguments:
  ARG  [required]

Options:
  --help  Show this message and exit.

$ python sub2.py cmd1 sth
sub2.cmd1(arg=sth)

$ python sub2.py cmd2 sth
sub2.cmd2(arg=sth)

[vlcinsky]

Well done @cataerogong

I had similar problem but in my case I knew, that once I want to add command group, another time single subcommand. For the later the callable can be simply registered via app.command(name="whatever")(callable).

user.py

import typer

app = typer.Typer()


@app.command()
def create(name: str):
    """Create user
    """
    typer.echo(f"Creating user: {name}")


@app.command()
def delete(name: str):
    """Delete user
    """
    typer.echo(f"Deleting user: {name}")


if __name__ == "__main__":
    app()

hello.py

import typer

app = typer.Typer()


@app.command(name="hello")
def action(name: str):
    """Say Hello to NAME.
    """
    typer.echo(f"Hello {name}")


if __name__ == "__main__":
    app()

main.py

import typer

import hello
import user

app = typer.Typer()


app.command(name="hello")(hello.action)
app.add_typer(user.app, name="user")


if __name__ == "__main__":
    app()

At Typer is not modifying the callable hello, it can be simply registered with any other Typer app instance.

Implicit expectations

As seen in this issue, there are more users having the same expectation, that app.add_typer on app with just single subcommand will add just one level of command without creating extra command group.

It would be nice, if Typer would work this (implicitly expected) way. So far it was the only case when Typer it surprised me. Great work @tiangolo.

[andrewasheridan]

Thank you both for your comments and suggestions! I'll be reviewing these in more depth over the weekend.

In the mean time, I'm curious if anyone could comment on my note about using app.registered_commands.append?

[pmsoltani]

Running on Typer v0.3.2, I tried all of the suggestions above for sub-modules with only 1 command. Unfortunately, none of them seem to work 😕

[andrewasheridan]

What about the app.registered_commands.append(CommandInfo(name=name, callback=entry_point.main)) method that has not been addressed yet?

[pmsoltani]

@andrewasheridan that as well. I started at the top. Here is my project structure:

.
├── main.py (FastAPI)
└── cli
    ├── commands
    │   ├── __init__.py
    │   └── cmd_crawl.py
    ├── __init__.py
    └── cli.py

cmd_crawl.py:

import typer

cli = typer.Typer()

def arg_check(values: List[str]) -> List[str]:
    for value in values:
        if value not in ['wikipedia', 'reddit']:
            raise typer.BadParameter(f"Bad value '{value}'.")
    return values

@cli.command()
def crawl(websites: List[str] = typer.Argument(..., callback=arg_check)):
    typer.echo(websites)

cli.py (initial version):

import typer

from cli.commands import cmd_crawl

cli = typer.Typer()
cli.add_typer(cmd_crawl.cli, name="crawl")

Which would make me type in the terminal: proj crawl crawl wikipedia instead of proj crawl wikipedia, but it works. Your solution (and others):

cli.py (new version):

import typer
from typer.models import CommandInfo

from cli.commands import cmd_crawl


cli = typer.Typer()
cli.registered_commands.append(CommandInfo(name="crawl", callback=cmd_crawl.crawl))

Now, these don't work:
proj crawl wikipedia -> Error: Invalid value for 'ENGINES...': Bad value 'crawl'.
proj crawl crawl wikipedia -> Error: Invalid value for 'ENGINES...': Bad value 'crawl'.

But this one does (not what I want):
proj wikipedia -> Good!

[pmsoltani]

OK... I solved my problem:

cmd_crawl.py:

import typer

# cli = typer.Typer() -> No need to create a Typer app instance

def arg_check(values: List[str]) -> List[str]:
    for value in values:
        if value not in ['wikipedia', 'reddit']:
            raise typer.BadParameter(f"Bad value '{value}'.")
    return values

# @cli.command() -> We just need to have a simple function.
def crawl(websites: List[str] = typer.Argument(..., callback=arg_check)):
    typer.echo(websites)

cli.py (current version):

import typer

from cli.commands import cmd_crawl

cli = typer.Typer()
# cli.add_typer(cmd_crawl.cli, name="crawl")
cli.command(name="crawl")(cmd_crawl.crawl)

Now this works: proj crawl wikipedia. Note that this is different from @vlcinsky's solution, as he turned the sub-module hello.py into a Typer app and I didn't. Anyway, my goal was just to break the code into smaller modules and I didn't necessarily want sub-commands.

[yobiscus]

Update: Steps below do not work, COMMAND is still expected.

---

@Andrew-Sheridan

If I set the main in bar.py (the plugin) as a callback then bar is unusable.

The invoke_without_command option appears to solve this problem.

@app.callback(invoke_without_command=True)
def main(path: str = typer.Argument(...)):
    typer.echo(f"Doing bar things to {path}")

With this option set, the command help appears correctly and it can be executed. I'm not sure if this is a gross misuse of the option, but it works as expected for adding a single subcommand to the command tree.