[FEATURE] Load command line options from config file

[timothybrooks]

Hello! I love Typer and have started to use it for my deep learning training.

One addition that would be really helpful for deep learning CLIs is the ability to load command line arguments and options from configuration files. This is particularly useful to help manage CLIs with lots of arguments and options, especially when certain sets of inputs are common. Sacred configs have this ability, as do WandB configs:

Here is a demo of how this could work with Typer:

Parameters specified in resnet.yml will automatically fill in args/options of the same names:

train.py --config resnet.yml

When multiple configs are provided, the latter will override the former should there be conflicts.

run.py --config resnet.yml small_test.yml

If args/options are also specified directly, those override anything provided in the config.

run.py --config resnet.yml small_test.yml --num_layers 4

An alternative to consider is just making each config its own boolean option. This has the following downsides:

• Requires adding more flags, even if they only modify other args/options. This does not scale well.
• Ambiguous overriding if multiple configs are provided or if args/options are also added directly.

With argparser, people will sometimes use the following pattern to achieve something similar. By changing the defaults and re-parsing rather than directly changing the args, this allows args/options defined directly on the command line to still override the new defaults. However, this too does not scale well, and there is no clear way to change defaults with Typer.

args, _ = parser.parse_known_args()
        if opt.small_test:
            parser.set_defaults(batch_size=1, num_layers=2, multiscale=False, ...)
args = parser.parse()

I believe the addition of loading args from config files would make Typer fantastic for deep learning research, making it much easier to scale and compose different configurations, and clarifying which parameters override others.

This unfortunately isn't something I know how to implement, but I would be happy to discuss or clarify this further if you agree that it would be a good addition.

Thanks!

[tiangolo]

I'm glad you're liking Typer!

So, this would actually fall in the scope of each specific CLI tool, how to handle it, etc.

But you can easily build something like that.

You could install:

$ pip install pyyaml

Then let's say you have a file dl.py:

import typer
import yaml
from pathlib import Path


def main(config: Path = None, num_layers: int = None):
    config_data = {}
    if config:
        config_data = yaml.load(config.read_text(), Loader=yaml.Loader)
    if num_layers:
        config_data["num_layers"] = num_layers
    typer.echo(f"DL config: {config_data}")

And let's say you also have a config.yml file:

version: "2.4"
num_layers: 5

And let's imagine you are using Typer CLI to get autocompletion for it.

You could use it like:

$ typer ./dl.py run

DL config: {}

$ typer ./dl.py run --config ./config.yml

DL config: {'version': '2.4', 'num_layers': 5}

$ typer ./dl.py run --config ./config.yml --num-layers 2

DL config: {'version': '2.4', 'num_layers': 2}

(I just tried all that) 🎁

[pakozm]

I think this feature will be very useful. @tiangolo many thanks for this awesome library. But the proposed solution is not convincing to me. Ideally you would like to have just a reverse of what is proposed. IMHO, someone will expect that config file integration will let you use the function arguments directly instead of using the new config dictionary which may be overridden by the given function arguments. Even more, no default values can be given to function arguments because None is used in the proposal to decide whether or not override the cofnig dictionary.

Is it possible to put a wrapper over the main function, so such wrapper will look for config file option, and if found it, it will call main with the values given there?

I was thinking in some kind of "partialization" of the main function, but I'm not sure how typer will react. Something like this (no real code):

import functools
import typer
import yaml
from pathlib import Path


def main(num_layers: int = None):
    typer.echo(f"DL num layers: {num_layers}")

def config_wrapper(main, *args):
    if config options are in command line:
        config = load yaml from given config path
        return functools.partial(main, **config)
    return main

typer.run(config_wrapper(main, '-c', '--config'))

Will something like this work? How far is typer of accepting something like this? Is it just something dumb what I'm proposing?

[tiangolo]

@pakozm I'm pretty sure that should work as Typer doesn't modify the original function. I think the original signature is also kept in the wrapped function, but I actually wouldn't be certain.

It's an interesting experiment for sure 😄 🧪 🥽

[github-actions]

Assuming the original issue was solved, it will be automatically closed now. But feel free to add more comments or create new issues.

[malarinv]

I am looking for something like https://pypi.org/project/ConfigArgParse/ but that integrates with typer

[malarinv]

hmm. hydra seems to handle them better https://hydra.cc/docs/tutorial/config_file

[maxb2]

UPDATE: I made a package for the solution below: maxb2/typer-config.

I found a simple solution to this inspired by phha/click_config_file.
In short, the config option is set with is_eager so that it's callback is called before the rest.
The callback then sets the default_map in the underlying click context based on the config file.
You can still set args and options to override the config file.

# typer_config.py
import typer
import yaml

app = typer.Typer( )

def conf_callback(ctx: typer.Context, param: typer.CallbackParam, value: str):
    if value:
        typer.echo(f"Loading config file: {value}")
        try: 
            with open(value, 'r') as f:    # Load config file
                conf = yaml.safe_load(f)
            ctx.default_map = ctx.default_map or {}   # Initialize the default map
            ctx.default_map.update(conf)   # Merge the config dict into default_map
        except Exception as ex:
            raise typer.BadParameter(str(ex))
    return value

@app.command()
def main(
    arg1: str,
    config: str = typer.Option("", callback=conf_callback, is_eager=True),
    opt1: str = typer.Option(...),
    opt2: str = typer.Option("hello"),
):
    typer.echo(f"{opt1} {opt2} {arg1}")


if __name__ == "__main__":
    app()

With a config file:

# config.yaml
arg1: stuff
opt1: things
opt2: nothing

And invoked with python:

$ python typer_config.py --config config.yml
things nothing stuff

$ python typer_config.py --config config.yml others
things nothing others

$ python typer_config.py --config config.yml --opt1 people
people nothing stuff

[real-yfprojects]

In short, the config option is set with is_eager so that it's callback is called before the rest.

Can this work-around be used for config files that aren't passed through cli e.g. pyproject.toml?

[maxb2]

In principle, yes. How you implement it depends on the desired behavior. The only requirement for this work-around is that you can manipulate the ctx.default_map before the main parameters are parsed (essentially rewriting the default values you gave in the source code). That's why you have to use is_eager=True for the --config option. It parses that option before any others.

• You could keep the --config option but it loads from a pyproject.toml by default. Something like:

  if value:
      with open(value, 'r') as f:    # Load config file
          conf = yaml.safe_load(f)
  else:
      conf = load_conf_from_pyproject()
  ctx.default_map = ctx.default_map or {}   # Initialize the default map
  ctx.default_map.update(conf)   # Merge the config dict into default_map

• You could use a callback for the app itself which would run before any command is invoked.

I'd suggest the first option, as it gives the end user some flexibility. Plus, you can hide the option with config: str = typer.Option("", hidden=True) if you don't want to advertise it to the end user. In the second option, it would be really bad if the callback unexpectedly raised an exception because it would block the cli from ever running (including --help).

[real-yfprojects]

In the second option, it would be really bad if the callback unexpectedly raised an exception because it would block the cli from ever running (including --help).

That is a problem. However qa tools should load their config from pyproject.toml by default without the need to specify a flag. At the same time I'd like to have the option to specify the --config flag to force a custom config file without loading pyproject.toml. Since I am still trying to decide whether to use typer, can tell me whether that's possible?

[maxb2]

At the same time I'd like to have the option to specify the --config flag to force a custom config file without loading pyproject.toml.

That's exactly what the first approach would do. If --config FILE is provided, use that otherwise use pyproject.toml.

[maxb2]

@real-yfprojects I put this workaround in an easy-to-use package: maxb2/typer-config. Right now it only supports the first approach that I presented, but I'll be adding the second soon.

[real-yfprojects]

Thanks! Looking good 💯

[imagejan]

Thanks @maxb2 for providing typer-config.

When I try to run an app containing a config option without providing --config (or just with --help), I get:

 Invalid value for '--config': [Errno 2] No such file or directory: ''

Is this intended? Is there a way to make the --config option optional? (I can open an issue in typer-config if it's not me doing something wrong here.)

[maxb2]

UPDATE: See typer-config=0.5.0 for a fix.

@imagejan That is both intended and not. I certainly want it to do that for --config non-existent.yml but not when --config is not provided. Failing during --help is even worse 🙃, so I'll definitely fix that.

However, you can easily get around this by defining your own config file loader with a conditional:

def new_loader(param_value: str) -> Dict[str, Any]:
    if not param_value:
        # Nothing provided, so return an empty dictionary
        # which is a no-op in the parameter callback function
        return {}
                        
    return yaml_loader(param_value)

new_callback = conf_callback_factory(new_loader)

# etc.

Could you open a new issue in maxb2/typer-config where we can discuss default behaviors?