[Discussion] First-class config overrides via pyproject.toml?

[BrendanJM]

Hi! Thanks for building this awesome tool. I'm loving using it so far, and excited to show our users how awesome our CLIs can be. I did want to get your thoughts on how you might envision cyclopts evolving. There are some areas the story seems a little unclear with how configs loaded from pyproject.toml interact with those passed in. These aren't feature requests, but it would be good to get your reaction to these, and the general idea of first-class support for pyproject.toml:

Meta App Parameter Overrides

While there is a nice example showing how to override values for commands, I was playing around with this a bit and didn't see an immediately obvious way to infer defaults for parameters passed to the meta app. Take the following example, using verbose and quiet logging flags as an example:

log_params = Group(
    "Logging",
    default_parameter=Parameter(
        negative="",  # Disable "--no-" flags
        show_default=False
    ),
    validator=validators.LimitedChoice(),  # Mutually Exclusive Options
)

# This method requires these types to be optional even if they're not
VERBOSE = Annotated[bool | None, Parameter(help="Increase log output", group=log_params)]
QUIET = Annotated[bool | None, Parameter(help="Decrease log output", group=log_params)]

# Separate dict to track actual defaults since we have to have the CLI default to None
DEFAULTS = {
    'verbose': False,
    'quiet': False
}

def get_meta_param(name: str, value: Any, config: dict[str, Any]) -> Any:
    """Get parameter for meta app from config, accounting for pyproject.toml and command line overrides.

    All meta app parameters must be optional/nullable in order to sue this as a flag for whether or not
    they have been set via the command line.

    Args:
        name: Name of the parameter
        value: Value of parameter received from CLI
        config: Config dict from pyproject.toml

    Returns:
        Parameter value
    """
    # Flag was passed via CLI
    if value is not None:
        return value

    # Flag was specified in pyproject.toml
    if config.get(name, None) is not None:
        return config[name]
    
    # Flag was specified in neither, set true default
    return DEFAULTS[name]


@app.meta.default
def meta(
    *tokens: Annotated[str, Parameter(show=False, allow_leading_hyphen=True)],
    verbose: VERBOSE = None,
    quiet: QUIET = None
) -> Any:
    """Meta App. Used to inject configs from pyproject.toml as defaults
    """
    config = load_pyproject_config()

    # The main Cyclopts parsing/conversion
    command, bound = app.parse_args(tokens)

    # Example of using these - the internals here don't matter
    configure_logging(
        verbose=get_meta_param('verbose', verbose, config),
        quiet=get_meta_param('quiet', quiet, config)
    )

    # Other config parsing for commands below here
    ...

[edit: in case it's not clear why I felt like I have to use None as a default, we otherwise couldn't tell the difference between a passed False config and the method default value of False]

It is entirely possible I am way off the prescribed path here, so if there is an easier way to do this, please let me know. If not, it should be hopefully a little clear how using meta app params ends up feeling a little complex and also locks you out of documenting default values via type hints.

Config Validation

Configs loaded via pyproject.toml seem to skip the standard validation for CLI parameters. E.g. I could have updated the defaults via pyprject to some invalid combination, and this would not be caught by e.g. validators.LimitedChoice(). You can test out the example above, by adding a section to your project.yaml with both verbose and quiet config set to true.

There's probably a number of ways to work around this, but I think you'd have to load in the pyproject.toml early to get these values set in advance of the current validator logic. Perhaps loading pyproject.toml on init and attaching default_config dict to the App class would be one way to go about this? There are probably some considerations for not accidentally changing the defaults as they show in the CLI help text.

Schema Validation

It would be really awesome to be able to generate schema validation json for any commands. By this, I mean some compatibility functionality to generate a schema.json file from the command definitions themselves. These schema definitions can be used by IDEs etc to provide inspection capability to users who are adding a section to configure a published tool. An example of ruff's schema.json is https://github.com/astral-sh/ruff/blob/main/ruff.schema.json.

Having a defined schema for pyproject.toml input would also allow auto-coercion for any types from pyproject.toml, so you wouldn't have to manually validate each config.

Thanks again for sharing this library, it really is great, and I'd love to hear more about your thoughts on this problem space.

[BrianPugh]

First off, thanks for all the kind words and for using Cyclopts! Lets see if I can try and answer some of your questions.

infer defaults for parameters passed to the meta app

If i'm to understand this correctly, it's to determine if the user is explicitly overriding the (possible) value provided in their pyproject.toml. A possible solution (albeit not very well tested!) is to use a meta-meta-app 😵 . Meta-apps should work recursively, but I never really tested this as I previously couldn't think of a good use-case.

@app.meta.meta.default
def meta_meta(*tokens: Annotated[str, Parameter(show=False, allow_leading_hyphen=True)]):
    config = load_pyproject_config()

    command, bound = app.parse_args(tokens)

    if command == meta:
        # if bound.arguments['verbose'] is set, then it was provided from the CLI
        bound.arguments.setdefault('verbose', False)
        bound.arguments.setdefault('quiet', False)
        return command(*bound.args, **bounds.kwargs)
    else:
        return command(*bound.args, **bounds.kwargs)
        
        
app.meta.meta()

Admittedly, it is getting a bit unwieldy, but at least it's consistent :P. Note: if meta.meta doesn't work as expected in this situation, there's a few follow ups:

1. I should fix it.
2. You can just use a totally independent App(help_flags=[]) object here. The only reason the meta app is special in anyway is that it combines help-pages with the main app. In this case, there's nothing additional to document.

Configs loaded via pyproject.toml seem to skip the standard validation for CLI parameters.

Correct, at that stage Cyclopts has "done it's job." It parsed, converted, and validated CLI arguments, and transformed them into a callable function with arguments. We're then just further modifying those bound arguments.

Schema Validation

I think as we get to this stage, the solution ends up being some sort of integration with pydantic/cattrs, and let those libraries perform the heavy lifting.

I think you brought up a bunch of good questions, that unfortunately I don't have any immediate good solutions for. I'm all for brainstorming up possible solutions. If we think we can achieve a lot of what you want with pydantic integration, we can combine discussions with #86.

[BrianPugh]

This feature is implemented in the newly released v2.7.0. See the example in the docs and the API docs for usage.