Spike: Provide a way for plugins to have runtime configuration and extend CLI arguments

[noklam]

Description

settings.py is the place that where you can disable/enable plugins, however it's not a good place to put runtime argument in.

Consider that I have a plugin which utilize hooks to do runtime validation. The ideal way of doing so is kedro run --my_plugin_check=True. However, it's not possible to directly override the kedro run command.

Possible Implementations/Issues

1. Override cli.py - this is documented badly - It's probably not a good choice because you need to do this per project, plugin have no control over it.
2. Override kedro run - If plugins are allowed to override/extend kedro run (Is this even feasible?) - there are risks that argument will conflict with each other.

What are the workarounds?

• Use of environment variable or hard-coded config in the plugin. Inject some if/else logic where plugin will behave differently when it sees a specific configuration. The config can be either change with mlflow.yml (take kedro-mlflow as an example) or inject via --params (Separate params to runtime_params and meta_params #2530 suggests that params should be parameters.yml specific, but this seems to be the most natural way to do it from the user perspective. In general, people don't really separate the "kedro parameters", they simply view everything that can be parameterized as "parameters".

Outcome of this Issue:

• Suggest a feasible option to support plugins development

Context

Related to #2384 in terms of adding new arguments for kedro run, but may not be a complete replacement.

Possible Implementation

• Research how pytest did this for its plugins ecosystem

Possible Alternatives

[noklam]

Potentially related to kedro-org/kedro-viz#662 too