-
Notifications
You must be signed in to change notification settings - Fork 75
Open
Description
Right now our configs mostly look like this:
Similar for Environments.
It should be possible to also load external classes as well, except that the schema checks reject them.
{
"class": "mlos_bench_custom.services.SomeNewServiceNameBeingLocallyDeveloped",
"config": {
// UNrestricted set of configs based on the class
}
}We have a hack around that using an environment variable currently:
| _SKIP_VALIDATION = environ.get(_VALIDATION_ENV_FLAG, "false").lower() in { |
However, an alternative option that would allow local extensibility is the following:
- Extend the schemas to also allow mostly free form
configsections forclassspecifications under particular new namespaces (e.g., the one listed above). - Create your new class under that namespace and
pip installit locally or else just adjusts yourPYTHONPATH
This would allow local configs and classes to be developed inside the local repo and outside of the main MLOS repo.
One challenge with this is that it would be more challenging to collect new Services and Environments into the main repo unless people submit a PR, but there would now be less incentive to do that.
Not sure I have a good solution to that.
If we decide to do this:
- Extend the schemas
- Add documentation in the README and docstrings
(especially after Rework documentation generation #869)
Metadata
Metadata
Assignees
Labels
No labels
{ "class": "mlos_bench.services.SomeKnownServiceName", "config": { // restricted set of configs based on the class } }