Skip to content

Add extensible json config option for externally developed Services and Environments #880

@bpkroth

Description

@bpkroth

Right now our configs mostly look like this:

{
   "class": "mlos_bench.services.SomeKnownServiceName",
   "config": {
   // restricted set of configs based on the class
   }
}

Similar for Environments.

It should be possible to also load external classes as well, except that the schema checks reject them.

{
   "class": "mlos_bench_custom.services.SomeNewServiceNameBeingLocallyDeveloped",
   "config": {
   // UNrestricted set of configs based on the class
   }
}

We have a hack around that using an environment variable currently:

_SKIP_VALIDATION = environ.get(_VALIDATION_ENV_FLAG, "false").lower() in {

However, an alternative option that would allow local extensibility is the following:

  • Extend the schemas to also allow mostly free form config sections for class specifications under particular new namespaces (e.g., the one listed above).
  • Create your new class under that namespace and pip install it locally or else just adjusts your PYTHONPATH

This would allow local configs and classes to be developed inside the local repo and outside of the main MLOS repo.

One challenge with this is that it would be more challenging to collect new Services and Environments into the main repo unless people submit a PR, but there would now be less incentive to do that.
Not sure I have a good solution to that.

If we decide to do this:

Metadata

Metadata

Assignees

No one assigned

    Labels

    No labels
    No labels

    Type

    No type

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions