Make the services and models in `CrossProviderInferenceEngine` data driven

[deanwampler]

Rather than hard-code the supported inference services, models, and corresponding implementation classes in CrossProviderInferenceEngine, how about making this data driven? E.g., move the definitions to a YAML file in the distribution with an option for users to add their own custom entries or, even better, create a separate YAML file that will be searched for in a defined search path. This would make it easy for people to add new services, like Replicate, or overload existing entries, without code changes.

As an alternative, allow the user to provide the same information in the constructor call.

[elronbandel]

Thanks for this feedback. You are absolutely right! This is an initial version to get things up and running. We did not want to decide, as a first step, on how people will supply this information so we wont need to change the api once we get a better understanding of how to implement it.

As of now we are leaning towards: (1) Allow users to also have the option to specify the original name of the model at the target provider instead of the standardized name we created. (2) Make a separate file for each provider that can be updated independently and have its own catalog entry on the website so pepole can browse (equivalent to having a yaml as catalog entries are basically json files). (3) Automate the process of updating the standard names.

Its important to mention that we still want to keep the concept of standard names so we can have a LLM as a Judge for example that uses llama-3-70b and can switch between providers based on user infrastructure. So you can specify: unitxt.settings.default_provider = "aws" and move all your judges to that api provider, this will be impossible without knowing the mapping between names at different providers.

We will refer to this issue at the PR that address that. Thanks again.

[github-actions]

This issue is stale because it has been open for 30 days with no activity.

[github-actions]

This issue was closed because it has been inactive for 14 days since being marked as stale.