Repository navigation
Should we have dedicated models for providers that are OpenAI-compatible? #1858
Replies: 1 comment
|
We've run into this exact issue when integrating OpenAI-compatible APIs from providers like MiniMax and LiteLLM in production. Superficially, the OpenAI interface covers 80% of the contract, but we've seen edge cases where inference arguments or response formats differ (e.g., Dedicated models make these differences explicit and easier to test. For example, we had to add custom retry logic and adjust payloads for MiniMax, even though their docs claim "OpenAI-compatibility." Here’s an illustrative stub: class MiniMaxModel(OpenAIModel):
def generate(self, prompt, **kwargs):
kwargs = self._patch_kwargs(kwargs) # fix keys for MiniMax
return super().generate(prompt, **kwargs)
def _patch_kwargs(self, kwargs):
if 'stream' in kwargs: # MiniMax doesn't support streaming
kwargs.pop('stream')
# handle other provider quirks
return kwargsIt also makes debugging easier—users can see why a provider-specific model exists and what’s different. We’ve found that even small differences (like If the repo aims to be a toolkit—not just a wrapper—dedicated models are the way to go. This also helps with automated testing and CI; you can write provider-specific test cases rather than trying to catch everything in generic tests. |
Uh oh!
There was an error while loading. Please reload this page.
There has been 2 PRs recently on adding dedicated models for providers that are OpenAI-compatible (MiniMax and LiteLLM) such that the OpenAI model could be used with a custom url. Should we create new models for those?
I lean towards yes for 3 reasons:
outlinesbecause they believe they could not use it with their favorite providerAll reactions