-
Notifications
You must be signed in to change notification settings - Fork 1.3k
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Support Deepinfra API #266
Comments
I've seen an increased number of people interested in adding more and more different kinds of providers (myself included #261 , and the list can get so large, I am voting in favor of the implementation for LiteLLM which supports the vast majority of all of these providers including the legacy ones (OpenAI, Anthropic, to mention a few)... LiteLLM does removes the verbosity behind the conundrum of adding more and more providers by supporting them right out-of-the-box, which like I expressed, I feel it would make this particular process a bit more tedious if not more complex due to the different factors of everything affecting the provider selections, adding models, etc., etc., Given the way SWE-Agent is growing with the interest of people wanting to add more providers, I propose that LiteLLM should be integrated with SWE-Agent so that way we just have one main chain of doing things, instead of focusing of adding all of these providers one by one, and why reinvent the wheel when there's a solution already that can be adopted? |
I agree with this. I'd be open to add litellm, ideally starting with some of the less important or newer models (anything but gpt4, claude 2, claude 3). Basically I'd want to move slowly so we don't disrupt the research that uses those 3 models. Please keep in mind that a lot of the cheaper models are too bad to really be used for swe-agent. |
We're going to support something like LiteLLM which will deprecate the need for this issue. Closing. |
Describe the feature
Deepinfra is currently one of the cheapest API providers of LLM inference (together with groq see #108), which makes it interesting to support their API. They currently have the cheapest OpenChat, mixtral and Llama 3 inference token pricing available.
Having more API providers supported also decreases dependability on any particular one.
Potential Solutions
Support the Deepinfra API. See their docs.
The text was updated successfully, but these errors were encountered: