[Question]: Configure OpenAI + vLLM (OpenAI endpoint) at the same time #1432
-
What is your question?I was wondering how I should configure things if I want access to both OpenAI endpoints, as well as models that I have hosted using vLLM with their OpenAI endpoint at the same time? More DetailsThis isn't specifically related to vLLM, but really any OpenAI compatible endpoint that you can self-host. What is the main subject of your question?Endpoints ScreenshotsNo response Code of Conduct
|
Beta Was this translation helpful? Give feedback.
Replies: 1 comment 3 replies
-
|
The only way to use both simultaneously is to have a dedicated endpoint for reverse proxy: #1344 Not yet implemented, will work on it soon. |
Beta Was this translation helpful? Give feedback.
-
|
I forgot to mention you could potentially configure both through LiteLLM: https://litellm.ai/ |
Beta Was this translation helpful? Give feedback.
-
|
Appreciate the info there! Is it as simple as copying the OpenAIClient.js file and adding it to the index.js? If so, I can attempt and submit a PR. |
Beta Was this translation helpful? Give feedback.
-
|
Appreciate that! No not that simple but not the hardest. This is a priority item for me but I don’t have my computer until the weekend. Will get right on it |
Beta Was this translation helpful? Give feedback.
The only way to use both simultaneously is to have a dedicated endpoint for reverse proxy: #1344
Not yet implemented, will work on it soon.