Add configurable timeout for OpenAI-compatible providers #7495
lyesrock
started this conversation in
Feature Requests
Replies: 1 comment
-
|
I would appreciate that too. For local usage, it is a neccessary feature, since some requests take a long time. |
Beta Was this translation helpful? Give feedback.
0 replies
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Uh oh!
There was an error while loading. Please reload this page.
-
Hi,
Could you please add the ability to adjust the request timeout for all OpenAI-compatible endpoints, not just Ollama?
We run large local models (llama.cpp, LM Studio, Ollama OpenAI mode, etc.) with big context windows, and we hit the same timeout issues. Since these setups behave like Ollama in terms of latency, having the “Request Timeout (ms)” option available for OpenAI-compatible models would solve the problem.

Thanks!
Beta Was this translation helpful? Give feedback.
All reactions