Ollama and LM Studio integration

[easygoer33]

Hello brilliant minds, I am having trouble connecting Onyx to Ollama and LM Studio. I have models loaded in both but Onyx cannot see them. Error 111.

[jmelahman]

The most common issue is that Onyx is running in a container which has a separate network from the host network and localhost inside the container resolves to the container's IP address rather than the host's where Ollama is running. To connect to the host from inside the container, you can use http://host.docker.internal:11434/ instead of localhost.

I merged fix(ollama): better default API Base URLs when running in docker (#10045) today which adds a hint when Onyx is running in a container and suggests this alternative URL.

[jmelahman]

I don't think it's relevant to this particular issue, but I also noticed that LM Studio does only binds 127.0.0.1 by default. You may also need this Running Onyx in Docker on the same machine as LM Studio? to get this host.docker.internal address to work correctly.

[yawoo]

I have another similar question. Both onyx and ollama are running in dockers under same docker network. onyx can use ollama llm models. But under Index settings, it cannot set this ollama model in captioning llm, returns error: could not resolve provider. Eventually, I have to setup custom models in llm setting for this ollama docker, and it can be used in captioning llm. Any idea to fix?