Feature request: Add LiteLLM as an LLM provider

[flefevre]

Hi OpenRAG team,

Would it be possible to add LiteLLM as an additional LLM provider, alongside the existing Ollama provider?

LiteLLM is fully compatible with OpenAI-style endpoints, which would make it easy to connect OpenRAG to a wide range of LLM backends through a single standardized interface.

Motivation

Currently, OpenRAG appears to support Ollama as an LLM provider. Adding LiteLLM support would make the project more flexible for users who want to run or route models through:

• OpenAI-compatible APIs
• Local or self-hosted inference servers
• Enterprise gateways
• Multiple model providers through the same API format

Since LiteLLM exposes OpenAI-compatible endpoints, the integration could likely reuse much of the existing OpenAI-style client logic, if applicable.

Proposed behavior

Add a new LLM provider option, for example:

llm_provider: litellm
llm_base_url: http://localhost:4000/v1
llm_api_key: <optional_or_required_key>
llm_model: <model_name>

This would allow users to point OpenRAG to a LiteLLM proxy endpoint and configure the model name just like they would with an OpenAI-compatible provider.
Benefits
Broader compatibility with many LLM providers
Easier deployment in self-hosted or enterprise environments
Ability to switch models/providers without changing OpenRAG internals
Support for users already using LiteLLM as a routing or proxy layer
Additional context
LiteLLM is commonly used as an OpenAI-compatible proxy, so this could be a lightweight but very useful addition for users who need more provider flexibility than Ollama alone.
Thanks for considering this feature request!