Can I run KubeIntellect with a local or self-hosted LLM (Ollama, vLLM)?

[MSKazemi]

We can't send cluster data to OpenAI or Azure. Can KubeIntellect run against a self-hosted or
local model — Ollama, vLLM, or another OpenAI-compatible endpoint?

[MSKazemi]

Partly today, and it is the most-requested item on the roadmap.

What works now: the client is OpenAI-compatible, so pointing OPENAI_BASE_URL at an
OpenAI-compatible server (Ollama, vLLM, LiteLLM, …) does function.

What is honestly missing: it is not a first-class, documented, tested path. There is no
CI coverage for a local backend, no guidance on which open models are good enough at the
tool-calling this system leans on heavily, and no tuned prompts for smaller models. The
coordinator delegates to subagents and relies on structured tool calls — weaker models degrade
in ways that matter for an incident-response tool.

So: usable if you are willing to experiment, not something to depend on in production yet.

Making it first-class is #17, and it
is open to contribution. The most useful things you can do right now:

• 👍 the item in #52 — reaction counts
  are the project's only prioritisation signal and genuinely reorder the roadmap.
• Tell us in #17 which model and
  serving stack you need, and whether it is a hard compliance requirement. "We cannot adopt
  this without X" is the most actionable feedback we get.
• Report what happens if you try OPENAI_BASE_URL today — negative results are useful.