Support for llama-swap, llama.cpp and also vllm

[elcire]

Could we please make the new cline compatible with local llm's? local llms are run for very long running tasks and also where privacy is required (e.g. on data containing government id's etc).

[Bennett-Wendorf]

Llama-swap for sure, and very likely others as well, function perfectly fine as OpenAI Compatible providers, which Kanban seems to support well.

For Llama-swap specifically, I've had luck creating a new provider in my list and pointing the base URL to my llama-swap instance. Notably, there seems to be a bug at the moment that API key is a required field, even if the provider doesn't actually validate it. If you llama-swap instance isn't configured to use a key, you should be able to put any value in the API key field and have it work.

[musaabhasan]

Local model support is most useful if it is treated as a provider contract rather than a special case for one runtime. Many local stacks can present an OpenAI-compatible API, but their capabilities differ a lot: tool calling, structured outputs, context length, streaming, multimodal support, and prompt caching are not guaranteed.

A practical implementation would let users configure:

• base URL and model name
• provider type: OpenAI-compatible, llama.cpp, vLLM, llama-swap, Ollama, etc.
• capability flags discovered or manually set
• timeout and max context controls for long-running tasks
• privacy mode that keeps prompts/results local and avoids server-side routing

For agentic workflows, capability detection matters. If a local model does not support reliable tool calling or structured output, the UI should show that clearly instead of failing later during task execution.