What do you want to change?
Add four official local-LLM provider extensions under packages/coding-agent/examples/extensions/: custom-provider-llamacpp, custom-provider-lmstudio, custom-provider-vllm, custom-provider-ollama- implementing the extension-based approach @badlogic outlined in #3357.
Why?
#3357 is still open. #3567 was closed pointing back to #3357. Today there's no first-party recipe for the four common local inference engines, and custom-provider-qwen-cli (the example people were modeling on) was removed in 8db0d28. New users hand-list models in ~/.pi/agent/models.json and hit the gaps the issue thread already identified.
How? (optional)
Each extension is an async-factory custom provider that probes its engine's model endpoint at startup and
registers discovered models with a curated (id-substring → contextWindow, maxTokens, reasoning) table. Users override per model via ~/.pi/agent/models.json. When the engine is unreachable the provider still registers (empty models) and a session_start ctx.ui.notify explains why.
- llama.cpp — OpenAI-completions against
:8080/v1; LLAMACPP_BASE_URL override.
- LM Studio — tries enriched
/api/v0/models first (richer metadata: max_context_length, vlm type,
capabilities), falls back to /v1/models. Sets compat.supportsDeveloperRole/Reasoning: false per model.
- vLLM — reads
max_model_len per model; forwards VLLM_API_KEY with authHeader: true when set.
- Ollama — discovers via native
/api/tags; OpenAI shim at /v1 for traffic; native /api/chat adapter
left as inline TODO (ollama#12557 makes the shim lossy for tool calls — out of scope here, separate
follow-up).
Branch is ready and npm run check passes: https://github.com/Sardor-M/pi-mono/tree/feat/local-llm-extensions
I have already implemented this. I'd be happy to send a PR once it is a lgtm.
What do you want to change?
Add four official local-LLM provider extensions under
packages/coding-agent/examples/extensions/:custom-provider-llamacpp,custom-provider-lmstudio,custom-provider-vllm,custom-provider-ollama- implementing the extension-based approach @badlogic outlined in #3357.Why?
#3357 is still open. #3567 was closed pointing back to #3357. Today there's no first-party recipe for the four common local inference engines, and
custom-provider-qwen-cli(the example people were modeling on) was removed in 8db0d28. New users hand-list models in~/.pi/agent/models.jsonand hit the gaps the issue thread already identified.How? (optional)
Each extension is an async-factory custom provider that probes its engine's model endpoint at startup and
registers discovered models with a curated
(id-substring → contextWindow, maxTokens, reasoning)table. Users override per model via~/.pi/agent/models.json. When the engine is unreachable the provider still registers (empty models) and asession_startctx.ui.notifyexplains why.:8080/v1;LLAMACPP_BASE_URLoverride./api/v0/modelsfirst (richer metadata:max_context_length,vlmtype,capabilities), falls back to/v1/models. Setscompat.supportsDeveloperRole/Reasoning: falseper model.max_model_lenper model; forwardsVLLM_API_KEYwithauthHeader: truewhen set./api/tags; OpenAI shim at/v1for traffic; native/api/chatadapterleft as inline TODO (ollama#12557 makes the shim lossy for tool calls — out of scope here, separate
follow-up).
Branch is ready and
npm run checkpasses: https://github.com/Sardor-M/pi-mono/tree/feat/local-llm-extensionsI have already implemented this. I'd be happy to send a PR once it is a
lgtm.