v1.2.0
Local LLM Provider Support
Add 6 local LLM inference engines as first-class providers, all using OpenAI-compatible HTTP APIs:
| Provider | Chat | Completions | Embeddings | Rerank | Audio | Images |
|---|---|---|---|---|---|---|
| Ollama | ✅ | ✅ | ✅ | — | — | — |
| LM Studio | ✅ | ✅ | ✅ | — | — | — |
| vLLM | ✅ | ✅ | ✅ | ✅ | — | — |
| llama.cpp | ✅ | ✅ | ✅ | ✅ | — | — |
| LocalAI | ✅ | ✅ | ✅ | ✅ | ✅ | ✅ |
| llamafile | ✅ | ✅ | ✅ | ✅ | — | — |
Quick Start
from liter_llm import LlmClient
client = LlmClient(api_key="")
response = await client.chat(
model="ollama/llama3.2",
messages=[{"role": "user", "content": "Hello!"}],
)No API key needed. Model routing uses the standard provider/model prefix convention.
What's New
- 6 local providers with accurate endpoint configurations
- Docker Compose setup for integration testing (Ollama + qwen2:0.5b + all-minilm)
- CI workflow for local LLM integration tests
- Comprehensive documentation at docs/usage/local-llms.md
- Integration tests for chat, streaming, embeddings, and model listing
- Removed duplicate
lm_studioandollama_chatentries (now 143 providers)
Fixed
- PHP
onErrorhook passes Exception not string - README template compliance for rumdl
- Added 404 to POST OpenAPI specs
Full Changelog: v1.1.1...v1.2.0