Release v2.8.0 · houtini-ai/houtini-lm

What's New

Quality metadata — every response includes structured quality signals (truncation, think-block detection, token estimation, finish reason) so Claude can make informed trust decisions about local LLM output
Session metrics resource — houtini://metrics/session MCP resource exposes cumulative offload stats and per-model performance as JSON, enabling proactive routing feedback
Request semaphore — inference calls are serialised to prevent stacked timeouts when parallel requests hit a single-model server
HuggingFace thinking detection — auto-detects enable_thinking support from HF chat templates at startup, suppresses thinking at inference time. No hardcoded model lists.
Token savings benchmark — benchmark.mjs tests realistic delegation patterns against real source files. 93.3% savings measured across code review, architecture review, and code explanation tasks.

SQLite statement leak in getCachedProfile — statement was not freed if getAsObject() threw (now wrapped in try/finally)
Unflushed SSE buffer — the final streaming chunk (often containing usage data) could be stranded in the buffer after loop exit, causing missing token counts on truncated responses
Session stats on truncated responses — token counts now estimated from content length (~4 chars/token) when the usage chunk is lost, instead of silently showing zero
Division by zero guard — tokPerSec calculation now requires minimum 50ms generation time

Prompt structure — custom_prompt uses multi-turn format (context acknowledged before instruction); code_task moves task into system message
Output constraints — per-family hints now include explicit markdown format instructions

Full changelog: https://github.com/houtini-ai/lm/blob/main/CHANGELOG.md
npm: npm install @houtini/lm@2.8.0