Skip to content

v0.4.1 — Thinking model support, queue depth API, 15 health checks

Choose a tag to compare

View all tags
@twinsgeeks twinsgeeks released this 14 Apr 23:30
· 227 commits to main since this release
v0.4.1
d368a24

Highlights

  • Thinking model support — auto-detects DeepSeek-R1, QwQ, phi-4-reasoning and inflates token budgets to prevent empty responses
  • Queue depth API — GET /fleet/queue for client-side backoff decisions
  • KV cache bloat detection — health check detects when OLLAMA_NUM_PARALLEL is too high
  • Stream reliability checks — client disconnect and incomplete stream tracking
  • 15 health checks total

Fixes

  • Embeddings proxy was incorrectly routing through chat pipeline
  • shutil.which() couldn't find mflux/DiffusionKit installed via uv tool
  • Client disconnects and incomplete streams now tracked correctly

See CHANGELOG.md for full details.

Install: pip install ollama-herd==0.4.1

Assets 2
Loading