Skip to content

v0.3.0

Choose a tag to compare

View all tags
@slrelease slrelease released this 29 Apr 11:22
· 175 commits to main since this release
v0.3.0
81d6ef5

Highlights

Document extraction and multimodal retrieval expand substantially, while self-hosted clusters move to a Rust gateway with a dedicated configuration service.

Features

  • Added document inputs and structured extraction results across the server and SDKs, including Docling processing for PDF, DOCX, and HTML.
  • Added GLM-OCR and PaddleOCR-VL-1.5; Qwen3-VL-Embedding-2B and Qwen3-VL-Reranker-2B; Qwen3-Reranker-0.6B and 4B; and SigLIP 2 image-text embeddings.
  • Added GLiNER2, GLiNER-bi and Modern GLiNER-bi, Stablebridge token pruning and highlighting, and a ModernBERT-base embedding configuration.
  • Added automatic GPU out-of-memory recovery and idle-model eviction, together with gateway and configuration-service metrics.

Bug fixes

  • Clients retry transient disconnects and capacity-related service errors without treating permanent connection failures as retryable.
  • Improved propagation of model configuration changes and reporting of unknown or unroutable models.

Breaking changes

  • Self-hosted Helm configuration moves from router to gateway settings and adds a separate config service. Configuration writes belong to that service, not the inference gateway; review custom values and configuration clients when upgrading. The chart enables NATS JetStream queue routing by default.
Assets 2
Loading