Skip to content

v0.4.2

Choose a tag to compare

View all tags
@slrelease slrelease released this 03 Jun 14:22
· 116 commits to main since this release
v0.4.2
38e8ea7

Highlights

This release introduces the Rust worker sidecar for queued inference and expands document extraction and image-text model support.

Features

  • Added MinerU2.5-Pro-2604-1.2B for document OCR and Marqo fashionSigLIP for image-text embeddings. Docling now also accepts image inputs.
  • Chat completions accept min_tokens and chat_template_kwargs; model profiles can supply default sampling settings.
  • Added an FP8 serving profile for Qwen3.6-27B on RTX PRO 6000 and increased Qwen3-0.6B's configured context window to 4,096 tokens.
  • Workers reconcile configuration changes after missed updates or reconnects.

Bug fixes

  • Kept generation dispatch separate from embedding, scoring, and extraction queues.
  • Fixed dense_dim handling in CLIP and PyTorch embedding adapters and CUDA-cache cleanup in visual-document adapters.

Breaking changes

  • Cluster inference is now queue-only. Queue workers require the Rust worker sidecar and NATS JetStream; the Helm chart enables the sidecar by default. Custom deployments must include the sidecar alongside the Python inference worker.
Assets 2
Loading