Skip to content

v0.4.0

Choose a tag to compare

View all tags
@slrelease slrelease released this 27 May 19:51
· 160 commits to main since this release
v0.4.0
99eeda7

Highlights

SIE now serves text generation alongside embeddings, reranking, and extraction, including streaming responses and structured output.

Features

  • Added Qwen3-0.6B, Qwen3-4B-Instruct-2507, and Qwen3.5-4B generation models through SGLang.
  • Added a native generation API, OpenAI-compatible chat and legacy completions endpoints, and an initial gateway Responses API implementation. The Python and TypeScript SDKs expose generation options and streaming.
  • Added multi-turn tool calls, multiple response candidates, log probabilities, seeded sampling, and per-request LoRA selection. JSON-schema, regex, and grammar-constrained output are available where supported by the selected model and backend.
  • Added a browsable API reference at /docs and optional bundled certificate management with self-signed TLS for self-hosted clusters.

Bug fixes

  • Streaming now surfaces backpressure failures instead of silently dropping output chunks, and cancellation prevents duplicate generation attempts.
  • Fixed decoding of base64 image inputs and included the system libraries needed by Docling in worker images.
  • GPU-aware health checks detect unusable CUDA contexts so unhealthy workers can be taken out of service.
Assets 2
Loading