Skip to content

0.2.0 - 26A4 and EmbeddingGemma2

Latest
Latest

Choose a tag to compare

View all tags
@leDissolution leDissolution released this 12 Oct 10:49
0.2.0
0ee5bc3
  • Added Gemma 4 26B A4B support, including HTTP serving, offline generation, image input, prefix caching, and MTP speculative decoding.
  • Added EmbeddingGemma 2 serving through serve-embeddings and /v1/embeddings, supporting text, images, video, audio, and mixed multimodal inputs.
  • Added batched embedding inputs, selectable 128/256/512/768 dimensions, float/base64 output, and dedicated embedding metrics.
  • Expanded tool calling with required calls, named-function selection, allowed-tool subsets, and parallel_tool_calls:false.
  • Added strict tool argument schemas and support for combining tool calls with constrained JSON answers.
  • Expanded non-strict tool schema support, including references, unions, metadata, and argument keys containing Unicode or punctuation.
  • Added reasoning_effort support for chat and chat-prefill requests: none disables thinking; low, medium, and high enable it.
  • Added configurable per-image token budgets of 70, 140, 280, 560, or 1120 through server defaults and request overrides.
  • Corrected image-prefill attention masking
  • Optimized vision processing and short cached-prompt prefill
  • Added packed text/image prefill across requests
  • Added adaptive MTP proposal depth through --decode-width
Assets 2
Loading