Skip to content

v1.3.0 — Live Streaming and Function Tools

Choose a tag to compare

View all tags
@skyguan92 skyguan92 released this 23 Jul 17:24
· 366 commits to main since this release
v1.3.0
032dc13

Highlights

  • True token-time HTTP/1.1 chunked SSE with incremental UTF-8, optional usage chunks, [DONE], and client-disconnect decode cancellation.
  • OpenAI-compatible function tools, tool_choice, parallel_tool_calls, structured stream/non-stream tool_calls, and assistant/tool history.
  • Byte-exact native Qwen tool/chat template parity without Transformers at runtime.
  • Self-contained AMD395 / gfx1151 native package with one static launcher and bundle-local ROCm userspace; no host Python, PyTorch, vLLM, Triton, Transformers, or ROCm userspace installation is required.

Qualification

  • 19/19 batch-1 performance cells passed the frozen >=0.97x gate. Minimum prefill retention: 1.0136x; minimum decode retention: 0.9839x.
  • Against v1.2, worst decode ratio was 0.9990x; worst prefill ratio was 0.9833x at the maximum-window 261632/512 cell.
  • 9/9 full-vocabulary correctness contexts passed; maximum KLD 0.002174 < 0.005, with top-1 agreement everywhere and exact 128-token identity at 8K.
  • Median q8192 command-to-ready startup: 44.69 s versus the frozen 51.41 s ceiling.
  • q32768 exact-prefix TTFT: 9.180 ms versus 23.98 s cold (2612x), with 0.99997x decode retention and identical output hashes.
  • SSE first content arrived at 633.3 ms; stream/non-stream text and token hashes matched exactly.
  • Stream and non-stream tool calls both produced get_weather({"city":"Paris"}); tool-result history completed correctly, and forced client disconnect left the one-load resident server healthy.
  • Fresh isolated extracted-bundle provider smokes passed at q1024, q16384, and q65536 with a complete relocatable ELF closure.

Runtime boundary

Model weights are not included. The runtime requires Linux x86-64, the AMDGPU/KFD kernel driver and render nodes, an AMD gfx1151 GPU, and a separately obtained Qwen3.6-35B-A3B-BF16 checkpoint.

Cold HTTP requests remain exact-static-context specialized. Tool definitions and assistant/tool history count toward that context; longer requests are accepted only when they extend the resident cached token prefix.

Integrity

Archive SHA-256: a60800b573915e804ee00180ec66ebce14b10780e380ecb5aaf35e6c9d768ced

Native engine SHA-256: 7b8ae090a80b90a066c39539313ebed92c899d370fc2ccf7021cda03cec0a81d

sha256sum -c aima-engine-native-portable-9442591da597.tar.zst.sha256
Assets 4
Loading