Skip to content

v1.4.1 — variable-length prompts and multi-turn fix

Choose a tag to compare

View all tags
@skyguan92 skyguan92 released this 04 Aug 09:21
· 356 commits to main since this release
v1.4.1
ba45639

What changed

v1.4.1 fixes variable-length cold prompts and ordinary multi-turn conversations in the native resident engine. Every positive prompt that fits the configured cache capacity is now admitted. Independent cache misses restart from clean resident state; exact and append-prefix hits remain latency optimizations rather than admission requirements.

The fast fixed-context AOT paths and provider policies are unchanged. For transparency, a cold prompt shorter than its selected specialization, or a tail beyond it, executes that unmatched portion through the correct resident token path and therefore runs at decode rather than AOT-prefill throughput.

AMD395 qualification

The exact published binary was requalified on Radeon 8060S / gfx1151:

  • all 19 batch-1 performance cells passed their frozen 0.97x floor;
  • worst median changes versus v1.4.0 were -2.283% prefill and -0.1143% decode, both within the 3% measurement protocol band;
  • nine full-vocabulary correctness contexts passed with maximum KLD 0.002174 < 0.005 and matching top-1 everywhere;
  • the frozen q8192 completion matched all 128 token IDs exactly;
  • q8192 command-to-ready median was 47.36 s against the 51.41 s ceiling;
  • q32768 exact-prefix cache reduced TTFT by 2624x with 1.0002 decode-throughput retention;
  • a 16-token cold prompt and exact replay, a normal 36-token next-user turn, and an unrelated short request after long-context work all returned HTTP 200 with clean cache-miss isolation;
  • live SSE, function tools, disconnect recovery, one-load residency, clean shutdown and isolated portable-bundle provider smokes all passed.

The immutable release tag points to ba45639c178061f9bdadd22c86744f6924f5bf44. The qualified native executable embeds source commit 4536dbaeb6d1d013232db8150fbb6f7c3100b20a and has SHA-256 90c55cb3185d37589c5b2f8afead9ecd4fb53759beae99a95ec763143f9ad7b0.

Install

The archive is self-contained at runtime: it does not load Python, PyTorch, vLLM, Triton, Transformers or a host ROCm userspace. The host still needs Linux x86-64, AMDGPU/KFD/render nodes, a gfx1151 GPU, the documented 96 GiB GTT configuration and separately obtained Qwen3.6-35B-A3B BF16 model weights.

sha256sum -c aima-engine-native-portable-b98b7bc698ae.tar.zst.sha256
tar --zstd -xf aima-engine-native-portable-b98b7bc698ae.tar.zst
cd aima-engine-native-portable-b98b7bc698ae

./bin/aima-engine serve \
  --model-dir /srv/models/Qwen3.6-35B-A3B \
  --context-tokens 8192 \
  --host 127.0.0.1 \
  --port 8000

Portable archive SHA-256: f75562537277af8b3a0e1a92fb012761a1522b7021f3014bc1f5b8355f650d1b.

The product result, portable-bundle audit and deterministic public-evidence archive are attached for independent verification. Raw sanitized qualification reports are also committed on main at c22aee94fff6bed34384642f3a1f4e358a2043a3.

Assets 8
Loading