Repository navigation
LSE 0.6.2: faster long-context DFlash2 on the gfx1151, no server hang when an idle session is evicted, and no kernel compiles in the middle of a decode.
- macOS: needs mac_linuxgpu build 267 or later; the archive bundles the HSA runtime.
- Linux: needs the amdgpu kernel driver with KFD and access to
/dev/kfd; the archive bundles the HSA runtime.
Performance
Prefill, tok/s (prompt tokens / time to first token, measured by the client on a new prompt length):
| GPU | OS | 256 | 1K | 2K | 4K | 32K |
|---|---|---|---|---|---|---|
| Radeon 8060S (gfx1151) | Linux | 437 | 479 | 511 | 539 | 482 |
| Radeon AI PRO R9700 | iPadOS (LemonSeed Studio) | Results pending | ||||
| Radeon AI PRO R9700 | Linux | 1,231 | 1,399 | 1,458 | 1,484 | 1,385 |
| Radeon AI PRO R9700 | macOS | Results pending |
Decode, code prompt, tok/s:
| GPU | OS | Baseline | dMTP | DFlash2 |
|---|---|---|---|---|
| Radeon 8060S (gfx1151) | Linux | 14.3 | 47.7 | 63.8 |
| Radeon AI PRO R9700 | iPadOS (LemonSeed Studio) | Results pending | ||
| Radeon AI PRO R9700 | Linux | 32.6 | 113.8 | 159.6 |
| Radeon AI PRO R9700 | macOS | Results pending |
Model: Qwen3.8-27B, 4-bit (MLX); drafts: dMTP (dynamic MTP) 8-bit, DFlash2 8-bit. Temperature 0.6, median of 3, after a warm-up that completes kernel variant selection.
Changes
- DFlash2 at long context on the gfx1151, code task: 50.1 tok/s at 64K, 42.9 at 128K and 28.8 at 256K (0.6.1: 39.2, 36.7 and 26.0). Draft-tree verify passes share each key and value read between two tiles, one-row decode batches its key reads, and the width policy prices wide trees correctly at a longer context.
- Evicting an idle session to make room for a growing one no longer hangs the server.
- Attention kernels read the KV cache's size at run time, so a session growing past a new cache rung compiles nothing in the middle of a decode.
- Kernels that stream large weights are timed cold when variants are compared.
- A run without a working GPU backend, or with an op that has no device kernel, fails instead of falling back to the CPU unless the CPU backend was chosen.