Skip to content

fastkernel 1.1.3

Choose a tag to compare

View all tags
@abhishekgahlot2 abhishekgahlot2 released this 07 Oct 22:28
· 12 commits to main since this release
v1.1.3
f4747f0

fastkernel 1.1.3: 1% decode speed improvement over v1.1.1, measured in a paired serving test (96 pairs, identical outputs). Outputs are byte-identical to v1.1.1.

  • A shorter sampling search. For sampled answers with top-k at most 32 and no min-p, the top-k and top-p search
    reads the 32 most likely tokens of each vocabulary shard instead of the whole vocabulary. It picks the same tokens.
  • Less state written by the linear-attention (GDN) layers. The checking pass no longer stores the layers' running
    state. When a step keeps fewer than all eight guessed tokens, the commit replays the kept ones instead, which skips a
    151 MB write on Qwen3.8-27B.
  • A deferred state commit. With one request, each step's GDN state commit runs inside the next step's pass instead
    of as separate GPU work.

All three are on by default. SPLASH_SAMPLER_TOPK32=0, SPLASH_GDN_SCAN_NOSTORE=0 and SPLASH_GDN_DEFER=0 turn
them off (docs/SWITCHES.md).

Needs: an M3 or newer Mac, macOS 26.4 or later, Python 3.12 to 3.14, and internet on the first run.

  1. Download fastkernel-1.1.3-macos-arm64.tar.gz below.
  2. tar -xzf fastkernel-1.1.3-macos-arm64.tar.gz && cd fastkernel
  3. Got it through a browser or AirDrop? Run /usr/bin/xattr -dr com.apple.quarantine . once.
  4. SPLASH_DRAFT_HEAD_IDS=$PWD/data/head-ranked.u32 ./splash serve --model incoai/Qwen3.8-27B-Splash
  5. Chat at http://127.0.0.1:8000, or run ./splash opencode (or claude, codex, hermes) in a second terminal.

24 GB Mac: first run sudo sysctl iogpu.wired_limit_mb=20480, and put SPLASH_TEXT_ONLY=1 in front of step 4.
Qwen3.6-35B-A3B: use incoai/Qwen3.6-35B-A3B-Splash in step 4.

The first run sets up its Python packages (20 seconds) and downloads the model (17.4 GB).

sha256 82065032637c798711dff5736227a8abc6f1dc45de6aa808edff509d0f2e77c4

Assets 3
Loading