v0.5.4

Engine (megakernel/)

• attn_core_q8 (sm_80+): decode attention for the q8_0 cache. It keeps three 32-position tiles in flight with one barrier per tile, and its partials are bit-identical to attn_core_t's. It is used past 640 positions on the engine's own cache; paged slots and Turing keep attn_core_t, and MK_DATTN=0 selects the old kernel. (Benjamin Cutler)

• pk_attn8 (sm_80+): prefill attention with Q·Kᵀ as int8 tensor-core MMAs on the q8_0 keys and P·V in fp16. pk_attn2 does Q·Kᵀ in fp16 (MK_ATTN_S8=0). Neither is bit-exact with pk_attn; MK_ATTN=1 keeps pk_attn. (Benjamin Cutler)

• Attention split-K partition: the chunk size is a multiple of the 32-position tile and grows one tile every 640 positions, and launches are capped (att_cap_T) so they don't cross a step. Greedy output no longer depends on the draft depth (-d). Output bits change. The parallel engine's single-slot step caps T the same way. (Benjamin Cutler)

• MK_KV: low-bit KV cache formats (4–8 bits per value, , optional Hadamard rotation). MK_KV_WIN keeps the newest positions in a q8_0 ring. Presets D (K*=6,V*=5) and F (K*=5r,V*=5) take 26,112 / 23,936 bytes per position, against 36,992 for q8_0. Single conversation and sm_80+ only. (Benjamin Cutler)

• Device memory down about 0.9 GiB:

  • CUDA memory pool instead of one cudaMalloc per tensor: −370 MiB (MK_CUDAMALLOC=1 restores the old allocation)
  • CUDA stack limit set to 0, so mk_cycle's frame goes from 2,496 to 624 bytes: −222 MiB
  • The draft head reads lm_head rows through draft_ids: −160 MiB
  • With a low-bit MK_KV cache, the prompt checkpoint is kept in host memory: −160 MiB

  Ported from Benjamin Cutler's fork.

• gguf.h: release() unmaps on Windows. capi.cpp exports

• megakernel_build.py: bmm.cuh and prefill_attn.cuh added to the rebuild sources.

Server (mk_server.py)

• POST /v1/messages and /v1/messages/count_tokens (Anth/responses (OpenAI Responses), next to/v1/chat/completions.
• Both new endpoints run under the --agents / --parallel scheduler.
• A context overflow returns the wording Claude Code's

CLI and setup

• openjet serve, and openjet connect claude-code | codex | opencode | openclaw.
• openjet setup --long-context (megakernel_long_contextV='K*=5r,V*=5' MK_KV_WIN=2048. The context limit on a 24GB card goes from 120,000 to 234,291 positions. The interactive setup offers it, and it is saved per model profile.
• The engine is downloaded prebuilt from the release mapenjet-engine-linux-x64-sm*.tar.gz,openjet-engine-windows-x64-sm*.zip). It is compiled with nvcc only when none matches.
• openjet bench: a fixed workload that writes a results

Install

• install.sh / install.ps1 (release files) install open-jet with uv and a uv-managed Python 3.12.

CI and release

• engine.yml builds the engine for sm_75, sm_86, sm_89 8) and Windows, and release.yml attaches the archives.
• release.yml attaches install.sh and install.ps1, and the PyPI upload uses skip-existing.
• benchmarks.yml checks benchmarks/community/. New Benc

Benchmarks

• megakernel/bench/engine_compare.py and run_compare.sh: OpenJet against llama.cpp on one GPU.
• scripts/bench_table.py builds the README's community

Repo

• ui/ is now the open-jet-harness submodule.

Docs

• docs/clients.md: new.
• docs/models.md: download links for every catalog model, with the Qwen3.8-27B Q4_K_M at unsloth revision 566347e.
• docs/installation.md: a requirements table.
• docs/AI_REFERENCE.md and docs/MEDIA_KIT.md: new.
• docs/models.md: download links for every catalog model, with the Qwen3.8-27B Q4_K_M at unsloth revision 566347e.
• docs/installation.md: a requirements table.
• docs/AI_REFERENCE.md and docs/MEDIA_KIT.md: new.
• README restructured. ROADMAP.md and CONTRIBUTING.md updated.

Tests

• test_engine_api_compat.py, test_connect.py, test_community_bench.py, test_megakernel_build.py.