Skip to content

Ember v0.6.2

Choose a tag to compare

@voidwest voidwest released this 11 Aug 21:35
· 111 commits to main since this release

Changed

  • Q4_K/Q6_K production matmul now uses canonical transient Q8_K activation
    packing and integer dots for decode and prefill. The exact-f32 dequantize/dot
    implementation remains an explicit slow oracle. Non-finite activations fail
    before destination mutation; warmed workspace and accumulate semantics are
    explicit contracts.
  • Execution plans retain loader fallback provenance and identify the numerical
    runtime with kernel_revision = 2 while preserving offline verification and
    hashes of historical revision-1 plans. Plan/dispatch disagreement is a
    release-mode error, and the cache key now includes Rayon thread count.
  • The superseded internal K-quant modules (k_gemv, k_prefill,
    k_matmul_x86) and their ad-hoc examples/benches were replaced by
    k_quant_matmul. Ember's Rust library is internal/unstable; this is an
    intentional source-level break rather than compatibility shims over dead hot
    paths.
  • The v0.4 planned-decode interpreter moved out of src/llama.rs into
    src/planned_decode.rs (resolved ops, scratch-arena session, planned and
    fused kernels, forward_last_logits_planned). Zero behavioral change;
    src/llama.rs keeps the model, eager forward, dispatch, and plan
    construction. The v0.5 research contract's reference to the interpreter
    path was updated accordingly.
  • Toolchain pinned via rust-toolchain.toml to 1.92.0 (the declared MSRV
    and CI toolchain); local builds now match CI.
  • Dependencies: crossbeam-epoch 0.9.18 → 0.9.20 (RUSTSEC-2026-0204),
    anyhow → 1.0.104 (RUSTSEC-2026-0190), memmap2 → 0.9.11
    (RUSTSEC-2026-0186). cargo audit is now part of CI.

Added

  • First-class KV-prefix snapshots for the Llama/Qwen CPU runtime through the
    ember kv command family: deterministic ember.kv-snapshot.v1 artifacts,
    strict integrity and compatibility checks, bit-exact same-model replay,
    cache comparison and perturbation diagnostics, replay traces, and explicit
    RoPE remove/reapply research utilities. The validation matrix and benchmark
    evidence are recorded under artifacts/benchmark-kv-v1/2026-08-08/.
  • The bilingual “Road to Ember 1.0” documentation series and a substantial
    native-console visual refresh, while preserving the same v0.5 experiment
    execution path and offline behavior.
  • A pinned llama.cpp known-answer verifier for Q8_K bytes and Q4_K/Q6_K dots,
    fail-closed 1B real-model scalar/x86 validation, a dedicated x86 CI gate,
    adversarial kernel matrices, and schema-4 path-interleaved benchmark output
    with checksums and full dispatch/workspace provenance.
  • tests/property.rs: proptest suite (tensor shape ops vs hand-rolled
    references, decode-arena disjointness/alignment/isolation, K-quant dequant
    contracts) plus fuzz-style robustness tests for the untrusted-input
    boundaries: the GGUF loader, the v0.5 spec parser, and the npy reader must
    never panic on arbitrary input.
  • benches/hooks_overhead.rs: planned-decode cost with a noop experiment
    runner attached vs bare (Gate H evidence; measured ≈0% overhead).
  • Rustdoc lints (broken_intra_doc_links, private_intra_doc_links) and a
    CI docs step (RUSTDOCFLAGS=-D warnings cargo doc).

Fixed

  • forward_last_logits_planned had hard-coded ExecutionMode::Planned, so
    planned-fused parity tests never executed F1-F5. It now routes the model's
    actual mode; tests use execution counters. This exposed Q8_0 F5 overwriting
    the residual (Q8_0 assigns rather than accumulates), so Q8_0 F5 now de-fuses
    with a serialized reason while f32 and canonical K kernels execute F5.
  • Owned reader-backed K weights now receive the loader-resolved execution tier
    at construction, matching mmap-backed weights. Loader fallback reasons and
    original dtype survive model construction into execution plans.
  • Build provenance now exports the EMBER_GIT_COMMIT name consumed by plans,
    traces, and benchmarks (plus dirty-tree status for benchmark records).
  • tests/k_parity.rs::v04_planned_inactive_hooks_real_model failed on Q8_0
    models: the plain run uses the v0.3 native fast path (contract D1) while
    the hooked run uses the generic hooked path, so bit-exact logits were not
    the right contract there (tokens always matched). The test now skips
    models without K-quant tensors; all parity suites (Q8_0, Q6_K, Q4_K_M)
    pass 5/5.
  • All 8 rustdoc warnings (math-notation brackets parsed as links,
    private-item links, unclosed HTML tag in a doc comment).
  • Stale #[allow(dead_code)] on KQuantWeight::try_from_mmap (the loader
    calls it), on the generic LlamaMlp/Attention/Block structs, and dead
    gemma4 forward_full paths (test-only attention ported behind
    #[cfg(test)]).
  • // Safety: comments on the previously undocumented unsafe sites: the
    matrixmultiply::sgemm call in tensor.rs and 29 SIMD kernels in
    simd.rs.