Open problems — the measured frontier (pick one) 🧭

[JustVugg]

The frontier — open problems where community help multiplies. Everything follows the house rule: measure before claiming.

1. int2 + incoherence rotation for streamed experts (#81)
Naive int2 is degenerate (~45-60% error). QuaRot-family rotation spreads outliers; the ablation tool measures it (-rot, #132). Needs: a rotation-aware converter pass + A/B.

2. aarch64 i8mm expert kernel
GB10/Grace/Apple/Graviton all go matmul-bound once warm (#136: matmul 130s vs disk 58s) and all have i8mm (SMMLA). The NEON path does not use it. Needs: someone with i8mm silicon; see #95 for the qualification pattern (numeric bound + fixed-prompt eval).

3. NUMA-aware placement (#82)
Dual-socket: +39.6% from thread pinning alone; a 4-socket box collapsed 30× from first-touch misplacement. Needs: interleaved expert allocation + topology-aware OMP defaults + a multi-socket box (one offered in #191).

4. Expert Atlas (#175)
What do the 19,456 experts do? The engine logs per-expert routing; co-activation clustering (#119) and the dashboard Brain page want the same data. Needs: trace-dump format + max-activating-examples probes.

If you start on one, say so here — duplicated effort is the only waste this project cannot afford.

[RonitBStudent]

I’d like to take problem 2, the aarch64 i8mm expert kernel. I verified that my Apple-silicon Mac reports hw.optional.arm.FEAT_I8MM: 1, and Apple Clang can emit SMMLA through a per-function target attribute while preserving the baseline NEON build.
My plan is to implement a runtime-gated kernel, validate it exactly against the scalar reference across real expert row sizes and tail cases, and run an interleaved SDOT-vs-SMMLA microbenchmark. I don’t have enough local storage for the full checkpoint, so after numeric and microbenchmark qualification I’ll package a fixed-prompt A/B for an M-series or GB10 owner with the model loaded. I’ll report measurements before making any speed claim.

[RonitBStudent]

implementation is up as draft PR #217. The exactness suite and local M2 Pro benchmarks pass; selected shapes measure 1.23–3.39× for int8 and 1.01–2.03× for int4. The remaining qualification is the full token oracle and an i8mm Linux/GB10 or Grace run. I8MM=0 provides the same-binary NEON control.

[ZacharyZcR]

Proposed frontier entry: the routing-determinism contract (one bug class, three sightings this week)

Three independent reports this week are the same defect wearing different clothes:

sighting	asymmetry	effect
#163 — MTP 33%→4% under --topp	draft runs S=1 (exact int4 kernel), verify runs S≥2 (IDOT)	acceptance collapses, guard kills drafting
#220 — EXPERT_BUDGET drops MTP 38%→21%	surviving experts depend on cache state at forward time; draft and verify see different caches	same
#161/#199 — CACHE_ROUTE (max-rank prefer)	K−J slots filled from current pin∪LRU residency	will show the same MTP interaction once measured

The invariant all three violate: speculative verification assumes the verifier reproduces the drafter's computation. In #163 we proved the converse experimentally — with the asymmetry removed (IDOT=0), the batched verify is bit-faithful to sequential decode (0 routing mismatches in 6,150 aligned rows, 0 argmax flips, byte-identical text across pre-#68 and current engines). The engine's speculation machinery is correct; it's the state-dependent inputs that betray it.

Right now each feature gets (or will get) its own patch. The alternative is one engine-level rule:

Anything that influences routing must be a pure function of (tokens, positions) for the lifetime of one draft/verify pair.

Concretely: a per-turn snapshot — the kernel choice, the budget survivor-set, the residency set CACHE_ROUTE consults — captured when the draft is proposed and reused for its verification. One struct, one capture point in spec_decode, and every current and future cache-aware routing feature composes with MTP for free instead of silently taxing it. (The features stay adaptive between pairs; they just stop drifting inside one.)

This costs nothing when speculation is off, and it converts "MTP is mysteriously slow with X enabled" from a recurring bug report into an impossibility.

@dnnspaul has the #163 kernel-pinning fix claimed (matmul_qt_ex(..., allow_idot)) — this proposal is the generalization around it, not a replacement. If the shape sounds right I can draft the snapshot struct as an RFC PR after his lands.