ExpertMesh v0.3.0 — Reference hardening and project site

ExpertMesh v0.3.0 — Reference hardening and project site

Reference-path hardening

• Adds adversarial INT4 coverage for odd packed dimensions and extreme signed values.
• Exercises malformed checkpoint manifests and a dropped remote expert worker.
• Expands the CPU suite to 23 passing unit/integration tests.

Independent planning evidence

• Independently tests the 32 × 1.6 TB/s HBM roofline arithmetic: 1,024–1,706.7 aggregate t/s for 50–30 GB active weights per token.
• Adds a pessimistic MI210 sensitivity profile: 55% achieved HBM bandwidth and 45–65 GB/token gives a 433–626 t/s HBM-only ceiling.
• Documents a $239k–$352k pessimistic acquisition sensitivity; this remains a planning scenario, not a quote.

Project site

• Adds the supplied standalone ExpertMesh field-note page at docs/index.html.
• GitHub Pages is enabled from the main branch /docs directory with HTTPS enforced.

Boundary

These are CPU/reference and planning validations. They do not certify HIP execution, ROCm transport, real-checkpoint compatibility, or measured cluster throughput.

ExpertMesh v0.2.0 — Kimi K2 target integration

ExpertMesh v0.2.0 — Kimi K2 target integration path

This release turns ExpertMesh from a generic stationary-expert reference into a strict target path
for the official open-weight moonshotai/Kimi-K2-Thinking family.

Included

• A fail-closed Kimi K2 configuration and Safetensors inventory adapter.
• Immutable checkpoint manifests and deterministic stationary-expert owner plans.
• A checked-in target profile for the documented 61-layer, 384-expert, top-8 MLA/SwiGLU design.
• Paged KV-cache allocation and one-token decode semantic oracle.
• Bounded continuous batching, runtime counters/latencies, and retry/cooldown failure semantics.
• A fused packed-INT4 SwiGLU HIP correctness ABI, source, and build integration.
• A K2 artifact-admission command: expertmesh-kimi-k2-inspect.

Validation

• 18 CPU unit and integration tests pass.
• Ruff static checks pass.
• Source distribution and wheel build successfully.
• GitHub Actions cpu-reference passed for pull request #4.

Important compatibility and performance boundary

v0.2.0 does not certify that Kimi K2 runs on ExpertMesh hardware. It intentionally refuses
incompatible checkpoint artifacts, but target compatibility remains gated on an official full
checkpoint, tokenizer/chat-template parity, MLA/RoPE/reference-output fixtures, native INT4 packing
verification, and ROCm GPU results.

Likewise, the HIP fused kernel is a correctness fallback—not a benchmarked MFMA implementation—and
TCP is a validated process-boundary transport, not GPU-direct RoCE. The documented release gates
must pass before either performance label is used.

Operator starting point

Read docs/KIMI_K2_TARGET.md, retain the official model license and revision, then run:

expertmesh-kimi-k2-inspect /models/Kimi-K2-Thinking --workers 32 --write-manifest

The command validates before GPU allocation and writes an immutable deployment manifest and owner
plan. See the release gates before treating any deployment as production-ready.

ExpertMesh v0.1.0 — Phase 1 Stationary-Expert Fabric

ExpertMesh Phase 1

The first public release of ExpertMesh introduces a model-aware, low-cost inference architecture for configurable trillion-parameter sparse mixture-of-experts models. Instead of streaming multi-terabyte weights through CXL or requiring a proprietary scale-up domain, the design keeps quantized experts stationary in distributed HBM and transports only compact activations.

Core architecture

• Reference fabric: 8 refurbished four-GPU EPYC nodes with 32 AMD Instinct MI210 64 GB accelerators.
• Aggregate capacity: 2 TiB HBM with 51.2 TB/s theoretical memory bandwidth.
• Transport: local Infinity Fabric where available and dual 100/200 GbE RoCE between nodes.
• Placement: model profiles distribute experts deterministically across stationary owners.
• Configuration: parameter count, expert count, top-k, packaged weight size, and active-weight traffic are explicit inputs.
• Estimated acquisition range: $135,000–$235,000, excluding engineering labor and site-specific infrastructure.

Example sizing profile

The checked-in Phase 1 profile uses the publicly announced characteristics of Kimi K3 as one demanding example: approximately 2.8T parameters, 896 experts, top-16 routing, and packed four-bit weights. This exercises the generic planner at trillion-parameter scale; it does not make ExpertMesh specific to that model.

Included implementation

• HIP routed-expert correctness kernel for signed INT4 weights with FP16 group scales.
• Stable C ABI and documented [expert, output, packed_input] runtime layout.
• NumPy quantizer, dequantizer, routed expert reference, and weighted top-k sparse MoE reduction.
• Expert-owner mapping and activation grouping that preserve token and top-k slot identity.
• Capacity planner and explicitly labeled HBM-bandwidth roofline calculator.
• Synthetic AMD GPU smoke test and six passing CPU correctness/planning tests.
• Architecture diagrams, activation transport contract, detailed BOM, checkpoint release gates, sources, and roadmap.

Validation

The release commit passes:

• pytest -q — 6 tests passed.
• ruff check src tests — passed.
• Package build and Python bytecode compilation — passed.
• Planner and roofline CLI execution — passed.
• Architecture SVG XML validation — passed.
• GitHub Actions CPU reference workflow — passed.
• Public-safety scan for common credential/private-key patterns — no matches.

Performance boundary

The published 1,024–1,707 aggregate token/s range is an ideal memory-bandwidth roofline for an example profile assuming 30–50 GB of active weights per token. It is not a measured benchmark and excludes attention work, communication, routing imbalance, KV traffic, dequantization, synchronization, and kernel inefficiency.

The HIP source and GPU smoke test require a separate Linux/ROCm system and have not been executed by the CPU-only CI runner. No target-model compatibility claim is made until its immutable checkpoint, tensor layout, quantization semantics, router behavior, attention operators, residual behavior, reference runtime, and license pass the documented release gates.

Next milestones

1. Validate the two-node activation-only RoCE path.
2. Adapt immutable open checkpoints without decode-time repacking.
3. Replace the correctness GEMV with grouped MFMA GEMM and fused expert projections.
4. Add GPU-resident dispatch queues, co-activation-aware placement, and bounded hot-expert replication.
5. Publish reproducible latency, throughput, energy-per-token, and fault-injection results.

See the README, CHANGELOG.md, and docs/ROADMAP.md for the full design and scope.