ExpertMesh v0.1.0 — Phase 1 Stationary-Expert Fabric

ExpertMesh Phase 1

The first public release of ExpertMesh introduces a model-aware, low-cost inference architecture for configurable trillion-parameter sparse mixture-of-experts models. Instead of streaming multi-terabyte weights through CXL or requiring a proprietary scale-up domain, the design keeps quantized experts stationary in distributed HBM and transports only compact activations.

Core architecture

• Reference fabric: 8 refurbished four-GPU EPYC nodes with 32 AMD Instinct MI210 64 GB accelerators.
• Aggregate capacity: 2 TiB HBM with 51.2 TB/s theoretical memory bandwidth.
• Transport: local Infinity Fabric where available and dual 100/200 GbE RoCE between nodes.
• Placement: model profiles distribute experts deterministically across stationary owners.
• Configuration: parameter count, expert count, top-k, packaged weight size, and active-weight traffic are explicit inputs.
• Estimated acquisition range: $135,000–$235,000, excluding engineering labor and site-specific infrastructure.

Example sizing profile

The checked-in Phase 1 profile uses the publicly announced characteristics of Kimi K3 as one demanding example: approximately 2.8T parameters, 896 experts, top-16 routing, and packed four-bit weights. This exercises the generic planner at trillion-parameter scale; it does not make ExpertMesh specific to that model.

Included implementation

• HIP routed-expert correctness kernel for signed INT4 weights with FP16 group scales.
• Stable C ABI and documented [expert, output, packed_input] runtime layout.
• NumPy quantizer, dequantizer, routed expert reference, and weighted top-k sparse MoE reduction.
• Expert-owner mapping and activation grouping that preserve token and top-k slot identity.
• Capacity planner and explicitly labeled HBM-bandwidth roofline calculator.
• Synthetic AMD GPU smoke test and six passing CPU correctness/planning tests.
• Architecture diagrams, activation transport contract, detailed BOM, checkpoint release gates, sources, and roadmap.

Validation

The release commit passes:

• pytest -q — 6 tests passed.
• ruff check src tests — passed.
• Package build and Python bytecode compilation — passed.
• Planner and roofline CLI execution — passed.
• Architecture SVG XML validation — passed.
• GitHub Actions CPU reference workflow — passed.
• Public-safety scan for common credential/private-key patterns — no matches.

Performance boundary

The published 1,024–1,707 aggregate token/s range is an ideal memory-bandwidth roofline for an example profile assuming 30–50 GB of active weights per token. It is not a measured benchmark and excludes attention work, communication, routing imbalance, KV traffic, dequantization, synchronization, and kernel inefficiency.

The HIP source and GPU smoke test require a separate Linux/ROCm system and have not been executed by the CPU-only CI runner. No target-model compatibility claim is made until its immutable checkpoint, tensor layout, quantization semantics, router behavior, attention operators, residual behavior, reference runtime, and license pass the documented release gates.

Next milestones

1. Validate the two-node activation-only RoCE path.
2. Adapt immutable open checkpoints without decode-time repacking.
3. Replace the correctness GEMV with grouped MFMA GEMM and fused expert projections.
4. Add GPU-resident dispatch queues, co-activation-aware placement, and bounded hot-expert replication.
5. Publish reproducible latency, throughput, energy-per-token, and fault-injection results.

See the README, CHANGELOG.md, and docs/ROADMAP.md for the full design and scope.