v0.3.1: Super Fat GEMM Prefill Kernel Suite & Concurrency Scaling

v0.3.1: Super Fat GEMM Prefill Kernel Suite & Concurrency Scaling for NVIDIA DGX Spark GB10

Version 0.3.1 delivers the Super Fat GEMM prefill kernel suite for wide-layer MoE dispatch, along with formal upstream attribution compliance and stability fixes for high-concurrency serving on NVIDIA DGX Spark GB10 (sm_121 Blackwell).

What's New in v0.3.1

• Super Fat GEMM Prefill Kernel Suite (csrc/exl3_fat_gemm.cu, csrc/exl3_fat_gemm.cuh):
  • High-throughput tiled chunked prefill kernel engineered for wide-layer and high-context routed expert evaluation.
  • Performs batched matrix multiplication directly over unquantized activation states and trellis-dequantized expert weights with register-level unrolling.
  • Dispatched automatically through apply_exl3_batched_fat in src/vllm_exl3/exl3.py.
• Kernel Stability & Dispatch Fix:
  • Implements a guard for k == 4 in apply_exl3_batched_fat dispatch to prevent illegal memory indexing during 4-bit trellis tile reconstruction.
• Upstream Attribution & Notice Compliance:
  • Full third-party attribution prominently placed at the top of README.md and documented in THIRD_PARTY_NOTICES.md and NOTICE.
  • Credits to Mia's AI Lab (@MiaAI-Lab and @plotarmordev) for the routed-expert EXL3 serving path and Fat GEMM CUDA kernels (GLM-5.3-Flash-EXL3-2x-DGX-Sparks, commit 4b8d3c7).
  • Credits to ExLlamaV3 (@turboderp) for the EXL3 trellis quantization format, MCG codebook, and core dequantization math.
• Hardware Validation:
  • Fully validated on NVIDIA DGX Spark GB10 (sm_121 Blackwell) with 128 GiB Unified Memory.

---

Upstream Credits & Notices

This project builds upon foundational open-source engineering:

• ExLlamaV3: Authored by @turboderp (MIT License). Defines the EXL3 trellis format, MCG codebook, and low-bit quantization mechanics.
• Mia's AI Lab: Authored by @MiaAI-Lab and @plotarmordev (MIT License). Pioneered the routed-expert EXL3 serving path and Fat GEMM CUDA kernels.

Full license texts and commit notices are preserved in THIRD_PARTY_NOTICES.md.