Fix long-prefill crash caused by B12X route-pack JIT
This release moves B12X route-pack Triton compilation into model startup, before CUDA graph capture. Every power-of-two capacity through MAX_NUM_BATCHED_TOKENS is prewarmed, including both aligned and generic scalar specializations.
Cold TP=2 validation completed unique 33,966-, 36,549-, and 40,720-token prefill requests without inference-time _pack_topk_routes_prefix_kernel compilation. The warmup adds approximately 10–11 seconds per rank during model loading and no request-path work.
Container
ghcr.io/anemll/dspark-vllm-gx10:0.1.1
sha256:a83948492cf13df455170fb42885f5ef4db54fefe0feff0f841ecbff464ac9d8
The image is public, Linux/ARM64, and pins source revision 47503f8e38dadd4dededca798150db2619594fce. Version 0.1.0 remains unchanged.
Known remaining first-use compilation
Cold 65K prefill and the first decode matrix can still expose separate vLLM/CuTeDSL specializations. They are not the route-pack kernel addressed here; normal serving keeps the JIT monitor in warning mode.