Skip to content

b11735

  • b11735
  • 098ce52
  • Verified

    This commit was created on GitHub.com and signed with GitHub’s verified signature.
  • Choose a tag to compare

  • b11735
  • 098ce52
  • Choose a tag to compare

  • Verified

    This commit was created on GitHub.com and signed with GitHub’s verified signature.
@ggerganov ggerganov tagged this 19 Oct 08:55
The fused moe_reduce kernel computes dst = sum(experts[e] * weights[e]) in a
single dispatch, reading experts and weights while writing dst. The graph
allocator computes tensor liveness on the unfused graph, where the inputs are
fully consumed by earlier nodes of the fused group, so it can legally recycle
the input memory as dst. In the fused kernel this becomes a
threadgroup-vs-threadgroup race that produces rare non-deterministic wrong
results (observed with test-llama-archs -a kolibri on the meta backend).

Reject the fusion when dst overlaps either input, same guard as the
mul_mat_add fusion. The unfused MUL + ADD path is always correct.

Assisted-by: pi:llama.cpp/ggml-org/GLM-5.3-Flash-GGUF (2x DGX Spark (RDMA, TP=2))
Assets 2
Loading