Repository navigation
Replies: 2 comments 2 replies
|
Maybe due to an unsupported interaction between The cache expects the selected expert IDs to be computed in an earlier scheduler split. It deliberately aborts when the expert selection and the cache lookup end up in the same split, which is the check you're hitting in Disabling KV offload changes the backend placement of the attention graph and appears to collapse those operations into the same split. That explains why the same preset works with The MoE cache was only merged in #29887 on October 7. Its review also noted that the cache planner and remapping path had no automated regression coverage. For now, the practical workarounds seem to be keeping This looks worth treating as a bug in the new MoE cache path. It would also help to include the exact build hash from |
|
This assertion triggers because the GPU MoE expert cache ( Why It HappensWhen When you set
How to Fix / Workaround It
If this explains and resolves the crash for your setup, please click Mark as answer! |
Uh oh!
There was an error while loading. Please reload this page.
Error message is:
app/src/llama-moe-cache.cpp:543: the experts of layer 0 are selected in the same split as their MoE cache lookupDoes anyone have an idea how to fix this? It all works fine with kvo=on, but I'd like to test performance like this
All reactions