Skip to content

b12126

  • b12126
  • 330f018
  • Verified

    This commit was created on GitHub.com and signed with GitHub’s verified signature.
  • Choose a tag to compare

  • b12126
  • 330f018
  • Choose a tag to compare

  • Verified

    This commit was created on GitHub.com and signed with GitHub’s verified signature.
@danbev danbev tagged this 05 Nov 05:29
This commit adds nvfp4 scales to qwen2's build_ffn call which are
currently passed as NULL.

The motivation for this is that a model converted to NVFP4 will
currently crash because the scales are loaded but never applied in
the FFN:
```console
llama-completion: llama.cpp/src/llama-sampler.cpp:1218:
void llama_sampler_dist_apply(llama_sampler*, llama_token_data_array*):
Assertion `found' failed.
```
The attention already applies its scales but ffn is missing them.
Assets 2
Loading