Are there any benchmarks of the mixed Q2 Q4 model comparing to the Q2 model? DeepSeek-V4-Flash-Layers37-42Q4KExperts-OtherExpertLayersIQ2XXSGateUp-Q2KDown-AProjQ8-SExpQ8-OutQ8-chat-v2-imatrix-fixed.gguf. It was mentioned in this tweet about generating an interpreted language. Should it be supported via the download script for 128GB machines?
Ideally ds4-eval or some other eval harness would give some information on if the Q2 Q4 model is actually better for real world coding examples?
Documenting for people who want to try out Q2 Q4:
hf download antirez/deepseek-v4-gguf DeepSeek-V4-Flash-Layers37-42Q4KExperts-OtherExpertLayersIQ2XXSGateUp-Q2KDown-AProjQ8-SExpQ8-OutQ8-chat-v2-imatrix-fixed.gguf --local-dir ./gguf
ln -sf ./gguf/DeepSeek-V4-Flash-Layers37-42Q4KExperts-OtherExpertLayersIQ2XXSGateUp-Q2KDown-AProjQ8-SExpQ8-OutQ8-chat-v2-imatrix-fixed.gguf ds4flash.gguf
If you already downloaded Q2 via the download script, then to go back to Q2:
ln -sf ./gguf/DeepSeek-V4-Flash-IQ2XXS-w2Q2K-AProjQ8-SExpQ8-OutQ8-chat-v2-imatrix.gguf ds4flash.gguf
Are there any benchmarks of the mixed Q2 Q4 model comparing to the Q2 model? DeepSeek-V4-Flash-Layers37-42Q4KExperts-OtherExpertLayersIQ2XXSGateUp-Q2KDown-AProjQ8-SExpQ8-OutQ8-chat-v2-imatrix-fixed.gguf. It was mentioned in this tweet about generating an interpreted language. Should it be supported via the download script for 128GB machines?
Ideally ds4-eval or some other eval harness would give some information on if the Q2 Q4 model is actually better for real world coding examples?
Documenting for people who want to try out Q2 Q4:
If you already downloaded Q2 via the download script, then to go back to Q2: