Spare 1-D norm weights from a blanket --type #1861
There are no files selected for viewing
| Original file line number | Diff line number | Diff line change |
|---|---|---|
|
|
@@ -1511,6 +1511,18 @@ bool ModelLoader::tensor_should_be_converted(const TensorStorage& tensor_storage | |
| if (type != GGML_TYPE_COUNT) { | ||
| if (ggml_is_quantized(type) && tensor_storage.ne[0] % ggml_blck_size(type) != 0) { | ||
| // Pass, do not convert | ||
| } else if (ggml_is_quantized(type) && tensor_storage.n_dims <= 1) { | ||
| // Pass, do not convert. A 1-D weight is a per-channel scale (LayerNorm/RMSNorm gain), | ||
| // never a matmul weight, so quantizing it buys almost nothing and costs a lot: every | ||
|
Comment on lines
+1515
to
+1516
There was a problem hiding this comment.
Correct the claim that every one-dimensional weight is a normalization gain: this repository also creates one-dimensional learned class embeddings ( AGENTS.md reference: AGENTS.md:L129-L132 Useful? React with 👍 / 👎. |
||
| // channel of the block shares one scale and one min, and a gain vector has no reason | ||
| // to be locally smooth. Until now these survived only by accident, when their length | ||
| // did not divide the block size (FLUX q_norm/k_norm are [128] and 128 % 256 != 0) or | ||
| // when a name rule above happened to match. A model whose norms DO divide the block | ||
| // size, such as MiniMax-H3 with [5376] and 5376 % 256 == 0, had 106 norm scales | ||
| // crushed to 4 bits by a blanket --type. The result still loads and still renders a | ||
| // plausible image, so a "does it run" check passes it, while measured against a bf16 | ||
| // render of the same prompt and seed it is destroyed: PSNR 9.87 / SSIM 0.074 / | ||
| // LPIPS 0.981, against 22.22 / 0.841 / 0.292 with this rule in place. | ||
|
Comment on lines
+1522
to
+1525
There was a problem hiding this comment.
Remove the model-specific history and PSNR/SSIM/LPIPS results from this comment: these non-reproducible measurements document this particular change rather than an enduring constraint and will become misleading as models or quantizers evolve. Keep only the invariant that quantizing one-dimensional gain vectors damages quality; the detailed evidence belongs in the commit description, documentation, or a regression test. AGENTS.md reference: AGENTS.md:L127-L134 Useful? React with 👍 / 👎. |
||
| } else if (ends_with(name, ".bias")) { | ||
| // Pass, do not convert | ||
| } else if (ends_with(name, ".scale")) { | ||
|
|
||
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Do not use
TensorStorage::n_dimshere as proof that a GGUF tensor is logically one-dimensional.src/model_io/gguf_io.cpppopulates it withggml_n_dims(dummy), which drops a trailing singleton dimension, so a real 2-D matmul such as Krea2'sLinear(text_layers, 1)projector is read back withn_dims == 1and silently excluded from a requested blanket quantization. This makes GGUF-to-GGUF conversion or runtime--typehandling depend on the input container rather than the tensor's logical shape.AGENTS.md reference: AGENTS.md:L178-L180
Useful? React with 👍 / 👎.