[OMNIML-4922] Four over Six PTQ & Updating Nemotron Ultra Example#1684
Conversation
|
Note Reviews pausedIt looks like this branch is under active development. To avoid overwhelming you with review comments due to an influx of new commits, CodeRabbit has automatically paused this review. You can configure this behavior by changing the Use the following commands to manage reviews:
Use the checkboxes below for quick actions:
📝 WalkthroughWalkthroughAdds Four-Over-Six (4/6) adaptive per-block NVFP4 weight scaling: parameterizes FP8 normalization, implements M=4 vs M=6 per-block candidate generation and MSE-based selection, threads flags through quantizers and backend, provides presets/recipes, and adds unit tests. ChangesNVFP4 Four-Over-Six Adaptive Per-Block Scaling
Sequence DiagramsequenceDiagram
participant Quantizer
participant NVFP4QTensor
participant _select_four_over_six_scale
participant triton_kernel
Quantizer->>NVFP4QTensor: quantize(four_over_six flag)
NVFP4QTensor->>_select_four_over_six_scale: per-block amax, generate M4/M6 candidates
_select_four_over_six_scale->>triton_kernel: fake-quant candidates (FP8→E2M1) round-trip
triton_kernel-->>_select_four_over_six_scale: quantized reconstructions
_select_four_over_six_scale-->>NVFP4QTensor: chosen per-block scales (lower MSE)
NVFP4QTensor->>triton_kernel: static_blockwise_fp4_fake_quant(fp8_max_for_normalization)
Estimated code review effort🎯 4 (Complex) | ⏱️ ~60 minutes Possibly related PRs
Suggested labels
Suggested reviewers
🚥 Pre-merge checks | ✅ 5 | ❌ 1❌ Failed checks (1 warning)
✅ Passed checks (5 passed)
✏️ Tip: You can configure your own custom pre-merge checks in the settings. ✨ Finishing Touches📝 Generate docstrings
🧪 Generate unit tests (beta)
Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out. Comment |
|
![coderabbitai[bot]](https://avatars.githubusercontent.com/in/347564?s=60&v=4){kind=small}
There was a problem hiding this comment.
Warning
CodeRabbit couldn't request changes on this pull request because it doesn't have sufficient GitHub permissions.
Please grant CodeRabbit Pull requests: Read and write permission and re-run the review.
Actionable comments posted: 4
🤖 Prompt for all review comments with AI agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.
Inline comments:
In `@modelopt_recipes/configs/ptq/units/w4a4_nvfp4_nvfp4_four_over_six.yaml`:
- Around line 16-18: The comment wrongly states that both weight and input
quantizers are dynamic NVFP4; update it to reflect that the imported weight
config uses the static four_over_six path (i.e. static weights with
Four-Over-Six scaling) while only the input/activation quantizers remain dynamic
NVFP4. Locate the QuantizerCfgList and the reference to the four_over_six weight
config (the w4a4_nvfp4_nvfp4_four_over_six unit) and change the wording to
“static weights (four_over_six) + dynamic NVFP4 inputs/activations” or
equivalent concise phrasing.
In `@tests/unit/torch/quantization/test_nvfp4_four_over_six.py`:
- Around line 168-169: The test currently imports choices inside the test body;
move the import statement "from modelopt.torch.quantization.config import
choices" to the module top-level so import errors surface at collection time and
follow project import guidelines, or if there is a true
circular/optional/heavy-import reason, keep it inline but add a concise comment
justifying the in-function import and mark the test appropriately (e.g., a skip
or note) so reviewers understand the exception.
- Around line 116-121: The selected-scale re-cast uses the default FP8
normalization max; change the second call so sel_scale is cast with the
four-over-six normalization max (256) to match the validation path. Update the
call to NVFP4QTensor._fake_quant_to_e2m1(...,
_cast_per_block_scale_to_fp8(sel_scale, 256).float(), ...) (or pass the project
constant for the 4/6 max) so sel_scale uses the 4/6 normalization like m6_scale
and BLOCK_SIZE remains unchanged.
In
`@tools/launcher/examples/nvidia/NVIDIA-Nemotron-3-Ultra-550B-A55B-BF16/megatron_lm_ptq.yaml`:
- Line 32: Update the QUANT_CFG and export tag values to the new 4/6 preset
names: replace any occurrence of QUANT_CFG value
"huggingface/nvidia/Nemotron-3-Ultra-550B-A55B/ptq/ultra-nvfp4-max-calib" with
"huggingface/nvidia/Nemotron-3-Ultra-550B-A55B/ptq/ultra-nvfp4-46-max", and
replace export tag usages of "super-nvfp4-max-calib" with "super-nvfp4-46-max"
so task_2 artifact lookup path derived from QUANT_CFG matches; search for the
QUANT_CFG key and the literal export tag "super-nvfp4-max-calib" in the YAML
(lines around the existing QUANT_CFG, lines ~55, ~76-77, ~83) and update them
consistently.
🪄 Autofix (Beta)
Fix all unresolved CodeRabbit comments on this PR:
- Push a commit to this branch (recommended)
- Create a new PR with the fixes
ℹ️ Review info
⚙️ Run configuration
Configuration used: Path: .coderabbit.yaml
Review profile: CHILL
Plan: Enterprise
Run ID: 922a3994-8d5b-4c0d-81b4-8d59a2509a7c
📒 Files selected for processing (14)
modelopt/torch/export/layer_utils.pymodelopt/torch/kernels/quantization/gemm/fp4_kernel.pymodelopt/torch/kernels/quantization/gemm/fp4_kernel_hopper.pymodelopt/torch/quantization/backends/nvfp4_gemm.pymodelopt/torch/quantization/config.pymodelopt/torch/quantization/nn/modules/tensor_quantizer.pymodelopt/torch/quantization/qtensor/nvfp4_tensor.pymodelopt/torch/quantization/tensor_quant.pymodelopt_recipes/configs/numerics/nvfp4_four_over_six.yamlmodelopt_recipes/configs/ptq/presets/model/nvfp4_four_over_six.yamlmodelopt_recipes/configs/ptq/units/w4a4_nvfp4_nvfp4_four_over_six.yamlmodelopt_recipes/huggingface/nvidia/Nemotron-3-Ultra-550B-A55B/ptq/ultra-nvfp4-46-max.yamltests/unit/torch/quantization/test_nvfp4_four_over_six.pytools/launcher/examples/nvidia/NVIDIA-Nemotron-3-Ultra-550B-A55B-BF16/megatron_lm_ptq.yaml
Codecov Report❌ Patch coverage is
Additional details and impacted files@@ Coverage Diff @@
## main #1684 +/- ##
===========================================
+ Coverage 62.88% 76.73% +13.84%
===========================================
Files 511 511
Lines 56634 56645 +11
===========================================
+ Hits 35615 43467 +7852
+ Misses 21019 13178 -7841
Flags with carried forward coverage won't be shown. Click here to find out more. ☔ View full report in Codecov by Harness. 🚀 New features to boost your workflow:
|
There was a problem hiding this comment.
Caution
Some comments are outside the diff and can’t be posted inline due to platform limitations.
⚠️ Outside diff range comments (1)
tools/launcher/examples/nvidia/NVIDIA-Nemotron-3-Ultra-550B-A55B-BF16/megatron_lm_ptq.yaml (1)
21-21:⚠️ Potential issue | 🟡 Minor | ⚡ Quick winUpdate the pipeline note to match the active quant config tag.
Line 21 still says
super-nvfp4-max-calibwhile tasks now usenvfp4-46-max. This creates doc/config drift for operators.As per coding guidelines, “Don't repeat yourself; keep a single source of truth.”
🤖 Prompt for AI Agents
Verify each finding against current code. Fix only still-valid issues, skip the rest with a brief reason, keep changes minimal, and validate. In `@tools/launcher/examples/nvidia/NVIDIA-Nemotron-3-Ultra-550B-A55B-BF16/megatron_lm_ptq.yaml` at line 21, Update the pipeline note string to reflect the active quant config tag: replace "super-nvfp4-max-calib" with "nvfp4-46-max" in the note field (the quoted value on the line containing note: in the megatron_lm_ptq.yaml) so the documentation/config matches the actual tasks using nvfp4-46-max.Source: Coding guidelines
🤖 Prompt for all review comments with AI agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.
Outside diff comments:
In
`@tools/launcher/examples/nvidia/NVIDIA-Nemotron-3-Ultra-550B-A55B-BF16/megatron_lm_ptq.yaml`:
- Line 21: Update the pipeline note string to reflect the active quant config
tag: replace "super-nvfp4-max-calib" with "nvfp4-46-max" in the note field (the
quoted value on the line containing note: in the megatron_lm_ptq.yaml) so the
documentation/config matches the actual tasks using nvfp4-46-max.
ℹ️ Review info
⚙️ Run configuration
Configuration used: Path: .coderabbit.yaml
Review profile: CHILL
Plan: Enterprise
Run ID: 8936915e-0ac5-43c0-a688-b25075b3ccf9
📒 Files selected for processing (6)
modelopt/torch/quantization/qtensor/nvfp4_tensor.pymodelopt_recipes/configs/numerics/nvfp4_four_over_six.yamlmodelopt_recipes/configs/ptq/presets/model/nvfp4_four_over_six.yamlmodelopt_recipes/configs/ptq/units/w4a4_nvfp4_nvfp4_four_over_six.yamlmodelopt_recipes/huggingface/models/nvidia/Nemotron-3-Ultra-550B-A55B/ptq/nvfp4-46-max.yamltools/launcher/examples/nvidia/NVIDIA-Nemotron-3-Ultra-550B-A55B-BF16/megatron_lm_ptq.yaml
💤 Files with no reviewable changes (1)
- modelopt_recipes/huggingface/models/nvidia/Nemotron-3-Ultra-550B-A55B/ptq/nvfp4-46-max.yaml
🚧 Files skipped from review as they are similar to previous changes (4)
- modelopt_recipes/configs/numerics/nvfp4_four_over_six.yaml
- modelopt_recipes/configs/ptq/presets/model/nvfp4_four_over_six.yaml
- modelopt_recipes/configs/ptq/units/w4a4_nvfp4_nvfp4_four_over_six.yaml
- modelopt/torch/quantization/qtensor/nvfp4_tensor.py
|
/claude review |
Add the NVFP4_FOUR_OVER_SIX_CFG preset (scoped to max calibration) and implement 4/6 scale selection in NVFP4QTensor, normalizing the selected per-block scale with F8_E4M3_MAX_46. Wire the supporting changes through the fp4 kernels, nvfp4_gemm backend, tensor_quant, tensor_quantizer, config, and layer_utils. Add recipe/preset YAMLs and the Megatron PTQ launcher example, plus unit tests covering 4/6 quantization and config registration. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> Signed-off-by: Jennifer Chen <jennifchen@nvidia.com>
mypy defaulted to the running interpreter's version and failed to parse 3.10 match/case syntax (e.g. precisionconverter.py). Pin python_version to 3.10 so mypy parses modern syntax. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> Signed-off-by: Jennifer Chen <jennifchen@nvidia.com>
jenchen13
force-pushed
the
jennifchen/omniml-4922-four-over-six
branch
from
c3f69af to
db5497e
Compare
|
/claude review |
Signed-off-by: Jennifer Chen <jennifchen@nvidia.com>
There was a problem hiding this comment.
Warning
CodeRabbit couldn't request changes on this pull request because it doesn't have sufficient GitHub permissions.
Please grant CodeRabbit Pull requests: Read and write permission and re-run the review.
Actionable comments posted: 1
🤖 Prompt for all review comments with AI agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.
Inline comments:
In `@tests/unit/torch/quantization/test_mse_calibrator.py`:
- Line 805: Move the function-scoped import "import
modelopt.torch.quantization.nn.modules.tensor_quantizer as tqm" out of the test
helper and place it at the top of the module imports in
tests/unit/torch/quantization/test_mse_calibrator.py; update any references
inside the helper to use the top-level symbol `tqm`, and if you believe the
import must remain local due to a genuine circular or optional dependency, add a
short comment above it documenting that justification—otherwise keep it as a
standard module-level import to ensure import errors surface at test collection
time.
🪄 Autofix (Beta)
Fix all unresolved CodeRabbit comments on this PR:
- Push a commit to this branch (recommended)
- Create a new PR with the fixes
ℹ️ Review info
⚙️ Run configuration
Configuration used: Path: .coderabbit.yaml
Review profile: CHILL
Plan: Enterprise
Run ID: 930d2b15-6a15-40ad-8113-ec2383f013f5
📒 Files selected for processing (5)
tests/gpu/torch/quantization/test_nvfp4_fp8_sweep_kernel.pytests/unit/torch/quantization/test_config_validation.pytests/unit/torch/quantization/test_mse_calibrator.pytests/unit/torch/quantization/test_nvfp4_four_over_six.pytests/unit/torch/quantization/test_nvfp4_static_export_cpu.py
🚧 Files skipped from review as they are similar to previous changes (1)
- tests/unit/torch/quantization/test_nvfp4_four_over_six.py
Signed-off-by: Jennifer Chen <jennifchen@nvidia.com>
Signed-off-by: Jenny Chen <jennifchen@nvidia.com>
…ests _nvfp4_availability_check crashed with AttributeError when a non-4/6 weight quantizer had block_sizes=None (e.g. an FP8 RealQuantLinear encountered during gemm_registry matching). Use the None-safe NVFP4QTensor._is_four_over_six helper at both the availability check and the real-quantize scale path. Also import FP8_E4M3_MAX in fp4_kernel_hopper from fp4_kernel (single source), and relocate the static-quantizer 4/6 normalization-threading test from test_mse_calibrator.py to test_nvfp4_four_over_six.py (4/6 is max-calibration only and has no MSE-calibrator relationship). Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> Signed-off-by: Jennifer Chen <jennifchen@nvidia.com>
|
/claude review |
| # See the License for the specific language governing permissions and | ||
| # limitations under the License. | ||
|
|
||
| # Static (max-calibrated) NVFP4 E2M1 quantization with Four-Over-Six (4/6) adaptive per-block scale selection enabled. |
There was a problem hiding this comment.
this numerics format should not involve the calibration algorithm in the comment right?
There was a problem hiding this comment.
yes it does, it requires max calibration to choose 4 or 6 range for NVFP4
There was a problem hiding this comment.
but it is ultimately static quantizers because it stores static amaxes
| block_sizes: | ||
| -1: 16 | ||
| type: static | ||
| scale_bits: e4m3 |
There was a problem hiding this comment.
I am a little bit confused, so this 4-over-6, is it that the per-tensor scaling factor is effectively 6 bits but we use FP8 just to hold the value range?
There was a problem hiding this comment.
4 over 6 means we compute NVFP4 quantizer using +- 4 and +- 6 dynamic range, then choose the one with lower MSE. it is still the same fp8 per tensor scaling factor, just with max range of 256 instead of 448
|
/claude review changes in modelopt/ directory |
The recipe used bare-string 'cfg: configs/numerics/<name>' values, which fail
ModelOptPTQRecipe schema validation (cfg must be a QuantizerAttributeConfig dict
or list). Add a top-level imports: block and switch to 'cfg: {$import: <name>}',
matching the validated recipe convention. Verified with check_modelopt_recipes.py.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Signed-off-by: Jennifer Chen <jennifchen@nvidia.com>
The M=4 dynamic range is expressed via the MSE stop_multiplier (1.5 = 6/4) and the 256 FP8 normalization, not this constant; no source code referenced it. Drop it and the test references. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> Signed-off-by: Jennifer Chen <jennifchen@nvidia.com>
4/6 is fake-quant + export only: mtq.compress raises for 4/6, so a 4/6 module never becomes a RealQuantLinear and the in-framework NVFP4 GEMM backend is never invoked. The availability guard and weight_fp8_max selection were therefore dead code, and the Hopper kernel's fp8_max_for_normalization param was never passed. Export bakes the M=4/M=6 scale via the static path (to_quantized_weight), unaffected. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> Signed-off-by: Jennifer Chen <jennifchen@nvidia.com>
meenchen
left a comment
meenchen
left a comment
There was a problem hiding this comment.
Isn't Four over Six a subset of MSE calibration? Did you see improvement over MSE calibration?
| -1: 16 | ||
| type: static | ||
| scale_bits: e4m3 | ||
| four_over_six: true |
There was a problem hiding this comment.
This feels a bit weird. Four over Fix is a calibration method, not a numeric format. Could we bake this information into the calibration method and hyperparameters?
There was a problem hiding this comment.
it's both a calibration method and a numerics change since 4 over 6 uses 256 instead of 448 for normalizing FP8 block scales. I could move it to algorithms: section by either creating a new mse_4o6 algorithm or add four_over_six: bool under it, but the TensorQuantizer needs to be able to access this field
There was a problem hiding this comment.
I just checked and the four_over_six needs to be part of QuantizerAttributeConfig (aka here) since the TensorQuantizer needs to access the attribute
There was a problem hiding this comment.
I see, in this case, can users set four_over_six: true and max calibration at the same time?
There was a problem hiding this comment.
in theory they could, but there is a four_over_six preset that they should be using as it only works in the NVFP4 MSE case
Signed-off-by: Jennifer Chen <jennifchen@nvidia.com>
Signed-off-by: Jennifer Chen <jennifchen@nvidia.com>
|
/claude review |
![claude[bot]](https://avatars.githubusercontent.com/in/1236702?s=60&v=4){kind=small}
| amax: torch.Tensor, | ||
| global_amax: torch.Tensor | None = None, | ||
| quantize_block_scales: bool = True, | ||
| fp8_max_for_normalization: float = E4M3_MAX, |
There was a problem hiding this comment.
[IMPORTANT Compatibility] This new fp8_max_for_normalization parameter is inserted before the existing out_dtype in the exported (__all__) static_blockwise_fp4_fake_quant. Any caller that passed out_dtype positionally — e.g. static_blockwise_fp4_fake_quant(x, amax, global_amax, True, torch.float16) — now silently binds the dtype to fp8_max_for_normalization while out_dtype stays None. The PR's own test confirms the break: test_nvfp4_fp8_sweep_kernel.py had to change (x, amax, global_amax, True, x.dtype) → (x, amax, global_amax, True, E4M3_MAX, x.dtype). The same reordering applies to StaticBlockwiseFP4FakeQuantFunction.forward.
Why it matters: public function, so external/downstream positional callers break with no error (a dtype is interpreted as a float scale). Note compute_fp4_scales (just above) correctly appends its new param at the end, preserving positional compatibility — this function diverges from that.
Suggestion: for consistency and positional safety, append fp8_max_for_normalization after out_dtype here and in StaticBlockwiseFP4FakeQuantFunction.forward (and its triton call), matching compute_fp4_scales. Risk is low (niche CUDA+triton kernel), but the asymmetry is easy to avoid.
There was a problem hiding this comment.
Claude Review — NVFP4 Four-Over-Six (4/6) adaptive weight scaling
Findings: CRITICAL: 0 · IMPORTANT: 1 · SUGGESTION: 0
Summary
This PR adds NVFP4 "4/6" (Four-Over-Six) adaptive weight scaling: a four_over_six block_sizes flag that switches the FP8 per-block-scale normalization max from 448 (E4M3_MAX) to 256, with the per-block M=6/M=4 choice made by MSE weight calibration. It threads fp8_max_for_normalization through the scale-computation paths, replaces magic numbers 6.0/448.0 with named constants E2M1_MAX/E4M3_MAX, ships the NVFP4_FOUR_OVER_SIX_CFG preset + recipe YAMLs, and explicitly rejects 4/6 in mtq.compress (real quantization cannot preserve the calibration-baked amax). Test coverage is thorough.
What I verified (looks correct)
- Numerics consistency: for the default path,
fp8_max_for_normalization=448makesE4M3_MAX/448 = 1, so every touched formula reduces to the original — fully backward-compatible. For the 4/6 path I traced the export path (_cast_per_block_scale_to_fp8: amax_block256/global_amax) against the fake-quant path (scale / (scale_fp8_quant_amax/448) = amax_block256/global_amax) — they match. - Backward-pass arity:
num_args=len(ctx.needs_input_grad)correctly auto-tracks the added forward input. - Config schema:
four_over_sixadded to the block_sizes allow-list and excluded from_get_block_quant_axes_and_sizes, so it is not mistaken for a block axis. - compress guard:
NotImplementedErroron the real-quantize path is a sensible fail-fast and is tested. - Lazy/plugin: no new hard imports;
fp8_max_for_normalization(quantizer)degrades gracefully whenblock_sizesis None/missing.
Most impactful finding
- [IMPORTANT Compatibility]
fp8_max_for_normalizationis inserted beforeout_dtypein the exportedstatic_blockwise_fp4_fake_quant(andStaticBlockwiseFP4FakeQuantFunction.forward), breaking positional callers that passedout_dtypepositionally — the PR's own test had to be updated to insertE4M3_MAX.compute_fp4_scales(same file) correctly appends its new param instead. Recommend appending for consistency and positional safety.
Risk
Low. Default behavior is provably unchanged; the new path is opt-in via config and well-tested. The single IMPORTANT item is a low-blast-radius signature-ordering nit on an internal triton kernel.
Signed-off-by: Jenny Chen <jennifchen@nvidia.com>
Signed-off-by: Jenny Chen <jennifchen@nvidia.com>
|
/ok to test 40c3db6 |
…tests (#1970) ### What does this PR do? **Type of change:** documentation (+ new tests) `modelopt_recipes/ptq.md` is meant to track every shipped PTQ recipe, but recent recipe PRs did not update it. This PR brings the doc back in sync and adds unit tests so it can't drift again. **Doc sync — general recipes:** - Add `nvfp4_experts_only_input_scale1-kv_fp8_cast` (#1947) to the shipped-recipes table (now 20 recipes) and document the `input_scale1` variant (constant amax 2688 → exported NVFP4 `input_scale == 1.0`, expert activations uncalibrated). - Add a pointer that the PTQ recipes' `quantize` sections also drive QAT/QAD, with the LSQ / Dual-LSQ QAD recipes under `general/qad/` (#1884). **Doc sync — model-specific recipes:** - Add `gemma4` (algorithm override, #1690), `diffusion_gemma` (extra `*self_conditioning*` exclusion, #1707), `vit` (FP8 with attention BMM quantizers for Torch-TRT, #1569), and the qwen3_5 / qwen3_5_moe `w4a16_nvfp4_mse-fp8_attn-kv_fp8_cast` twins (#1620). - Rewrite the checkpoint-mirror section for the `huggingface/models/nvidia/` tier: Super recipe rename (`super-nvfp4.yaml` → `nvfp4-mse.yaml` / `nvfp4-max-calib.yaml`), Nano-4B GGUF Q4_K_M mirror (#1327/#1606), Ultra-550B `nvfp4-4o6` (#1684); fix stale `models/<checkpoint>` paths. **Enforcement — new `tests/unit/recipe/test_recipe_docs.py`:** 1. Every `general/ptq/*.yaml` stem must appear (backticked) in `ptq.md`. 2. Every recipe row in the shipped-recipes table must exist on disk (catches renames/removals). 3. The "All N `general/ptq/` recipes" count must match the file count. 4. Every model folder under `huggingface/` containing `ptq/*.yaml` must be mentioned in the doc. ### Usage ```bash pytest tests/unit/recipe/test_recipe_docs.py -q ``` ### Testing - `pytest tests/unit/recipe/ -q` → 218 passed (includes the 4 new tests). - Verified enforcement bites: against the pre-update `ptq.md`, 3 of the 4 new tests fail with actionable messages. - `pre-commit run --files modelopt_recipes/ptq.md tests/unit/recipe/test_recipe_docs.py` → all hooks pass. ### Before your PR is "*Ready for review*" Make sure you read and follow [Contributor guidelines](https://github.com/NVIDIA/Model-Optimizer/blob/main/CONTRIBUTING.md) and your commits are signed (`git commit -s -S`). Make sure you read and follow the [Security Best Practices](https://github.com/NVIDIA/Model-Optimizer/blob/main/SECURITY.md#security-coding-practices-for-contributors) (e.g. avoiding hardcoded `trust_remote_code=True`, `torch.load(..., weights_only=False)`, `pickle`, etc.). - Is this change backward compatible?: ✅ (docs + tests only; no runtime code changed) - If you copied code from any other sources or added a new PIP dependency, did you follow guidance in `CONTRIBUTING.md`: N/A - Did you write any new necessary tests?: ✅ — `tests/unit/recipe/test_recipe_docs.py` - Did you update [Changelog](https://github.com/NVIDIA/Model-Optimizer/blob/main/CHANGELOG.rst)?: N/A (documentation/test change) - Did you get Claude approval on this PR?: ❌ (pending) ### Additional Information Follow-up to the recipe guide added in #1662. The doc-consistency tests could also be wired into `tools/precommit/check_modelopt_recipes.py` for commit-time feedback if reviewers prefer both layers. 🤖 Generated with [Claude Code](https://claude.com/claude-code) <!-- This is an auto-generated comment: release notes by coderabbit.ai --> ## Summary by CodeRabbit * **Documentation** * Added the `nvfp4_experts_only_input_scale1-kv_fp8_cast` recipe to the general PTQ catalog. * Documented the new `input_scale1` calibration behavior and related quantization/caching details. * Expanded and reworked PTQ documentation, including updated navigation, “Beyond PTQ” guidance, new/updated model-specific recipes, and expanded checkpoint mirror entries. * Updated the general PTQ shipped-recipe count to 20. * **Tests** * Added automated consistency checks between on-disk shipped recipe YAML files and the PTQ documentation (coverage, existence, and counts). <!-- end of auto-generated comment: release notes by coderabbit.ai --> --------- Signed-off-by: Chenjie Luo <chenjiel@nvidia.com> Co-authored-by: Claude Fable 5 <noreply@anthropic.com>
6 participants
What does this PR do?
Four Over Six PTQ implementation for weight-only quantization. Four Over Six was used to produce the Nemotron 3 Ultra NVFP4 checkpoint. Also updates the Ultra PTQ example in the launcher to use this new 4/6 config
huggingface/nvidia/Nemotron-3-Ultra-550B-A55B/ptq/nvfp4-4o6Usage
Testing
Before your PR is "Ready for review"
Make sure you read and follow Contributor guidelines and your commits are signed (
git commit -s -S).Make sure you read and follow the Security Best Practices (e.g. avoiding hardcoded
trust_remote_code=True,torch.load(..., weights_only=False),pickle, etc.).CONTRIBUTING.md: ✅ / ❌ / N/AAdditional Information
Summary by CodeRabbit
New Features
Documentation
Tests
Chores