Skip to content

fix(recipes): sync INT4 blockwise recipe defaults#1836

Closed
realAsma wants to merge 1 commit into
mainfrom
beebot/nvbug-6311597
Closed

fix(recipes): sync INT4 blockwise recipe defaults#1836
realAsma wants to merge 1 commit into
mainfrom
beebot/nvbug-6311597

Conversation

@realAsma

@realAsma realAsma commented Jun 26, 2026

Copy link
Copy Markdown
Contributor

What does this PR do?

Fixes the public general/ptq/int4_blockwise_weight_only recipe added in PR #1172 by composing it from the shared recipe units instead of carrying a manually duplicated quantizer exclusion list.

The duplicated list had drifted from configs/ptq/units/default_disabled_quantizers, so the public INT4 recipe missed newer standard exclusions such as linear-attention projections, VLM vision/projector modules, and nn.Embedding. Recipe mode can apply directly to the broader model path, so those missing exclusions can quantize modules that the shared preset/qformat path avoids.

NVBug: 6311597

Testing

  • python_pwd tools/precommit/check_modelopt_recipes.py modelopt_recipes/general/ptq/int4_blockwise_weight_only.yaml
  • pytest_pwd tests/unit/recipe/test_loader.py::test_load_recipe_all_builtins tests/unit/recipe/test_loader.py::test_general_ptq_yaml_matches_config_dicts -q
  • pytest_pwd tests/unit/recipe/test_presets.py -q
  • Direct assertion that general/ptq/int4_blockwise_weight_only includes current standard exclusions for *linear_attn.in_proj_a*, *linear_attn.in_proj_b*, VLM modules, and nn.Embedding.

Notes

The NVBug page redirects to Microsoft sign-in from my shell, so this fix is based on the Slack-linked PR diff anchor resolving to modelopt_recipes/general/ptq/int4_blockwise_weight_only.yaml and local recipe/preset drift analysis.

Summary by CodeRabbit

  • New Features

    • Added a new INT4 blockwise weight-only PTQ recipe for easier low-bit quantization setup.
    • Streamlined recipe configuration so shared defaults are reused automatically.
  • Bug Fixes

    • Improved recipe loading and validation for cases where some configuration entries are not present.
    • Updated consistency checks to handle these configurations more reliably.

Signed-off-by: realAsma <akuriparambi@nvidia.com>
@realAsma
realAsma requested review from a team as code owners June 26, 2026 21:24
@realAsma
realAsma requested a review from shengliangxu June 26, 2026 21:24
@coderabbitai

coderabbitai Bot commented Jun 26, 2026

Copy link
Copy Markdown
Contributor

Review Change Stack

No actionable comments were generated in the recent review. 🎉

ℹ️ Recent review info
⚙️ Run configuration

Configuration used: Path: .coderabbit.yaml

Review profile: CHILL

Plan: Enterprise

Run ID: 4201f0af-ee20-414d-9249-4be696da4efc

📥 Commits

Reviewing files that changed from the base of the PR and between 33bfa8b and 8a5cf80.

📒 Files selected for processing (2)
  • modelopt_recipes/general/ptq/int4_blockwise_weight_only.yaml
  • tests/unit/recipe/test_loader.py

📝 Walkthrough

Walkthrough

The INT4 blockwise weight-only PTQ recipe now assembles its quantization config from imported fragments instead of an inline disable list. Loader tests add the recipe, allow mappings without a kv config, and conditionally build the expected quant config.

Changes

INT4 blockwise PTQ recipe

Layer / File(s) Summary
Recipe imports and quantizer overrides
modelopt_recipes/general/ptq/int4_blockwise_weight_only.yaml
imports now assembles quantize.quant_cfg from base_disable_all, int4_per_block for *weight_quantizer, *input_quantizer disabled, and default_disabled_quantizers, replacing the inline disable list.
Loader test updates for the new recipe
tests/unit/recipe/test_loader.py
Adds the recipe to built-in PTQ coverage, maps its YAML with kv_cfg_name=None, and makes the YAML/config comparison skip kv_cfg concatenation when no kv config exists.

Estimated code review effort

🎯 2 (Simple) | ⏱️ ~10 minutes

Possibly related PRs

  • NVIDIA/Model-Optimizer#1826: Updates default_disabled_quantizers.yaml, which is imported by the new INT4 blockwise weight-only recipe.

Suggested reviewers

  • shengliangxu
  • juhi10071998
  • kevalmorabia97
  • cjluo-nv
🚥 Pre-merge checks | ✅ 6
✅ Passed checks (6 passed)
Check name Status Explanation
Description Check ✅ Passed Check skipped - CodeRabbit’s high-level summary is enabled.
Title check ✅ Passed The title clearly matches the main change: syncing the INT4 blockwise recipe with shared default exclusions.
Docstring Coverage ✅ Passed No functions found in the changed files to evaluate docstring coverage. Skipping docstring coverage check.
Linked Issues check ✅ Passed Check skipped because no linked issues were found for this pull request.
Out of Scope Changes check ✅ Passed Check skipped because no linked issues were found for this pull request.
Security Anti-Patterns ✅ Passed Only a YAML recipe and test changed; no banned patterns (weights_only=False, allow_pickle=True, trust_remote_code=True, eval/exec, # nosec, or new deps) were found.
✨ Finishing Touches
📝 Generate docstrings
  • Create stacked PR
  • Commit on current branch
🧪 Generate unit tests (beta)
  • Create PR with unit tests
  • Commit unit tests in branch beebot/nvbug-6311597

Comment @coderabbitai help to get the list of available commands.

@codecov

codecov Bot commented Jun 26, 2026

Copy link
Copy Markdown

Codecov Report

✅ All modified and coverable lines are covered by tests.
✅ Project coverage is 77.29%. Comparing base (33bfa8b) to head (8a5cf80).

Additional details and impacted files
@@           Coverage Diff           @@
##             main    #1836   +/-   ##
=======================================
  Coverage   77.29%   77.29%           
=======================================
  Files         513      513           
  Lines       56920    56920           
=======================================
  Hits        43997    43997           
  Misses      12923    12923           
Flag Coverage Δ
unit 54.61% <ø> (-0.02%) ⬇️

Flags with carried forward coverage won't be shown. Click here to find out more.

☔ View full report in Codecov by Harness.
📢 Have feedback on the report? Share it here.

🚀 New features to boost your workflow:
  • ❄️ Test Analytics: Detect flaky tests, report on failures, and find test suite problems.

@realAsma
realAsma requested a review from meenchen June 26, 2026 22:03
@realAsma
realAsma marked this pull request as draft June 26, 2026 22:05
@copy-pr-bot

copy-pr-bot Bot commented Jun 26, 2026

Copy link
Copy Markdown

Auto-sync is disabled for draft pull requests in this repository. Workflows must be run manually.

Contributors can view more details about this message here.

@realAsma

Copy link
Copy Markdown
Contributor Author

Moved this diff into draft PR #1838 per owner request; closing this separate draft PR.

@github-actions

Copy link
Copy Markdown
Contributor
PR Preview Action v1.8.1
Preview removed because the pull request was closed.
2026-06-26 22:29 UTC

realAsma added a commit that referenced this pull request Jun 27, 2026
## Summary
- Skip pre-quant LayerNorm fusion when the representative input
quantizer has no `_pre_quant_scale` buffer.
- Add CPU unit coverage for the no-scale weight-only path and the
existing fusion/removal behavior when pre-quant scales are present.
- Compose the public `general/ptq/int4_blockwise_weight_only` recipe
from the shared recipe units so it stays aligned with
`INT4_BLOCKWISE_WEIGHT_ONLY_CFG`.

## Context
NVBug: https://nvbugspro.nvidia.com/bug/6311597

NVBug 6311597 reports a deterministic HF export crash for
`general/ptq/int4_blockwise_weight_only` on Llama-3.1-8B. That
weight-only recipe disables input quantization and skips calibration, so
`_pre_quant_scale` is not registered, but export still reaches
`fuse_prequant_layernorm` through the AWQ-like format path.

This PR also carries the recipe-sync diff that was previously in draft
PR #1836; PR #1836 has been closed after moving that diff here.

## Tests
- `pre-commit run --files modelopt/torch/export/quant_utils.py
tests/unit/torch/export/test_unified_export_hf.py
modelopt_recipes/general/ptq/int4_blockwise_weight_only.yaml
tests/unit/recipe/test_loader.py`
- `pytest_pwd
tests/unit/torch/export/test_unified_export_hf.py::test_fuse_prequant_layernorm_skips_modules_without_pre_quant_scale
tests/unit/recipe/test_loader.py::test_load_recipe_all_builtins
tests/unit/recipe/test_loader.py::test_general_ptq_yaml_matches_config_dicts
tests/unit/recipe/test_presets.py -q` -> 29 passed

Full GB10/Llama repro was not run locally.


<!-- This is an auto-generated comment: release notes by coderabbit.ai
-->

## Summary by CodeRabbit

* **Bug Fixes**
* Improved handling of layer norm fusion so it safely skips cases where
pre-quantization scale data is unavailable, avoiding unexpected errors.
* Updated the layer norm fusion flow to correctly apply scaling when
pre-quantization data is present.

* **Tests**
* Expanded coverage for export and recipe loading scenarios, including
cases with and without pre-quantization scale data.
  * Added validation for the new int4 blockwise weight-only recipe.

<!-- end of auto-generated comment: release notes by coderabbit.ai -->

---------

Signed-off-by: realAsma <akuriparambi@nvidia.com>
kevalmorabia97 pushed a commit that referenced this pull request Jul 3, 2026
- Skip pre-quant LayerNorm fusion when the representative input
quantizer has no `_pre_quant_scale` buffer.
- Add CPU unit coverage for the no-scale weight-only path and the
existing fusion/removal behavior when pre-quant scales are present.
- Compose the public `general/ptq/int4_blockwise_weight_only` recipe
from the shared recipe units so it stays aligned with
`INT4_BLOCKWISE_WEIGHT_ONLY_CFG`.

NVBug: https://nvbugspro.nvidia.com/bug/6311597

NVBug 6311597 reports a deterministic HF export crash for
`general/ptq/int4_blockwise_weight_only` on Llama-3.1-8B. That
weight-only recipe disables input quantization and skips calibration, so
`_pre_quant_scale` is not registered, but export still reaches
`fuse_prequant_layernorm` through the AWQ-like format path.

This PR also carries the recipe-sync diff that was previously in draft
PR #1836; PR #1836 has been closed after moving that diff here.

- `pre-commit run --files modelopt/torch/export/quant_utils.py
tests/unit/torch/export/test_unified_export_hf.py
modelopt_recipes/general/ptq/int4_blockwise_weight_only.yaml
tests/unit/recipe/test_loader.py`
- `pytest_pwd
tests/unit/torch/export/test_unified_export_hf.py::test_fuse_prequant_layernorm_skips_modules_without_pre_quant_scale
tests/unit/recipe/test_loader.py::test_load_recipe_all_builtins
tests/unit/recipe/test_loader.py::test_general_ptq_yaml_matches_config_dicts
tests/unit/recipe/test_presets.py -q` -> 29 passed

Full GB10/Llama repro was not run locally.

<!-- This is an auto-generated comment: release notes by coderabbit.ai
-->

* **Bug Fixes**
* Improved handling of layer norm fusion so it safely skips cases where
pre-quantization scale data is unavailable, avoiding unexpected errors.
* Updated the layer norm fusion flow to correctly apply scaling when
pre-quantization data is present.

* **Tests**
* Expanded coverage for export and recipe loading scenarios, including
cases with and without pre-quantization scale data.
  * Added validation for the new int4 blockwise weight-only recipe.

<!-- end of auto-generated comment: release notes by coderabbit.ai -->

---------

Signed-off-by: realAsma <akuriparambi@nvidia.com>
kevalmorabia97 pushed a commit that referenced this pull request Jul 3, 2026
- Skip pre-quant LayerNorm fusion when the representative input
quantizer has no `_pre_quant_scale` buffer.
- Add CPU unit coverage for the no-scale weight-only path and the
existing fusion/removal behavior when pre-quant scales are present.
- Compose the public `general/ptq/int4_blockwise_weight_only` recipe
from the shared recipe units so it stays aligned with
`INT4_BLOCKWISE_WEIGHT_ONLY_CFG`.

NVBug: https://nvbugspro.nvidia.com/bug/6311597

NVBug 6311597 reports a deterministic HF export crash for
`general/ptq/int4_blockwise_weight_only` on Llama-3.1-8B. That
weight-only recipe disables input quantization and skips calibration, so
`_pre_quant_scale` is not registered, but export still reaches
`fuse_prequant_layernorm` through the AWQ-like format path.

This PR also carries the recipe-sync diff that was previously in draft
PR #1836; PR #1836 has been closed after moving that diff here.

- `pre-commit run --files modelopt/torch/export/quant_utils.py
tests/unit/torch/export/test_unified_export_hf.py
modelopt_recipes/general/ptq/int4_blockwise_weight_only.yaml
tests/unit/recipe/test_loader.py`
- `pytest_pwd
tests/unit/torch/export/test_unified_export_hf.py::test_fuse_prequant_layernorm_skips_modules_without_pre_quant_scale
tests/unit/recipe/test_loader.py::test_load_recipe_all_builtins
tests/unit/recipe/test_loader.py::test_general_ptq_yaml_matches_config_dicts
tests/unit/recipe/test_presets.py -q` -> 29 passed

Full GB10/Llama repro was not run locally.

<!-- This is an auto-generated comment: release notes by coderabbit.ai
-->

* **Bug Fixes**
* Improved handling of layer norm fusion so it safely skips cases where
pre-quantization scale data is unavailable, avoiding unexpected errors.
* Updated the layer norm fusion flow to correctly apply scaling when
pre-quantization data is present.

* **Tests**
* Expanded coverage for export and recipe loading scenarios, including
cases with and without pre-quantization scale data.
  * Added validation for the new int4 blockwise weight-only recipe.

<!-- end of auto-generated comment: release notes by coderabbit.ai -->

---------

Signed-off-by: realAsma <akuriparambi@nvidia.com>
Signed-off-by: Keval Morabia <28916987+kevalmorabia97@users.noreply.github.com>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants