[6008361][ONNX][Quantization] Clarify autotune guidance - #1989
Conversation
Signed-off-by: Gwenaelle Cunha Sergio <gcunhasergio@nvidia.com>
|
Auto-sync is disabled for draft pull requests in this repository. Workflows must be run manually. Contributors can view more details about this message here. |
📝 WalkthroughWalkthroughChangesAutotune guidance
Estimated code review effort: 2 (Simple) | ~10 minutes Suggested reviewers: 🚥 Pre-merge checks | ✅ 6✅ Passed checks (6 passed)
✨ Finishing Touches🧪 Generate unit tests (beta)
Comment |
Codecov Report✅ All modified and coverable lines are covered by tests. Additional details and impacted files@@ Coverage Diff @@
## main #1989 +/- ##
==========================================
+ Coverage 75.81% 76.36% +0.55%
==========================================
Files 518 524 +6
Lines 58574 59658 +1084
==========================================
+ Hits 44406 45559 +1153
+ Misses 14168 14099 -69
Flags with carried forward coverage won't be shown. Click here to find out more. ☔ View full report in Codecov by Harness. 🚀 New features to boost your workflow:
|
Signed-off-by: Gwenaelle Cunha Sergio <gcunhasergio@nvidia.com>
Signed-off-by: Gwenaelle Cunha Sergio <gcunhasergio@nvidia.com>
Signed-off-by: Gwenaelle Cunha Sergio <gcunhasergio@nvidia.com>
Signed-off-by: Gwenaelle Cunha Sergio <gcunhasergio@nvidia.com>
Signed-off-by: Gwenaelle Cunha Sergio <gcunhasergio@nvidia.com>
cjluo-nv
left a comment
There was a problem hiding this comment.
Bot review (bedrock-claude-opus-4-8) — DM the bot to share feedback.
Documentation/help-text-only PR (+31/-6, 5 files) clarifying when to use direct Autotune vs. calibrated ONNX PTQ with --autotune. Verified all technical claims against the code:
moq.quantize(..., autotune=True)Python API example is correct —quantize()acceptsautotune: bool = False, applicable only for int8/fp8 (consistent with the docs).- The
--autotune=<quick|default|extensive>CLI guidance matches the CLI surface. - Warning/description text additions in
9_autotune.rst,_onnx_quantization.rst, the two READMEs, and the autotune__main__.pyargparse description are internally consistent and cross-linked sensibly. - No licensing changes (the
2026copyright header inautotune/__main__.pyis pre-existing, not touched by this diff). - No prompt-injection attempts in PR title/body/diff.
No tests required for a pure docs change. Straightforward and correct.
There was a problem hiding this comment.
Warning
CodeRabbit couldn't request changes on this pull request because it doesn't have sufficient GitHub permissions.
Please grant CodeRabbit Pull requests: Read and write permission and re-run the review.
Actionable comments posted: 1
🤖 Prompt for all review comments with AI agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.
Inline comments:
In `@docs/source/guides/9_autotune.rst`:
- Around line 64-65: Update the FAQ entry near the Direct Autotune calibration
guidance to align with the workflow described in the new introduction: for
accuracy-sensitive end-to-end ONNX PTQ, perform calibration during ONNX
quantization and enable --autotune, rather than recommending calibration
separately afterward. Remove or revise the conflicting separate-calibration
statement while preserving the surrounding FAQ content.
🪄 Autofix (Beta)
Fix all unresolved CodeRabbit comments on this PR:
- Push a commit to this branch (recommended)
- Create a new PR with the fixes
ℹ️ Review info
⚙️ Run configuration
Configuration used: Path: .coderabbit.yaml
Review profile: CHILL
Plan: Enterprise
Run ID: 78f9a076-44c1-4f23-bf5a-944d8262fd16
📒 Files selected for processing (5)
docs/source/guides/9_autotune.rstdocs/source/guides/_onnx_quantization.rstexamples/onnx_ptq/README.mdexamples/onnx_ptq/autotune/README.mdmodelopt/onnx/quantization/autotune/__main__.py
| Autotune searches for Q/DQ placement schemes that improve TensorRT runtime. It does not by itself define the full calibration and quantization policy for an accuracy-sensitive deployment. For end-to-end ONNX PTQ that starts from an unquantized model, run ONNX quantization with calibration data and enable ``--autotune`` there. See the `ONNX quantization Autotune options <_onnx_quantization.html#python-m-modelopt.onnx.quantization-autotune-only-applicable-when-autotune-is-set>`_. | ||
|
|
There was a problem hiding this comment.
🎯 Functional Correctness | 🟠 Major | ⚡ Quick win
Reconcile the calibration guidance in this guide.
This new guidance correctly directs accuracy-sensitive users to calibrated PTQ with --autotune, but the FAQ at Lines 855-858 still says calibration can be performed separately after Direct Autotune. Update the FAQ so the guide presents one consistent workflow.
🤖 Prompt for AI Agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.
In `@docs/source/guides/9_autotune.rst` around lines 64 - 65, Update the FAQ entry
near the Direct Autotune calibration guidance to align with the workflow
described in the new introduction: for accuracy-sensitive end-to-end ONNX PTQ,
perform calibration during ONNX quantization and enable --autotune, rather than
recommending calibration separately afterward. Remove or revise the conflicting
separate-calibration statement while preserving the surrounding FAQ content.
What does this PR do?
Type of change: documentation
This PR clarifies when users should use ONNX quantization with Autotune enabled versus the direct Autotune entry point.
autotune=Truein the Python API and explain that it uses default Autotune settings.python -m modelopt.onnx.quantization ... --autotune=<mode>for accuracy-sensitive PTQ from an unquantized model.Usage
Testing
python -m py_compile modelopt/onnx/quantization/autotune/__main__.py.python -m sphinx -b html docs/source docs/build/html; build succeeded. Remaining warnings are from optional documentation imports and existing cross-reference labels.Before your PR is "Ready for review"
Make sure you read and follow Contributor guidelines and your commits are signed (
git commit -s -S).Make sure you read and follow the Security Best Practices (e.g. avoiding hardcoded
trust_remote_code=True,torch.load(..., weights_only=False),pickle, etc.).CONTRIBUTING.md: N/ASummary by CodeRabbit
autotune=Truesetting, expected calibration-time impact, and representative calibration data requirements.