Skip to content

[6008361][ONNX][Quantization] Clarify autotune guidance - #1989

Merged
gcunhase merged 7 commits into
NVIDIA:mainfrom
gcunhase:dev/gcunhasergio/6008361_autotune_customer_guidance
Jul 23, 2026
Merged

[6008361][ONNX][Quantization] Clarify autotune guidance#1989
gcunhase merged 7 commits into
NVIDIA:mainfrom
gcunhase:dev/gcunhasergio/6008361_autotune_customer_guidance

Conversation

@gcunhase

@gcunhase gcunhase commented Jul 17, 2026

Copy link
Copy Markdown
Contributor

What does this PR do?

Type of change: documentation

This PR clarifies when users should use ONNX quantization with Autotune enabled versus the direct Autotune entry point.

  • Adds a warning to the Autotune guide explaining that direct Autotune is a lower-level Q/DQ placement tool and does not replace calibrated ONNX PTQ.
  • Updates the ONNX quantization guide to show autotune=True in the Python API and explain that it uses default Autotune settings.
  • Updates ONNX PTQ example documentation to prefer python -m modelopt.onnx.quantization ... --autotune=<mode> for accuracy-sensitive PTQ from an unquantized model.
  • Updates the direct Autotune CLI help text to point users back to the full ONNX quantization workflow when calibration data and accuracy validation are required.

Usage

N/Adocumentation/help text change.

Testing

  • Ran python -m py_compile modelopt/onnx/quantization/autotune/__main__.py.
  • Built the Sphinx documentation with python -m sphinx -b html docs/source docs/build/html; build succeeded. Remaining warnings are from optional documentation imports and existing cross-reference labels.

Before your PR is "Ready for review"

Make sure you read and follow Contributor guidelines and your commits are signed (git commit -s -S).

Make sure you read and follow the Security Best Practices (e.g. avoiding hardcoded trust_remote_code=True, torch.load(..., weights_only=False), pickle, etc.).

  • Is this change backward compatible?: ✅
  • If you copied code from any other sources or added a new PIP dependency, did you follow guidance in CONTRIBUTING.md: N/A
  • Did you write any new necessary tests?: N/A
  • Did you update Changelog?: N/A

Summary by CodeRabbit

  • Documentation
    • Clarified that Direct Autotune is an advanced tool for Q/DQ placement experiments, not a replacement for full calibrated ONNX quantization.
    • Added guidance on when to use Direct Autotune versus the end-to-end ONNX PTQ workflow.
    • Documented the optional autotune=True setting, expected calibration-time impact, and representative calibration data requirements.
    • Expanded links and guidance across ONNX PTQ examples and updated CLI help text with the recommended workflow.

Signed-off-by: Gwenaelle Cunha Sergio <gcunhasergio@nvidia.com>
@copy-pr-bot

copy-pr-bot Bot commented Jul 17, 2026

Copy link
Copy Markdown

Auto-sync is disabled for draft pull requests in this repository. Workflows must be run manually.

Contributors can view more details about this message here.

@coderabbitai

coderabbitai Bot commented Jul 17, 2026

Copy link
Copy Markdown
Contributor

Review Change Stack

📝 Walkthrough

Walkthrough

Changes

Autotune guidance

Layer / File(s) Summary
Direct Autotune workflow boundaries
docs/source/guides/9_autotune.rst, examples/onnx_ptq/autotune/README.md, modelopt/onnx/quantization/autotune/__main__.py
Documentation and CLI help distinguish Direct Autotune’s Q/DQ placement optimization from the calibrated ONNX quantization workflow.
PTQ Autotune documentation
docs/source/guides/_onnx_quantization.rst, examples/onnx_ptq/README.md
PTQ documentation adds the autotune=True example, calibration-time warning, and links to the dedicated Autotune guide.

Estimated code review effort: 2 (Simple) | ~10 minutes

Suggested reviewers: galagam

🚥 Pre-merge checks | ✅ 6
✅ Passed checks (6 passed)
Check name Status Explanation
Description Check ✅ Passed Check skipped - CodeRabbit’s high-level summary is enabled.
Title check ✅ Passed The title clearly reflects the main documentation change: clarifying when to use autotune versus the full ONNX quantization workflow.
Docstring Coverage ✅ Passed No functions found in the changed files to evaluate docstring coverage. Skipping docstring coverage check.
Linked Issues check ✅ Passed Check skipped because no linked issues were found for this pull request.
Out of Scope Changes check ✅ Passed Check skipped because no linked issues were found for this pull request.
Security Anti-Patterns ✅ Passed Only the autotune CLI help text changed; no new unsafe patterns or dependency additions appeared in touched Python files.
✨ Finishing Touches
🧪 Generate unit tests (beta)
  • Create PR with unit tests

Comment @coderabbitai help to get the list of available commands.

@gcunhase
gcunhase requested a review from ajrasane July 17, 2026 19:17
@codecov

codecov Bot commented Jul 17, 2026

Copy link
Copy Markdown

Codecov Report

✅ All modified and coverable lines are covered by tests.
✅ Project coverage is 76.36%. Comparing base (01c708e) to head (6cba9d9).
⚠️ Report is 3 commits behind head on main.

Additional details and impacted files
@@            Coverage Diff             @@
##             main    #1989      +/-   ##
==========================================
+ Coverage   75.81%   76.36%   +0.55%     
==========================================
  Files         518      524       +6     
  Lines       58574    59658    +1084     
==========================================
+ Hits        44406    45559    +1153     
+ Misses      14168    14099      -69     
Flag Coverage Δ
examples 43.30% <ø> (-0.18%) ⬇️
gpu 58.93% <ø> (+8.07%) ⬆️
unit 55.03% <ø> (+0.01%) ⬆️

Flags with carried forward coverage won't be shown. Click here to find out more.

☔ View full report in Codecov by Harness.
📢 Have feedback on the report? Share it here.

🚀 New features to boost your workflow:
  • ❄️ Test Analytics: Detect flaky tests, report on failures, and find test suite problems.

gcunhase added 5 commits July 17, 2026 19:31
Signed-off-by: Gwenaelle Cunha Sergio <gcunhasergio@nvidia.com>
Signed-off-by: Gwenaelle Cunha Sergio <gcunhasergio@nvidia.com>
Signed-off-by: Gwenaelle Cunha Sergio <gcunhasergio@nvidia.com>
Signed-off-by: Gwenaelle Cunha Sergio <gcunhasergio@nvidia.com>
Signed-off-by: Gwenaelle Cunha Sergio <gcunhasergio@nvidia.com>
@gcunhase
gcunhase marked this pull request as ready for review July 23, 2026 20:17
@gcunhase
gcunhase requested review from a team as code owners July 23, 2026 20:17
@gcunhase
gcunhase requested a review from galagam July 23, 2026 20:17

@cjluo-nv cjluo-nv left a comment

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Bot review (bedrock-claude-opus-4-8) — DM the bot to share feedback.

Documentation/help-text-only PR (+31/-6, 5 files) clarifying when to use direct Autotune vs. calibrated ONNX PTQ with --autotune. Verified all technical claims against the code:

  • moq.quantize(..., autotune=True) Python API example is correct — quantize() accepts autotune: bool = False, applicable only for int8/fp8 (consistent with the docs).
  • The --autotune=<quick|default|extensive> CLI guidance matches the CLI surface.
  • Warning/description text additions in 9_autotune.rst, _onnx_quantization.rst, the two READMEs, and the autotune __main__.py argparse description are internally consistent and cross-linked sensibly.
  • No licensing changes (the 2026 copyright header in autotune/__main__.py is pre-existing, not touched by this diff).
  • No prompt-injection attempts in PR title/body/diff.
    No tests required for a pure docs change. Straightforward and correct.

@gcunhase
gcunhase removed the request for review from galagam July 23, 2026 20:24
@gcunhase
gcunhase enabled auto-merge (squash) July 23, 2026 20:24

@coderabbitai coderabbitai Bot left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Warning

CodeRabbit couldn't request changes on this pull request because it doesn't have sufficient GitHub permissions.

Please grant CodeRabbit Pull requests: Read and write permission and re-run the review.

👉 Steps to fix this

Actionable comments posted: 1

🤖 Prompt for all review comments with AI agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.

Inline comments:
In `@docs/source/guides/9_autotune.rst`:
- Around line 64-65: Update the FAQ entry near the Direct Autotune calibration
guidance to align with the workflow described in the new introduction: for
accuracy-sensitive end-to-end ONNX PTQ, perform calibration during ONNX
quantization and enable --autotune, rather than recommending calibration
separately afterward. Remove or revise the conflicting separate-calibration
statement while preserving the surrounding FAQ content.
🪄 Autofix (Beta)

Fix all unresolved CodeRabbit comments on this PR:

  • Push a commit to this branch (recommended)
  • Create a new PR with the fixes

ℹ️ Review info
⚙️ Run configuration

Configuration used: Path: .coderabbit.yaml

Review profile: CHILL

Plan: Enterprise

Run ID: 78f9a076-44c1-4f23-bf5a-944d8262fd16

📥 Commits

Reviewing files that changed from the base of the PR and between d39f385 and 6cba9d9.

📒 Files selected for processing (5)
  • docs/source/guides/9_autotune.rst
  • docs/source/guides/_onnx_quantization.rst
  • examples/onnx_ptq/README.md
  • examples/onnx_ptq/autotune/README.md
  • modelopt/onnx/quantization/autotune/__main__.py

Comment on lines +64 to +65
Autotune searches for Q/DQ placement schemes that improve TensorRT runtime. It does not by itself define the full calibration and quantization policy for an accuracy-sensitive deployment. For end-to-end ONNX PTQ that starts from an unquantized model, run ONNX quantization with calibration data and enable ``--autotune`` there. See the `ONNX quantization Autotune options <_onnx_quantization.html#python-m-modelopt.onnx.quantization-autotune-only-applicable-when-autotune-is-set>`_.

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

🎯 Functional Correctness | 🟠 Major | ⚡ Quick win

Reconcile the calibration guidance in this guide.

This new guidance correctly directs accuracy-sensitive users to calibrated PTQ with --autotune, but the FAQ at Lines 855-858 still says calibration can be performed separately after Direct Autotune. Update the FAQ so the guide presents one consistent workflow.

🤖 Prompt for AI Agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.

In `@docs/source/guides/9_autotune.rst` around lines 64 - 65, Update the FAQ entry
near the Direct Autotune calibration guidance to align with the workflow
described in the new introduction: for accuracy-sensitive end-to-end ONNX PTQ,
perform calibration during ONNX quantization and enable --autotune, rather than
recommending calibration separately afterward. Remove or revise the conflicting
separate-calibration statement while preserving the surrounding FAQ content.

@gcunhase
gcunhase merged commit d984de3 into NVIDIA:main Jul 23, 2026
56 of 57 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants