Skip to content

test: direct unit suite for model_calib CPU orchestration (42 tests)#1906

Closed
arham766 wants to merge 2 commits into
NVIDIA:mainfrom
arham766:tests/test-model-calib
Closed

test: direct unit suite for model_calib CPU orchestration (42 tests)#1906
arham766 wants to merge 2 commits into
NVIDIA:mainfrom
arham766:tests/test-model-calib

Conversation

@arham766

@arham766 arham766 commented Jul 5, 2026

Copy link
Copy Markdown
Contributor

What does this PR do?

Type of change: new tests

Part of the unit-coverage initiative in #1902 (one PR per module; this one is self-contained). Covers: max_calibrate weight/input amax math and running-max accumulation, enable/finish stats choreography incl. use_constant_amax, tied-module single-visit, mse_calibrate refinement (asserted to genuinely differ from max), smoothquant per-channel-to-per-tensor conversion invariants, awq entry-point guards, svd properties, shared-states plumbing.

Usage

N/A — tests only.

Testing

Suite is hermetic (CPU-only, no network, deterministic, <3s) and passes locally; the full collectable tests/unit tree passes with it (1999 passed). The suite was adversarially reviewed and verified to kill seeded mutations of the target module (details in #1902).

Before your PR is "Ready for review"

  • Is this change backward compatible?: ✅
  • If you copied code from any other sources or added a new PIP dependency, did you follow guidance in CONTRIBUTING.md: N/A
  • Did you write any new necessary tests?: ✅
  • Did you update Changelog?: N/A
  • Did you get Claude approval on this PR?: N/A (external contributor)

Additional Information

Issue: #1902

Summary by CodeRabbit

  • Tests
    • Added a new unit test suite covering quantization calibration orchestration, including CPU-testable weight-only calibration (no forward loop), max_calibrate accumulation, and input activation amax consistency across batches.
    • Validated shared-state handling for weight amax grouping (default and custom matching), distributed-sync flag equivalence in single-process runs, and calibration state restoration/cleanup across forward passes.
    • Extended coverage for MSE calibration, SmoothQuant, and AWQ variants (missing/unknown algorithm handling), plus SVD-related and helper/iteration edge cases.

Hermetic CPU-only unit tests; adversarially reviewed and verified to
kill seeded mutations of the target module. Part of the coverage
initiative in NVIDIA#1902.

Signed-off-by: arham766 <arhamislam766@yahoo.com>
@arham766 arham766 requested a review from a team as a code owner July 5, 2026 20:10
@copy-pr-bot

copy-pr-bot Bot commented Jul 5, 2026

Copy link
Copy Markdown

This pull request requires additional validation before any workflows can run on NVIDIA's runners.

Pull request vetters can view their responsibilities here.

Contributors can view more details about this message here.

@coderabbitai

coderabbitai Bot commented Jul 5, 2026

Copy link
Copy Markdown
Contributor

Review Change Stack

No actionable comments were generated in the recent review. 🎉

ℹ️ Recent review info
⚙️ Run configuration

Configuration used: Path: .coderabbit.yaml

Review profile: CHILL

Plan: Enterprise

Run ID: 0cd5ebcf-27b1-4d10-a2cf-0f12a3d5fb9e

📥 Commits

Reviewing files that changed from the base of the PR and between ea3a566 and 1414aed.

📒 Files selected for processing (1)
  • tests/unit/torch/quantization/test_model_calib.py
🚧 Files skipped from review as they are similar to previous changes (1)
  • tests/unit/torch/quantization/test_model_calib.py

📝 Walkthrough

Walkthrough

This PR adds a new unit test module for modelopt.torch.quantization.model_calib, covering calibration orchestration, stats collection lifecycle, entry points, and helper utilities.

Changes

Model calibration test coverage

Layer / File(s) Summary
Test module setup and helpers
tests/unit/torch/quantization/test_model_calib.py
Adds licensing header, imports, config constants, and small model/util helpers shared across the test suite.
max_calibrate orchestration tests
tests/unit/torch/quantization/test_model_calib.py
Tests weight-only vs activation calibration, batch-driven amax computation, running-max accumulation, distributed-sync equivalence, and shared weight_global_amax grouping behavior.
Stats collection choreography and aliasing tests
tests/unit/torch/quantization/test_model_calib.py
Tests enable_stats_collection and finish_stats_collection mode toggling, amax loading/restoration, no-data behavior, calibrator-less quantizers, and aliased-module deduplication in weight_only_quantize.
mse_calibrate and _mse_quant_func tests
tests/unit/torch/quantization/test_model_calib.py
Tests weight amax refinement bounds and determinism for mse_calibrate, plus temporary trial amax application and restoration in _mse_quant_func.
smoothquant entry-point tests
tests/unit/torch/quantization/test_model_calib.py
Tests the forward_loop requirement, per-tensor smoothing conversion, and warnings or skips for unsupported or uncalibrated linears.
AWQ entry-point tests
tests/unit/torch/quantization/test_model_calib.py
Tests awq_lite warning behavior without forward_loop, awq_clip raising without forward_loop, and awq no-op behavior for unknown algorithm strings.
Pre-quant-scale and svd helper tests
tests/unit/torch/quantization/test_model_calib.py
Tests pre-quant-scale attribute deletion and assertions, plus svd reconstruction accuracy and rank-dependent padding or warning behavior.
Pure helper predicate tests
tests/unit/torch/quantization/test_model_calib.py
Tests AWQ block-size lookup, routed-expert detection, EP-sync predicate behavior, and leaf-quantizer iteration over SequentialQuantizer.

Estimated code review effort: 2 (Simple) | ~15 minutes

Possibly related issues

Suggested labels: tests, quantization

Suggested reviewers: None identified.

🚥 Pre-merge checks | ✅ 6
✅ Passed checks (6 passed)
Check name Status Explanation
Description Check ✅ Passed Check skipped - CodeRabbit’s high-level summary is enabled.
Title check ✅ Passed The title clearly matches the PR’s main change: adding a CPU-only unit test suite for model_calib orchestration.
Docstring Coverage ✅ Passed No functions found in the changed files to evaluate docstring coverage. Skipping docstring coverage check.
Linked Issues check ✅ Passed Check skipped because no linked issues were found for this pull request.
Out of Scope Changes check ✅ Passed Check skipped because no linked issues were found for this pull request.
Security Anti-Patterns ✅ Passed PASS: PR only adds a test file; no modelopt/examples code, dependency changes, or prohibited patterns (torch.load/allow_pickle/trust_remote_code/eval/nosec) were present.
✨ Finishing Touches
🧪 Generate unit tests (beta)
  • Create PR with unit tests

Comment @coderabbitai help to get the list of available commands.

@coderabbitai coderabbitai Bot left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

🧹 Nitpick comments (1)
tests/unit/torch/quantization/test_model_calib.py (1)

1-2: 📐 Maintainability & Code Quality | 🔵 Trivial | 💤 Low value

Stale copyright year in a newly-added file.

This is a new file being added in this PR, but the header states Copyright (c) 2024. Likely copied from an older template without updating the year.

✏️ Suggested fix
-# SPDX-FileCopyrightText: Copyright (c) 2024 NVIDIA CORPORATION & AFFILIATES. All rights reserved.
+# SPDX-FileCopyrightText: Copyright (c) 2026 NVIDIA CORPORATION & AFFILIATES. All rights reserved.
🤖 Prompt for AI Agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.

In `@tests/unit/torch/quantization/test_model_calib.py` around lines 1 - 2, The
new test file’s copyright header is stale and should be updated to the current
year. In test_model_calib.py, replace the 2024 copyright year in the SPDX header
with the appropriate current year so the new file matches the project’s standard
for newly added sources.
🤖 Prompt for all review comments with AI agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.

Nitpick comments:
In `@tests/unit/torch/quantization/test_model_calib.py`:
- Around line 1-2: The new test file’s copyright header is stale and should be
updated to the current year. In test_model_calib.py, replace the 2024 copyright
year in the SPDX header with the appropriate current year so the new file
matches the project’s standard for newly added sources.

ℹ️ Review info
⚙️ Run configuration

Configuration used: Path: .coderabbit.yaml

Review profile: CHILL

Plan: Enterprise

Run ID: 494209a4-92a7-41f7-abb4-d12149944967

📥 Commits

Reviewing files that changed from the base of the PR and between b96a785 and ea3a566.

📒 Files selected for processing (1)
  • tests/unit/torch/quantization/test_model_calib.py

Signed-off-by: arham766 <arhamislam766@yahoo.com>
@arham766

arham766 commented Jul 6, 2026

Copy link
Copy Markdown
Contributor Author

Consolidated into #1927 per maintainer feedback in #1902 — the suite was trimmed to only the lines codecov reports uncovered, with parametrization clusters deduplicated. Closing in favor of that PR.

@arham766 arham766 closed this Jul 6, 2026
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant