[None][fix] Fix Qwen3.5 NVFP4 weight loading by preserving weight_scales by achartier · Pull Request #13716 · NVIDIA/TensorRT-LLM

achartier · 2026-05-03T16:22:47Z

Summary by CodeRabbit

Bug Fixes
- Improved Qwen 3.5 model checkpoint loading with enhanced detection and processing of different ModelOpt checkpoint formats, providing better support for nvfp4-style checkpoints and ensuring proper handling of model weight parameters throughout the checkpoint loading process.

Description

_preprocess_modelopt_ckpt was blindly renaming all weight_scale keys to weight_scale_inv, which is correct for FP8 but breaks NVFP4 since NVFP4LinearMethod.load_weight_scales expects weight_scale. Detect NVFP4 layers by the presence of weight_scale_2 and skip the renaming for those.

Test Coverage

TestQwen3_5_379B_A17B

PR Checklist

Please review the following before submitting your PR:

PR description clearly explains what and why. If using CodeRabbit's summary, please make sure it makes sense.
PR Follows TRT-LLM CODING GUIDELINES to the best of your knowledge.
Test cases are provided for new code paths (see test instructions)
Any new dependencies have been scanned for license and vulnerabilities
CODEOWNERS updated if ownership changes
Documentation updated as needed
Update tava architecture diagram if there is a significant design change in PR.
The reviewers assigned automatically/manually are appropriate for the PR.
Please check this after reviewing the above items as appropriate for this PR.

GitHub Bot Help

To see a list of available CI bot commands, please comment /bot help.

…le keys _preprocess_modelopt_ckpt was blindly renaming all weight_scale keys to weight_scale_inv, which is correct for FP8 but breaks NVFP4 since NVFP4LinearMethod.load_weight_scales expects weight_scale. Detect NVFP4 layers by the presence of weight_scale_2 and skip the renaming for those. Signed-off-by: Aurelien Chartier <2567591+achartier@users.noreply.github.com>

achartier · 2026-05-03T16:23:03Z

/bot run

coderabbitai · 2026-05-03T16:25:46Z

No actionable comments were generated in the recent review. 🎉

ℹ️ Recent review info

⚙️ Run configuration

Configuration used: Path: .coderabbit.yaml

Review profile: CHILL

Plan: Enterprise

Run ID: d06b9e9c-a144-419c-aa28-ff4e1ee9c2d9

📥 Commits

Reviewing files that changed from the base of the PR and between 9dce3fc and 1aee667.

📒 Files selected for processing (1)

tensorrt_llm/_torch/models/checkpoints/hf/qwen3_5_weight_mapper.py

📝 Walkthrough

Walkthrough

Modified _preprocess_modelopt_ckpt to detect nvfp4-style ModelOpt checkpoint groups by collecting prefixes from keys ending in .weight_scale_2, then conditionally applies .weight_scale renaming and tensor squeezing only for non-nvfp4 prefixes.

Changes

ModelOpt Checkpoint Preprocessing Enhancement

Layer / File(s)	Summary
Prefix Detection `tensorrt_llm/_torch/models/checkpoints/hf/qwen3_5_weight_mapper.py`	Collects prefixes from checkpoint keys ending with `.weight_scale_2` to identify nvfp4 checkpoint groups within `_preprocess_modelopt_ckpt`.
Conditional Scale Processing `tensorrt_llm/_torch/models/checkpoints/hf/qwen3_5_weight_mapper.py`	Modifies `.weight_scale` handling to rename to `.weight_scale_inv` and squeeze tensor dimensions only for prefixes not in the detected nvfp4 set; nvfp4-related entries remain unmodified.

Estimated code review effort

🎯 2 (Simple) | ⏱️ ~12 minutes

🚥 Pre-merge checks | ✅ 4 | ❌ 1

❌ Failed checks (1 warning)

Check name	Status	Explanation	Resolution
Docstring Coverage	⚠️ Warning	Docstring coverage is 0.00% which is insufficient. The required threshold is 80.00%.	Write docstrings for the functions missing them to satisfy the coverage threshold.

✅ Passed checks (4 passed)

Check name	Status	Explanation
Title check	✅ Passed	The title clearly and specifically describes the main change: fixing Qwen3.5 NVFP4 weight loading by preserving weight_scales, which directly aligns with the changeset.
Description check	✅ Passed	The description explains the issue and solution clearly, includes test coverage (TestQwen3_5_379B_A17B), and has the PR checklist completed, meeting the template requirements.
Linked Issues check	✅ Passed	Check skipped because no linked issues were found for this pull request.
Out of Scope Changes check	✅ Passed	Check skipped because no linked issues were found for this pull request.

_{✏️ Tip: You can configure your own custom pre-merge checks in the settings.}

✨ Finishing Touches

🧪 Generate unit tests (beta)

Create PR with unit tests

Tip

💬 Introducing Slack Agent: The best way for teams to turn conversations into code.

Slack Agent is built on CodeRabbit's deep understanding of your code, so your team can collaborate across the entire SDLC without losing context.

Generate code and open pull requests
Plan features and break down work
Investigate incidents and troubleshoot customer tickets together
Automate recurring tasks and respond to alerts with triggers
Summarize progress and report instantly

Built for teams:

Shared memory across your entire org—no repeating context
Per-thread sandboxes to safely plan and execute work
Governance built-in—scoped access, auditability, and budget controls

One agent for your entire SDLC. Right inside Slack.

👉 Get started

_{Review rate limit: 9/10 reviews remaining, refill in 6 minutes.}

_{Comment @coderabbitai help to get the list of available commands and usage tips.}

tensorrt-cicd · 2026-05-03T16:29:04Z

PR_Github #46603 [ run ] triggered by Bot. Commit: 1aee667 Link to invocation

tensorrt-cicd · 2026-05-03T20:23:11Z

PR_Github #46603 [ run ] completed with state SUCCESS. Commit: 1aee667
/LLM/main/L0_MergeRequest_PR pipeline #36650 completed with status: 'FAILURE'

CI Report

⚠️ Action Required:

Please check the failed tests and fix your PR
If you cannot view the failures, ask the CI triggerer to share details
Once fixed, request an NVIDIA team member to trigger CI again

CI Agent Failure Analysis

Link to invocation

achartier · 2026-05-04T03:46:46Z

/bot run --disable-fail-fast

tensorrt-cicd · 2026-05-04T03:52:36Z

PR_Github #46616 [ run ] triggered by Bot. Commit: 1aee667 Link to invocation

tensorrt-cicd · 2026-05-04T08:10:31Z

PR_Github #46616 [ run ] completed with state SUCCESS. Commit: 1aee667
/LLM/main/L0_MergeRequest_PR pipeline #36663 completed with status: 'FAILURE'

CI Report

⚠️ Action Required:

Please check the failed tests and fix your PR
If you cannot view the failures, ask the CI triggerer to share details
Once fixed, request an NVIDIA team member to trigger CI again

CI Agent Failure Analysis

Link to invocation

VALLIS-NERIA · 2026-05-04T10:40:54Z

/bot run --disable-fail-fast

tensorrt-cicd · 2026-05-04T10:48:37Z

PR_Github #46635 [ run ] triggered by Bot. Commit: 1aee667 Link to invocation

tensorrt-cicd · 2026-05-04T14:08:54Z

PR_Github #46635 [ run ] completed with state SUCCESS. Commit: 1aee667
/LLM/main/L0_MergeRequest_PR pipeline #36679 completed with status: 'SUCCESS'

CI Report

Link to invocation

achartier requested review from a team as code owners May 3, 2026 16:22

achartier requested review from Wanli-Jiang and yechank-nvidia May 3, 2026 16:22

github-actions Bot assigned achartier May 3, 2026

yechank-nvidia approved these changes May 4, 2026

View reviewed changes

moraxu approved these changes May 4, 2026

View reviewed changes

VALLIS-NERIA enabled auto-merge (squash) May 4, 2026 13:39

VALLIS-NERIA merged commit 4757fec into NVIDIA:main May 4, 2026
10 checks passed

achartier deleted the fix-qwen3_5-nvfp4-weight-scale branch May 4, 2026 14:11

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

[None][fix] Fix Qwen3.5 NVFP4 weight loading by preserving weight_scales#13716

[None][fix] Fix Qwen3.5 NVFP4 weight loading by preserving weight_scales#13716
VALLIS-NERIA merged 1 commit intoNVIDIA:mainfrom
achartier:fix-qwen3_5-nvfp4-weight-scale

achartier commented May 3, 2026 •

edited by coderabbitai Bot

Loading

Uh oh!

achartier commented May 3, 2026

Uh oh!

coderabbitai Bot commented May 3, 2026

Walkthrough

Changes

Estimated code review effort

❌ Failed checks (1 warning)

Uh oh!

tensorrt-cicd commented May 3, 2026

Uh oh!

tensorrt-cicd commented May 3, 2026

Uh oh!

achartier commented May 4, 2026

Uh oh!

tensorrt-cicd commented May 4, 2026

Uh oh!

tensorrt-cicd commented May 4, 2026

Uh oh!

VALLIS-NERIA commented May 4, 2026

Uh oh!

tensorrt-cicd commented May 4, 2026

Uh oh!

tensorrt-cicd commented May 4, 2026

Uh oh!

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

5 participants

Conversation

achartier commented May 3, 2026 • edited by coderabbitai Bot Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Summary by CodeRabbit

Description

Test Coverage

PR Checklist

GitHub Bot Help

Uh oh!

achartier commented May 3, 2026

Uh oh!

coderabbitai Bot commented May 3, 2026

Walkthrough

Changes

Estimated code review effort

❌ Failed checks (1 warning)

Uh oh!

tensorrt-cicd commented May 3, 2026

Uh oh!

tensorrt-cicd commented May 3, 2026

Uh oh!

achartier commented May 4, 2026

Uh oh!

tensorrt-cicd commented May 4, 2026

Uh oh!

tensorrt-cicd commented May 4, 2026

Uh oh!

VALLIS-NERIA commented May 4, 2026

Uh oh!

tensorrt-cicd commented May 4, 2026

Uh oh!

tensorrt-cicd commented May 4, 2026

Uh oh!

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

5 participants

achartier commented May 3, 2026 •

edited by coderabbitai Bot

Loading