[Cherry-pick] PRs #1256 #1305 #1322 #1317 #1321 #1289 #1311 #1332 #1104 #1318

[kevalmorabia97]

Cherry-picked PRs

• Exclude small-k and small-n Matmul nodes from Int8 quantization #1256
• Fix vLLM fakequant MoE megatron export bug #1305
• docs: update installation pages with legal-approved license notices #1322
• Update vLLM deployment docs for heterogeneous models #1317
• Fix lm_eval version checking #1321
• [OMNIML-3349][6114983] Add FP8 MHA quantization support for HuggingFace ViT #1289
• fix: bug hf_ptq.py max_length setting ignored for LLMs #1311
• Deprecate Mllama support in llm_ptq/vlm_ptq examples #1332
• fix: EAGLE mix_hidden_states in-place op crash (#1088) #1104
• [NVBug 6061382] Fix PTQ for VLMs with image calibration #1318

Summary by CodeRabbit

Release Notes

• Documentation

  • Updated installation guides with third-party software license disclaimers.
  • Added vLLM deployment instructions for model deployment.
  • Expanded NGC container image recommendations.

• Deprecations

  • Mllama/vision model image processor support deprecated; users directed to use --calib_with_images with supported models.

• Bug Fixes

  • Fixed ONNX quantization to exclude small MatMul/Gemm operations from INT8/FP8 quantization.
  • Improved FP8 export with enhanced cast-folding and attention fusion optimizations.

• New Features

  • Added LayerNorm quantization support for improved FP8 attention quantization.

[coderabbitai]

📝 Walkthrough

Walkthrough

This pull request deprecates Mllama support in PTQ examples, introduces FP8 multi-head attention quantization optimizations in ONNX export, adds MatMul dimension-based exclusion logic to prevent small-GeMM quantization, refactors weight/bias extraction in quantization exports, updates documentation with third-party license notices, and includes comprehensive test coverage for new quantization features.

Changes

Cohort / File(s)	Summary
Documentation & License CHANGELOG.rst, LICENSE_HEADER, README.md, docs/source/getting_started/_installation_for_Linux.rst, docs/source/getting_started/windows/_installation_for_Windows.rst	License year update (2024→2026) and expanded installation/license disclaimers clarifying third-party software downloads; replaces single TensorRT-LLM image reference with multiple NVIDIA NGC container options; updates Linux container guidance with explicit pre-installation confirmation.
Mllama Deprecation examples/llm_ptq/example_utils.py, examples/llm_ptq/hf_ptq.py, modelopt/torch/utils/image_processor.py, modelopt/torch/utils/vlm_dataset_utils.py	Removes MllamaImageProcessor and all mllama-specific branches from calibration, processor initialization, and post-quantization logic; narrows processor type annotations from BaseImageProcessor | ProcessorMixin to ProcessorMixin | None; unifies VLM image processing via args.calib_with_images and AutoProcessor.
ONNX FP8 MHA Quantization modelopt/onnx/export/fp8_exporter.py, tests/unit/onnx/quantization/test_fp8_mha_exporter.py	Centralizes FP8 constants, adds transpose-folding for constant weights, and introduces three graph-rewrite passes: moving Mul/Transpose before Q/DQ and inserting fixed-scale Q/DQ after Softmax; new test validates rewrite patterns and non-MatMul consumer skipping.
MatMul Exclusion & Shape Inference modelopt/onnx/quantization/graph_utils.py, tests/unit/onnx/quantization/test_graph_utils.py	Expands MatMul quantization exclusion from GEMV-only to include small-GeMM (K or N < 16); adds _get_inp_b_k_dim helper for K derivation with transB awareness; extends symbolic/runtime inference to infer K from constant initializers, graph inputs, and execution results; comprehensive test coverage including transB and deduplication logic.
FP16/FP32 Cast Folding modelopt/onnx/utils.py, modelopt/torch/_deploy/utils/torch_onnx.py, tests/unit/onnx/test_fold_casts.py	Adds fold_q_fp16_to_fp32_casts for Q/DQ scale cast elimination; gates FP16 folding functions by minimum opset version; integrates FP8-specific cast folding into ONNX optimization pipeline; test validates cast removal and scale rewriting across opset versions.
PyTorch FP8 Export & Quantization modelopt/torch/quantization/export_onnx.py, examples/torch_onnx/torch_quant_to_onnx.py	Removes intermediate Cast operations and trt_high_precision_dtype constraints from FP8 Q/DQ; adds FP8 MHA-specific LayerNorm quantizer config; introduces runtime helpers to disable quantizers for 4D+ inputs, low-channel Conv2d, and "dead" quantizers with NaN/non-positive amax.
Quantization Registry & Modules modelopt/torch/quantization/nn/__init__.py, modelopt/torch/quantization/nn/modules/quant_layernorm.py	Exports new LayerNorm quantization module via registry; enables LayerNorm output quantizers for FP8 attention fusion scenarios.
Weight/Bias Extraction Refactoring modelopt/torch/export/unified_export_megatron.py, modelopt/torch/export/plugins/vllm_fakequant_megatron.py	Centralizes weight/bias tensor extraction via new _get_weight_bias helper; replaces direct hasattr checks with dict-based detection; refactors vLLM quantized-weight swap to safely re-wrap as Parameter and restore in finally block.
Attention & Plugin Improvements modelopt/torch/quantization/plugins/huggingface.py, modelopt/torch/quantization/plugins/vllm.py, modelopt/torch/speculative/plugins/transformers.py	Introduces nested-Attention detection to skip double-patching; adds parameter type-preservation logic for quantized weights; clones eagle_input_hiddens before in-place assignment to prevent autograd breakage.
Example & Utility Updates examples/llm_eval/lm_eval_hf.py, examples/puzzletron/README.md, examples/vllm_serve/vllm_reload_utils.py	Replaces direct version import with importlib.metadata; updates vLLM deployment instructions with AnyModel fork and config.json requirements; broadens quantizer-key matching from "quantizer_" substring to any "quantizer" occurrence.
New Test Coverage tests/unit/torch/quantization/plugins/test_nested_attention_skip.py, tests/unit/torch/quantization/test_quant_layernorm.py, tests/unit/torch/speculative/plugins/test_hf_speculative.py	Validates nested-attention skip logic, LayerNorm quantization exactness, and EAGLE speculative decoding backward-pass gradient flow.

Estimated code review effort

🎯 4 (Complex) | ⏱️ ~70 minutes

🚥 Pre-merge checks | ✅ 5 | ❌ 1

❌ Failed checks (1 warning)

Check name	Status	Explanation	Resolution
Docstring Coverage	⚠️ Warning	Docstring coverage is 78.02% which is insufficient. The required threshold is 80.00%.	Write docstrings for the functions missing them to satisfy the coverage threshold.

✅ Passed checks (5 passed)

Check name	Status	Explanation
Description Check	✅ Passed	Check skipped - CodeRabbit’s high-level summary is enabled.
Linked Issues check	✅ Passed	Check skipped because no linked issues were found for this pull request.
Out of Scope Changes check	✅ Passed	Check skipped because no linked issues were found for this pull request.
Security Anti-Patterns	✅ Passed	Comprehensive security scan found no instances of torch.load with weights_only=False, numpy.load with allow_pickle=True, hardcoded trust_remote_code=True, eval/exec calls, or # nosec comments bypassing security checks.
Title check	✅ Passed	The title directly summarizes the main change: cherry-picking multiple PRs into a release branch. It is specific, clear, and accurately reflects the changeset.

_{✏️ Tip: You can configure your own custom pre-merge checks in the settings.}

✨ Finishing Touches

📝 Generate docstrings

• Create stacked PR
• Commit on current branch

🧪 Generate unit tests (beta)

• Create PR with unit tests
• Commit unit tests in branch cherry-picks/release-0.44.0

---

_{Comment @coderabbitai help to get the list of available commands and usage tips.}

[github-actions]

PR Preview Action v1.8.1
Preview removed because the pull request was closed.
2026-04-27 17:06 UTC

[coderabbitai]

Actionable comments posted: 5

Caution

Some comments are outside the diff and can’t be posted inline due to platform limitations.

⚠️ Outside diff range comments (1)

modelopt/torch/utils/vlm_dataset_utils.py (1)

441-448: ⚠️ Potential issue | 🔴 Critical

Fix logic bug in max_length condition that prevents truncation from ever being applied.

The condition "images" not in kwargs at line 447 will always be False because "images" is unconditionally added to kwargs at line 443. This means the truncation and max_length parameters are never passed to the processor, making the max_length function parameter ineffective.

If the intent is to skip truncation for multimodal cases (when images are present), the condition should be inverted to "images" in kwargs. If truncation should always apply when max_length is provided, the condition should be removed entirely.

🤖 Prompt for AI Agents

Verify each finding against the current code and only fix it if needed.

In `@modelopt/torch/utils/vlm_dataset_utils.py` around lines 441 - 448, The
current logic never applies truncation because "images" is always present in
kwargs; update the max_length handling so truncation is applied when max_length
is provided: remove the `"images" not in kwargs` check and simply, when
max_length is not None, call kwargs.update({"truncation": True, "max_length":
max_length}) (referencing the kwargs dict, max_length parameter, and the earlier
creation that includes "text": list(prompts) and "images": list(images)).

🤖 Prompt for all review comments with AI agents

Verify each finding against the current code and only fix it if needed.

Inline comments:
In `@examples/torch_onnx/torch_quant_to_onnx.py`:
- Around line 273-274: The current guard on the quantizer uses
torch.any(torch.isnan(amax)) or torch.all(amax <= 0) so it only disables q when
every channel/block is non-positive; change it to disable the quantizer when any
entry is invalid (NaN) or any entry is <= 0. Locate the use of amax and
q.disable() in torch_quant_to_onnx.py and replace the torch.all(amax <= 0)
condition with torch.any(amax <= 0) (or equivalent) so per-channel/per-block
zero/negative values trigger q.disable() and avoid 448/amax infinities.
- Around line 595-600: The auto-mode gate is too broad: change the any(...)
check used to set uses_fp8_conv_input so it only triggers when the
auto_quantization_formats actually include an FP8-family format (not merely
anything other than "INT8_DEFAULT_CFG"); update the condition in the
uses_fp8_conv_input computation to detect FP8-style formats (e.g., check fmt
strings for FP8-family identifiers like "fp8", "mxfp8", "nvfp4" or a specific
FP8-format set) and only call
_disable_low_channel_conv_input_quantizers(quantized_model) when such an
FP8-family format is present, keeping the existing references to
args.quantize_mode, args.auto_quantization_formats, uses_fp8_conv_input and
_disable_low_channel_conv_input_quantizers.

In `@modelopt/onnx/export/fp8_exporter.py`:
- Around line 88-122: The transpose-folding code currently rewires Transpose to
consume dq_op.outputs[0] without ensuring that dq_op.outputs[0] has no other
live consumers; update the logic around dq_op, transpose_to_remove and
cast_to_remove to first verify that the DQ output node (dq_op.outputs[0]) has
exactly one live consumer (the candidate Transpose or Cast→Transpose) before
mutating torch_weights and setting transpose_to_remove/cast_to_remove; if there
are any other consumers, skip folding and leave
cast_to_remove/transpose_to_remove as None. Apply the same single-consumer check
to the analogous block later in the file (the second transpose-folding
occurrence).

In `@modelopt/onnx/utils.py`:
- Around line 1422-1437: The underflow/overflow warning in _scale_fp32_to_fp16
is too coarse: change the condition to check elementwise where a non-zero FP32
value becomes zero in FP16 by replacing the combined np.any(...) logic with an
elementwise mask (e.g., mask = (fp16_data == 0) & (scale_data != 0)) and warn
only if np.any(np.isinf(fp16_data)) or np.any(mask); keep the existing inf check
and the rest of the in-place conversion logic using scale_init, fp16_data and
scale_data unchanged.
- Around line 1440-1474: fold_q_fp16_to_fp32_casts currently treats any
Cast(..., to=FLOAT) as FP16→FP32 and rewrites Q scales; update it to first
verify the cast source is actually FP16 before proceeding: for each Cast node
(in function fold_q_fp16_to_fp32_casts) look up the input tensor's dtype (from
graph.value_info, graph.input, or initializers) and only continue when that
dtype == onnx.TensorProto.FLOAT16; keep the existing logic that calls
_scale_fp32_to_fp16(initializers[...]) and _bypass_cast_node(onnx_model, node)
but skip/break for casts from BF16 or other dtypes so you do not rewrite Q
scales for non-FP16→FP32 casts.

---

Outside diff comments:
In `@modelopt/torch/utils/vlm_dataset_utils.py`:
- Around line 441-448: The current logic never applies truncation because
"images" is always present in kwargs; update the max_length handling so
truncation is applied when max_length is provided: remove the `"images" not in
kwargs` check and simply, when max_length is not None, call
kwargs.update({"truncation": True, "max_length": max_length}) (referencing the
kwargs dict, max_length parameter, and the earlier creation that includes
"text": list(prompts) and "images": list(images)).

🪄 Autofix (Beta)

Fix all unresolved CodeRabbit comments on this PR:

• Push a commit to this branch (recommended)
• Create a new PR with the fixes

---

ℹ️ Review info

⚙️ Run configuration

Configuration used: Path: .coderabbit.yaml

Review profile: CHILL

Plan: Enterprise

Run ID: 8d54ab83-f2cc-4766-b089-ee6c30d50c0e

📥 Commits

Reviewing files that changed from the base of the PR and between f3da713 and 2892b95.

📒 Files selected for processing (31)

• CHANGELOG.rst
• LICENSE_HEADER
• README.md
• docs/source/getting_started/_installation_for_Linux.rst
• docs/source/getting_started/windows/_installation_for_Windows.rst
• examples/llm_eval/lm_eval_hf.py
• examples/llm_ptq/example_utils.py
• examples/llm_ptq/hf_ptq.py
• examples/puzzletron/README.md
• examples/torch_onnx/torch_quant_to_onnx.py
• examples/vllm_serve/vllm_reload_utils.py
• modelopt/onnx/export/fp8_exporter.py
• modelopt/onnx/quantization/graph_utils.py
• modelopt/onnx/utils.py
• modelopt/torch/_deploy/utils/torch_onnx.py
• modelopt/torch/export/plugins/vllm_fakequant_megatron.py
• modelopt/torch/export/unified_export_megatron.py
• modelopt/torch/quantization/export_onnx.py
• modelopt/torch/quantization/nn/__init__.py
• modelopt/torch/quantization/nn/modules/quant_layernorm.py
• modelopt/torch/quantization/plugins/huggingface.py
• modelopt/torch/quantization/plugins/vllm.py
• modelopt/torch/speculative/plugins/transformers.py
• modelopt/torch/utils/image_processor.py
• modelopt/torch/utils/vlm_dataset_utils.py
• tests/unit/onnx/quantization/test_fp8_mha_exporter.py
• tests/unit/onnx/quantization/test_graph_utils.py
• tests/unit/onnx/test_fold_casts.py
• tests/unit/torch/quantization/plugins/test_nested_attention_skip.py
• tests/unit/torch/quantization/test_quant_layernorm.py
• tests/unit/torch/speculative/plugins/test_hf_speculative.py

💤 Files with no reviewable changes (1)

• modelopt/torch/utils/image_processor.py

[coderabbitai]

⚠️ Potential issue | 🟠 Major

Handle partially-dead amax tensors too.

This only disables a quantizer when all amax entries are <= 0, but FP8 export will still blow up if a per-channel/per-block quantizer has just one zero/negative entry. That makes the current guard miss exactly the mixed-validity case that still produces 448 / amax infinities.

Proposed fix

-            if torch.any(torch.isnan(amax)) or torch.all(amax <= 0):
+            if torch.any(torch.isnan(amax)) or torch.any(amax <= 0):
                 q.disable()

🤖 Prompt for AI Agents

Verify each finding against the current code and only fix it if needed.

In `@examples/torch_onnx/torch_quant_to_onnx.py` around lines 273 - 274, The
current guard on the quantizer uses torch.any(torch.isnan(amax)) or
torch.all(amax <= 0) so it only disables q when every channel/block is
non-positive; change it to disable the quantizer when any entry is invalid (NaN)
or any entry is <= 0. Locate the use of amax and q.disable() in
torch_quant_to_onnx.py and replace the torch.all(amax <= 0) condition with
torch.any(amax <= 0) (or equivalent) so per-channel/per-block zero/negative
values trigger q.disable() and avoid 448/amax infinities.

[coderabbitai]

⚠️ Potential issue | 🟠 Major

Narrow the auto-mode gate to actual FP8-family formats.

any(fmt != "INT8_DEFAULT_CFG" ...) also fires for INT4_AWQ_CFG, so auto mode will disable low-channel Conv input quantizers even when no FP8-style format is in the search space. That changes the search budget/accuracy for a workaround that is only needed for TRT_FP8QuantizeLinear.

Proposed fix

+    fp8_family_formats = {
+        "FP8_DEFAULT_CFG",
+        "MXFP8_DEFAULT_CFG",
+        "NVFP4_AWQ_LITE_CFG",
+        "NVFP4_DEFAULT_CFG",
+    }
     uses_fp8_conv_input = args.quantize_mode in ("fp8", "mxfp8", "nvfp4") or (
         args.quantize_mode == "auto"
-        and any(fmt != "INT8_DEFAULT_CFG" for fmt in args.auto_quantization_formats)
+        and any(fmt in fp8_family_formats for fmt in args.auto_quantization_formats)
     )

🤖 Prompt for AI Agents

Verify each finding against the current code and only fix it if needed.

In `@examples/torch_onnx/torch_quant_to_onnx.py` around lines 595 - 600, The
auto-mode gate is too broad: change the any(...) check used to set
uses_fp8_conv_input so it only triggers when the auto_quantization_formats
actually include an FP8-family format (not merely anything other than
"INT8_DEFAULT_CFG"); update the condition in the uses_fp8_conv_input computation
to detect FP8-style formats (e.g., check fmt strings for FP8-family identifiers
like "fp8", "mxfp8", "nvfp4" or a specific FP8-format set) and only call
_disable_low_channel_conv_input_quantizers(quantized_model) when such an
FP8-family format is present, keeping the existing references to
args.quantize_mode, args.auto_quantization_formats, uses_fp8_conv_input and
_disable_low_channel_conv_input_quantizers.

[coderabbitai]

⚠️ Potential issue | 🟠 Major

Only fold the transpose when the DQ output has no other live consumers.

This rewrite pre-transposes the stored weight and then rewires the Transpose branch to consume dq_op.outputs[0] directly, but it never checks whether that same DQ output also feeds some other branch. If it does, that branch now starts seeing the transposed weight too, which is a silent graph corruption.

Proposed fix

                 transpose_to_remove = None
                 cast_to_remove = None
-                for candidate in list(dq_op.outputs[0].outputs):
+                dq_consumers = list(dq_op.outputs[0].outputs)
+                for candidate in dq_consumers:
                     if candidate.op == "Cast":
                         cast_to_remove = candidate
                         candidate = next(
                             (c for c in candidate.outputs[0].outputs if c.op == "Transpose"),
                             None,
@@
                     if t_consumers and all(c.op == "MatMul" for c in t_consumers):
+                        if len(dq_consumers) != 1:
+                            transpose_to_remove = None
+                            cast_to_remove = None
+                            break
                         perm = candidate.attrs.get("perm", None)
                         torch_weights = (
                             torch_weights.permute(*perm).contiguous()
                             if perm is not None
                             else torch_weights.T.contiguous()

Also applies to: 142-151

🤖 Prompt for AI Agents

Verify each finding against the current code and only fix it if needed.

In `@modelopt/onnx/export/fp8_exporter.py` around lines 88 - 122, The
transpose-folding code currently rewires Transpose to consume dq_op.outputs[0]
without ensuring that dq_op.outputs[0] has no other live consumers; update the
logic around dq_op, transpose_to_remove and cast_to_remove to first verify that
the DQ output node (dq_op.outputs[0]) has exactly one live consumer (the
candidate Transpose or Cast→Transpose) before mutating torch_weights and setting
transpose_to_remove/cast_to_remove; if there are any other consumers, skip
folding and leave cast_to_remove/transpose_to_remove as None. Apply the same
single-consumer check to the analogous block later in the file (the second
transpose-folding occurrence).

[coderabbitai]

⚠️ Potential issue | 🟡 Minor

Make the underflow warning elementwise.

np.any(fp16_data == 0) and np.any(scale_data != 0) fires whenever the tensor contains any real zero plus any non-zero value. That produces false overflow/underflow warnings for mixed scale tensors.

Suggested fix

-    if np.any(np.isinf(fp16_data)) or (np.any(fp16_data == 0) and np.any(scale_data != 0)):
+    if np.any(np.isinf(fp16_data)) or np.any((fp16_data == 0) & (scale_data != 0)):
         logger.warning(f"Q/DQ scale '{scale_init.name}' overflows or underflows when cast to FP16")

📝 Committable suggestion

‼️ IMPORTANT
Carefully review the code before committing. Ensure that it accurately replaces the highlighted code, contains no missing lines, and has no issues with indentation. Thoroughly test & benchmark the code to ensure it meets the requirements.

Suggested change

	def _scale_fp32_to_fp16(scale_init: onnx.TensorProto) -> None:
	"""Convert a scalar Q/DQ scale initializer in-place from FP32 to FP16.
	Warns if any non-zero scale saturates to 0/inf in FP16 (out of FP16 representable range).
	"""
	if scale_init.data_type != onnx.TensorProto.FLOAT:
	return
	scale_data = np.frombuffer(scale_init.raw_data, dtype=np.float32)
	if not scale_data.size:
	scale_data = np.array(scale_init.float_data, dtype=np.float32)
	fp16_data = scale_data.astype(np.float16)
	if np.any(np.isinf(fp16_data)) or (np.any(fp16_data == 0) and np.any(scale_data != 0)):
	logger.warning(f"Q/DQ scale '{scale_init.name}' overflows or underflows when cast to FP16")
	scale_init.data_type = onnx.TensorProto.FLOAT16
	scale_init.raw_data = fp16_data.tobytes()
	del scale_init.float_data[:]
	def _scale_fp32_to_fp16(scale_init: onnx.TensorProto) -> None:
	"""Convert a scalar Q/DQ scale initializer in-place from FP32 to FP16.
	Warns if any non-zero scale saturates to 0/inf in FP16 (out of FP16 representable range).
	"""
	if scale_init.data_type != onnx.TensorProto.FLOAT:
	return
	scale_data = np.frombuffer(scale_init.raw_data, dtype=np.float32)
	if not scale_data.size:
	scale_data = np.array(scale_init.float_data, dtype=np.float32)
	fp16_data = scale_data.astype(np.float16)
	if np.any(np.isinf(fp16_data)) or np.any((fp16_data == 0) & (scale_data != 0)):
	logger.warning(f"Q/DQ scale '{scale_init.name}' overflows or underflows when cast to FP16")
	scale_init.data_type = onnx.TensorProto.FLOAT16
	scale_init.raw_data = fp16_data.tobytes()
	del scale_init.float_data[:]

🤖 Prompt for AI Agents

Verify each finding against the current code and only fix it if needed.

In `@modelopt/onnx/utils.py` around lines 1422 - 1437, The underflow/overflow
warning in _scale_fp32_to_fp16 is too coarse: change the condition to check
elementwise where a non-zero FP32 value becomes zero in FP16 by replacing the
combined np.any(...) logic with an elementwise mask (e.g., mask = (fp16_data ==
0) & (scale_data != 0)) and warn only if np.any(np.isinf(fp16_data)) or
np.any(mask); keep the existing inf check and the rest of the in-place
conversion logic using scale_init, fp16_data and scale_data unchanged.

[coderabbitai]

⚠️ Potential issue | 🟠 Major

Guard this fold to actual FP16→FP32 casts.

The implementation only checks Cast(..., to=FLOAT), so it will also bypass BF16→FP32 or any other *→FP32 cast and then rewrite the Q scale to FP16. That changes semantics and can leave the quantizer consuming a dtype this pass was not meant to legalize.

Suggested fix

 def fold_q_fp16_to_fp32_casts(onnx_model: onnx.ModelProto) -> onnx.ModelProto:
@@
-    consumer_map: dict[str, list[onnx.NodeProto]] = {}
+    consumer_map: dict[str, list[onnx.NodeProto]] = {}
     for node in onnx_model.graph.node:
         for inp in node.input:
             consumer_map.setdefault(inp, []).append(node)
     initializers = {init.name: init for init in onnx_model.graph.initializer}
+    type_map = _build_tensor_type_map(onnx_model)
@@
         cast_to = next((a.i for a in node.attribute if a.name == "to"), None)
         if cast_to != onnx.TensorProto.FLOAT:
             continue
+        if type_map.get(node.input[0]) != onnx.TensorProto.FLOAT16:
+            continue
         consumers = consumer_map.get(node.output[0], [])
         if not consumers or not all(c.op_type in _Q_OPS for c in consumers):
             continue

🤖 Prompt for AI Agents

Verify each finding against the current code and only fix it if needed.

In `@modelopt/onnx/utils.py` around lines 1440 - 1474, fold_q_fp16_to_fp32_casts
currently treats any Cast(..., to=FLOAT) as FP16→FP32 and rewrites Q scales;
update it to first verify the cast source is actually FP16 before proceeding:
for each Cast node (in function fold_q_fp16_to_fp32_casts) look up the input
tensor's dtype (from graph.value_info, graph.input, or initializers) and only
continue when that dtype == onnx.TensorProto.FLOAT16; keep the existing logic
that calls _scale_fp32_to_fp16(initializers[...]) and
_bypass_cast_node(onnx_model, node) but skip/break for casts from BF16 or other
dtypes so you do not rewrite Q scales for non-FP16→FP32 casts.

[codecov]

Codecov Report

❌ Patch coverage is 91.25874% with 25 lines in your changes missing coverage. Please review.
✅ Project coverage is 75.87%. Comparing base (f3da713) to head (2892b95).

Files with missing lines	Patch %	Lines
modelopt/onnx/export/fp8_exporter.py	89.61%	16 Missing ⚠️
modelopt/onnx/utils.py	89.58%	5 Missing ⚠️
modelopt/torch/export/unified_export_megatron.py	78.57%	3 Missing ⚠️
modelopt/torch/utils/vlm_dataset_utils.py	0.00%	1 Missing ⚠️

Additional details and impacted files

@@                Coverage Diff                 @@
##           release/0.44.0    #1350      +/-   ##
==================================================
+ Coverage           75.38%   75.87%   +0.49%     
==================================================
  Files                 462      462              
  Lines               49960    50173     +213     
==================================================
+ Hits                37662    38069     +407     
+ Misses              12298    12104     -194     

Flag	Coverage Δ
examples	41.64% <56.29%> (+0.93%)	⬆️
gpu	58.39% <20.62%> (-0.74%)	⬇️
regression	14.78% <2.09%> (-0.01%)	⬇️
unit	52.87% <70.62%> (+0.48%)	⬆️

Flags with carried forward coverage won't be shown. Click here to find out more.

☔ View full report in Codecov by Sentry.
📢 Have feedback on the report? Share it here.

🚀 New features to boost your workflow:

• ❄️ Test Analytics: Detect flaky tests, report on failures, and find test suite problems.