[NVBug: 6524370] use sequential device_map for DiffusionGemma - #2041
Conversation
|
Auto-sync is disabled for draft pull requests in this repository. Workflows must be run manually. Contributors can view more details about this message here. |
|
Note Reviews pausedIt looks like this branch is under active development. To avoid overwhelming you with review comments due to an influx of new commits, CodeRabbit has automatically paused this review. You can configure this behavior by changing the Use the following commands to manage reviews:
Use the checkboxes below for quick actions:
📝 WalkthroughWalkthroughThe change detects DiffusionGemma models from Hugging Face configuration metadata. Multi-GPU, non-CPU DiffusionGemma loads now use sequential device mapping. Tests cover detection and device-map selection. ChangesDiffusionGemma device mapping
Estimated code review effort: 2 (Simple) | ~10 minutes 🚥 Pre-merge checks | ✅ 6✅ Passed checks (6 passed)
✨ Finishing Touches📝 Generate docstrings
🧪 Generate unit tests (beta)
Comment |
juhi10071998
changed the title
Codecov Report✅ All modified and coverable lines are covered by tests. Additional details and impacted files@@ Coverage Diff @@
## main #2041 +/- ##
==========================================
- Coverage 67.00% 66.83% -0.18%
==========================================
Files 520 520
Lines 59545 59545
==========================================
- Hits 39900 39796 -104
- Misses 19645 19749 +104
Flags with carried forward coverage won't be shown. Click here to find out more. ☔ View full report in Codecov by Harness. 🚀 New features to boost your workflow:
|
7929ddd to
3a02bfa
Compare
3a02bfa to
6ef39d9
Compare
![coderabbitai[bot]](https://avatars.githubusercontent.com/in/347564?s=60&v=4){kind=small}
There was a problem hiding this comment.
Warning
CodeRabbit couldn't request changes on this pull request because it doesn't have sufficient GitHub permissions.
Please grant CodeRabbit Pull requests: Read and write permission and re-run the review.
Actionable comments posted: 2
🤖 Prompt for all review comments with AI agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.
Inline comments:
In `@examples/hf_ptq/example_utils.py`:
- Around line 715-720: Update the conditional around
is_diffusion_gemma(hf_config) so automatic use of use_seq_device_map is limited
to multi-GPU runs, preserving existing single-GPU memory behavior;
alternatively, retain the mapping and add a regression test covering a
near-capacity one-GPU run and its max_memory handling.
- Around line 710-721: Update the pack-quantized model loading path guarded by
has_pack_quantized_config(hf_config) to use the selected device map, including
use_seq_device_map for DiffusionGemma, instead of hardcoding device_map="auto".
Pass through the computed max_memory value as well, preserving the sequential
mapping and GPU memory limits.
🪄 Autofix (Beta)
Fix all unresolved CodeRabbit comments on this PR:
- Push a commit to this branch (recommended)
- Create a new PR with the fixes
ℹ️ Review info
⚙️ Run configuration
Configuration used: Path: .coderabbit.yaml
Review profile: CHILL
Plan: Enterprise
Run ID: 0d1bb738-0b8a-48a3-b0a3-44a783508295
📒 Files selected for processing (1)
examples/hf_ptq/example_utils.py
6ef39d9 to
fafe1db
Compare
cjluo-nv
left a comment
cjluo-nv
left a comment
There was a problem hiding this comment.
Bot review (bedrock-claude-opus-5) — DM the bot to share feedback.
Small, well-motivated fix: auto-select device_map="sequential" for DiffusionGemma in examples/hf_ptq/example_utils.get_model, mirroring the existing per-model bart/t5 handling. Placement (before the use_seq_device_map block, so max_memory capping still applies) is correct, and the multi-GPU + non-CPU guard is right. Three non-blocking points:
-
No test, although the harness already exists. The PR body says "no existing unit coverage for
get_modeldevice-map selection", buttests/examples/hf_ptq/test_example_utils.py::test_get_model_uses_expected_dtype_kwargalready monkeypatchesAutoConfig.from_pretrained,init_empty_weights,get_max_memory, andinfer_auto_device_mapand asserts on thefrom_pretrainedkwargs. A case with adiffusion_gemmaconfig + patchedtorch.cuda.device_count()assertingdevice_map == "sequential"(plus agemma3negative for the substring-collision concern the PR body describes as manually verified) is a handful of lines there. Worth adding since the whole fix is a detection heuristic. -
The comment overstates the guarantee.
sequentialonly keeps a tied pair together when the model fits on the first GPU; becauseuse_seq_device_mapalso capsmax_memorytogpu_mem_percentage(0.8), a model that doesn't fit under that cap will still spill and can still split the tied pair ontometa. Thet5/bartprecedent it cites actually usesdevice_map=None(single device), which is the stronger guarantee. Either soften the comment or explain why sequential is sufficient for this family. -
Minor robustness/dedup:
hf_config.architecturesis accessed directly while the rest of this file usesgetattr(config, "architectures", []); and DiffusionGemma detection now lives in three places (MODEL_NAME_TO_TYPE,_reorder_canonical_first, and this new helper).
No licensing changes, no prompt-injection content in the PR text.
| Underscores are ignored: the family is spelled ``diffusion_gemma`` in configs | ||
| and ``DiffusionGemma`` in class names. | ||
| """ | ||
| names = [getattr(hf_config, "model_type", None) or "", *(hf_config.architectures or [])] |
There was a problem hiding this comment.
Bot comment.
Two small things:
hf_config.architecturesis accessed directly here, while the rest of this file consistently usesgetattr(config, "architectures", [])(see_is_multimodal_config,is_nemotron_vl). Configs built in tests / non-PretrainedConfigstubs won't have the attribute and will raiseAttributeError. Suggest*(getattr(hf_config, "architectures", None) or []).- DiffusionGemma detection now exists in three places:
MODEL_NAME_TO_TYPE(modelopt/torch/export/model_utils.py),_reorder_canonical_firstin the same file (which does the same"diffusiongemma"/"diffusion_gemma"normalization on the class name), and this helper. Consider putting the config-level predicate next to the others inmodel_utils.pyso future spellings only need updating once.
There was a problem hiding this comment.
Done in b45f9c36 — switched to *(getattr(hf_config, "architectures", None) or []).
Worth noting the file is split rather than consistent here: _is_multimodal_config (179) and is_nemotron_vl (204) use getattr, but is_speculative (302) and get_model (792-794) access .architectures directly. Taking the defensive form anyway.
On consolidating into model_utils.py: agreed there is drift risk, though the two existing spots key off the class name (type(model).__name__) while this one keys off the config, so it is not a pure duplicate. Leaving it here to keep this an examples-only bugfix — happy to do the move as a follow-up if you would rather have it now.
There was a problem hiding this comment.
Confirming the owner call: consolidating the DiffusionGemma predicate into model_utils.py will be a follow-up PR, not this one. Keeping this change scoped to the examples-level bugfix so it can land for NVBug 6524370.
The drift risk is real and noted — MODEL_NAME_TO_TYPE, _reorder_canonical_first, and is_diffusion_gemma would all need updating if a new spelling appears.
|
|
||
| # DiffusionGemma ties encoder/decoder weights. device_map "auto" (balanced) can split | ||
| # a tied pair across GPUs, leaving one side on the meta device and breaking generation. | ||
| # Sequential mapping keeps tied modules together. Multi-GPU only: a single-GPU split |
There was a problem hiding this comment.
Bot comment.
"Sequential mapping keeps tied modules together" holds only when the model fits on the first GPU. Because setting use_seq_device_map = True also caps max_memory to gpu_mem_percentage (0.8 by default), a checkpoint larger than 0.8 x one GPU will still spill to the next device and can still land one side of a tied pair on meta — i.e. the same crash, just harder to hit. The t5/bart precedent referenced in the PR body actually sets device_map = None (whole model on one device), which is the stronger guarantee. Please either soften this comment to state the limitation, or explain why sequential is always sufficient for this family (e.g. size bound of the released checkpoints).
There was a problem hiding this comment.
Fair — the comment claimed more than the code delivers. Softened in b45f9c36:
# Sequential packs the model onto GPU 0 first (up to gpu_mem_percentage), keeping tied
# modules together for checkpoints that fit; larger ones can still spill and split a
# tied pair, and need an explicit single-device map.
You are right that this is a mitigation, not a guarantee, and that t5/bart use the stronger device_map = None. I did not follow them there because None puts the whole model on one device with no headroom cap, which would regress large checkpoints that legitimately need the split. For the reported case (26B-A4B, ~52 GB vs 0.8 x 186 GB on GB200) sequential is comfortably sufficient; beyond that bound the limitation is now stated rather than implied.
fafe1db to
b45f9c3
Compare
There was a problem hiding this comment.
Warning
CodeRabbit couldn't request changes on this pull request because it doesn't have sufficient GitHub permissions.
Please grant CodeRabbit Pull requests: Read and write permission and re-run the review.
Actionable comments posted: 1
🤖 Prompt for all review comments with AI agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.
Inline comments:
In `@tests/examples/hf_ptq/test_example_utils.py`:
- Around line 321-342: Add a parameterized multi-GPU case to
test_get_model_device_map_for_diffusion_gemma with model_type=None,
architecture="DiffusionGemmaForConditionalGeneration", and
expected_device_map="sequential", so the device-map selection is verified when
DiffusionGemma is identified solely by architectures.
🪄 Autofix (Beta)
Fix all unresolved CodeRabbit comments on this PR:
- Push a commit to this branch (recommended)
- Create a new PR with the fixes
ℹ️ Review info
⚙️ Run configuration
Configuration used: Path: .coderabbit.yaml
Review profile: CHILL
Plan: Enterprise
Run ID: 21fab311-75ef-4a84-84d7-1b929860eeeb
📒 Files selected for processing (2)
examples/hf_ptq/example_utils.pytests/examples/hf_ptq/test_example_utils.py
🚧 Files skipped from review as they are similar to previous changes (1)
- examples/hf_ptq/example_utils.py
cjluo-nv
left a comment
cjluo-nv
left a comment
There was a problem hiding this comment.
Bot review (bedrock-claude-opus-5) — DM the bot to share feedback.
Re-review of #2041 (auto device_map="sequential" for DiffusionGemma in examples/hf_ptq/example_utils.get_model). The head commit resolves the substantive items from the previous round; one flagged-Major item is labeled "addressed" but is not present in the code, so a human should confirm scope.
Addressed since last round (verified against the file at head):
- Missing test →
tests/examples/hf_ptq/test_example_utils.pynow hastest_get_model_device_map_for_diffusion_gemma(multi-GPU →sequential+max_memory == {0: 819.2}; single-GPU →autowith nomax_memory;gemma3negative →auto) andtest_is_diffusion_gemma(both spellings, missing attrs). I tracedget_modelwith these stubs — the patchedAutoConfig/init_empty_weights/get_max_memory/infer_auto_device_map/torch.cuda.device_countpath reachesfrom_pretrainedandis_model_on_gpucleanly (emptyparameters()), so the tests exercise the real selection logic rather than a stub. - Overstated comment → softened to state the "checkpoints that fit" limitation and the need for an explicit single-device map beyond it.
hf_config.architectures→ now*(getattr(hf_config, "architectures", None) or []).- Single-GPU regression → gated on
torch.cuda.device_count() > 1, so single-GPU keeps unrestrictedauto(pinned by a test case).
Remaining for owner judgment:
- 💬 CodeRabbit's Major finding on the
has_pack_quantized_config(hf_config)branch is annotated "✅ Addressed in commit 6ef39d9", but the code at head still hardcodesdevice_map="auto"(and dropsmax_memory) in that branch. Published DiffusionGemma compressed-tensors checkpoints exist, and such a load would bypass this fix and can still split the tied pair ontometa— i.e. the same crash on a different entry path. It's pre-existing code and arguably out of scope for this bugfix, but the resolution label doesn't match the diff, so please confirm intent. - 💬 Author replied on consolidation: the two existing spots key off
type(model).__name__while this keys off the config, so it's not a pure duplicate; happy to move tomodel_utils.pyas a follow-up — still worth an owner call, since DiffusionGemma spelling normalization now lives in three places (MODEL_NAME_TO_TYPE,_reorder_canonical_first,is_diffusion_gemma) and future spellings must be updated in all of them. - Minor, optional: CodeRabbit's last suggestion (a multi-GPU
model_type=None+architectures=["DiffusionGemma..."]device-map case) is not added;test_is_diffusion_gemmacovers architecture-only detection at the predicate level, which I consider adequate.
No licensing changes and no prompt-injection content in the PR text or comments.
b45f9c3 to
6acad70
Compare
|
/claude review |
![claude[bot]](https://avatars.githubusercontent.com/in/1236702?s=60&v=4){kind=small}
| # DiffusionGemma ties encoder/decoder weights. device_map "auto" (balanced) can split | ||
| # a tied pair across GPUs, leaving one side on the meta device and breaking generation. | ||
| # Sequential packs the model onto GPU 0 first (up to gpu_mem_percentage), keeping tied | ||
| # modules together for checkpoints that fit; larger ones can still spill and split a | ||
| # tied pair, and need an explicit single-device map. Multi-GPU only: a single-GPU split | ||
| # cannot separate a tied pair, and sequential would needlessly cap max_memory there. | ||
| if device != "cpu" and torch.cuda.device_count() > 1 and is_diffusion_gemma(hf_config): | ||
| print( | ||
| "Detected DiffusionGemma model. Using device_map='sequential'; the balanced " | ||
| "'auto' mapping can split its tied encoder/decoder weights across GPUs." | ||
| ) | ||
| use_seq_device_map = True |
There was a problem hiding this comment.
[SUGGESTION] No escape hatch: this override is unconditional, so a multi-GPU DiffusionGemma user can no longer get device_map="auto" at all.
The forced switch changes two things for every multi-GPU DiffusionGemma load — device_map auto → sequential, and max_memory unset → capped at gpu_mem_percentage (0.8) per GPU. Packing onto GPU 0 first means activations and calibration buffers now compete for the same device rather than spreading across all GPUs, so a checkpoint that happened to load fine under auto (small enough that the balanced split didn't separate the tied pair) can now OOM during calibration with no way to opt back out. That's a real, if narrow, behavior change with no CLI path around it.
The print makes it visible rather than silent, and for the reported 26B-A4B-on-GB200 case the trade is clearly right — so this is non-blocking. But the surrounding per-model handling in this function (bart, t5, mxfp4) is all for cases that are broken under the alternative, whereas here auto is merely risky. Worth considering a --no_seq_device_map / tri-state flag so the override is a default rather than a hard rule, or at minimum noting in examples/llm_ptq/README.md that DiffusionGemma multi-GPU is force-sequential and how to work around an OOM (fewer visible GPUs + explicit single-device map).
Happy to defer this to a follow-up if you'd rather keep the bugfix minimal.
There was a problem hiding this comment.
Agreed on the analysis, deferring to a follow-up. The distinction you draw is the right one: bart/t5/mxfp4 special-case loads that are broken under the alternative, whereas auto here is risky rather than always fatal, so a hard override is a stronger stance than the surrounding code takes.
Keeping this PR scoped to the NVBug 6524370 fix. A tri-state flag is new CLI surface and deserves its own review; the README note belongs with it so the documented workaround matches whatever the flag ends up being. Tracking alongside the model_utils.py consolidation follow-up.
Noting the escape hatch that exists today: CUDA_VISIBLE_DEVICES=0 bypasses the override entirely, since it is gated on torch.cuda.device_count() > 1.
There was a problem hiding this comment.
Bot review (bedrock-claude-opus-5) — DM the bot to share feedback.
Claude review passed — no blocking issues found. LGTM
Findings: CRITICAL: 0 · IMPORTANT: 0 · SUGGESTION: 3
Scope
Full review of both changed files (examples/hf_ptq/example_utils.py, tests/examples/hf_ptq/test_example_utils.py), plus surrounding get_model context and the DiffusionGemma detection sites in modelopt/torch/export/model_utils.py.
What I verified
- Placement is correct. The new block sits after
model_kwargsis built and before theif use_seq_device_map:block, so settinguse_seq_device_map = Truestill picks up themax_memorycap. The laterbart/t5device_map = Noneoverrides can't collide with DiffusionGemma. - Guard is right.
device != "cpu" and torch.cuda.device_count() > 1matches the failure mode described in the PR body — a single-GPU balanced split can't separate a tied pair, and gating on multi-GPU avoids regressing single-GPU runs with an unnecessary 0.8 memory cap. - No over-match.
"gemma" in "diffusiongemma"is the real collision risk here (the same oneMODEL_NAME_TO_TYPEorders around atmodel_utils.py:38-40), and the substring direction is checked correctly:"diffusiongemma" in name, not the reverse.gemma3/Gemma3ForCausalLMdoes not match, and that's pinned by a test. - Defensive attribute access.
getattr(hf_config, "model_type", None) or ""and*(getattr(hf_config, "architectures", None) or [])handleNone,[], and missing-attribute configs — all four covered bytest_is_diffusion_gemma. - Tests exercise the real path. I traced
get_modelunder the stubs: the patchedAutoConfig/init_empty_weights/get_max_memory/infer_auto_device_map/torch.cuda.device_countchain reachesfrom_pretrainedandis_model_on_gpucleanly (parameters()returns an empty iterator), andinfer_auto_device_mapreturning{"": 0}keepson_cpuFalse so theautocases legitimately assertmax_memoryabsent. The assertions land on real selection logic, not on a stub. - Backward compatible. No signature or default change;
--use_seq_device_mapstill works and is only ever turned on, never off.
Suggestions posted (all non-blocking)
- No escape hatch (
example_utils.py:712-723) — the override is unconditional, so a multi-GPU DiffusionGemma load can no longer getautoat all. Packing onto GPU 0 plus the 0.8 cap means a checkpoint that loaded fine underautocould now OOM during calibration with no CLI way back. Theprintmakes it visible and for the reported 26B case the trade is clearly right; worth a follow-up flag or a README note. is_diffusion_gemmaskips nestedtext_config(example_utils.py:307-317) — its two neighbours in this file (get_original_hf_quant_method,has_pack_quantized_config) both walk(config, config.text_config). A conditional-generation wrapper config that only names the family on the inner config would slip past and silently keepauto. Speculative for published checkpoints, since top-levelarchitecturescovers them.- Stale test module docstring (
test_example_utils.py:15-19, outside the diff so noted here) — still claims the file covers onlyload_mtp_weights, though it already coveredget_original_hf_quant_method,_resolve_init_config, andget_modeldtype selection, and now adds device-map selection. One-line fix while you're in the file.
Note on the earlier has_pack_quantized_config item
The previous round flagged that branch hardcoding device_map="auto"; it still does at head. I'm not counting it as blocking: it's pre-existing code that this diff doesn't touch, and the existing --use_seq_device_map flag is equally inert there — so it's a standing limitation of that branch rather than a gap introduced by this fix. Reasonable to leave for a follow-up.
Risk
Low. Examples-only change, additive, single narrowly-gated branch with an explanatory print, and the detection heuristic is now covered by eight parameterized cases.
6acad70 to
6c28e5b
Compare
cjluo-nv
left a comment
cjluo-nv
left a comment
There was a problem hiding this comment.
Bot review (bedrock-claude-opus-5) — DM the bot to share feedback.
Re-review of #2041 (auto device_map="sequential" for DiffusionGemma in examples/hf_ptq/example_utils.get_model). Everything substantive from the prior rounds is now resolved in-tree, and the two items left open are explicitly deferred with reasoning the author has stated on-thread.
Verified at head:
- Nested
text_config(previous SUGGESTION) →is_diffusion_gemmanow iterates(hf_config, getattr(hf_config, "text_config", None)), matchingget_original_hf_quant_method(625) andhas_pack_quantized_config(746). Three newtest_is_diffusion_gemmacases cover nestedmodel_type, nestedarchitectures, and a nestedgemma3negative.getattr(None, ...)on a missingtext_configdegrades to""/[]correctly, and the bareSimpleNamespace()case pins that. - Missing test (critical, prior round) →
test_get_model_device_map_for_diffusion_gemmanow has four parametrized cases:diffusion_gemma@2 GPUs →sequential+max_memory == {0: 819.2};model_type=None+ architecture-only @2 GPUs →sequential(CodeRabbit's last request, addressed in6acad706); @1 GPU →autowith nomax_memory;gemma3@2 GPUs →auto. I tracedget_modelunder the stubs —AutoConfig/init_empty_weights/get_max_memory/infer_auto_device_map/torch.cuda.device_countall patch the real path,infer_auto_device_map → {"": 0}keepson_cpuFalse so theautocases legitimately assertmax_memoryabsent, and the emptyparameters()keepsis_model_on_gpufrom blowing up. Real selection logic, not a stub. - Single-GPU regression + overstated comment → still fixed (
torch.cuda.device_count() > 1gate; comment states the "checkpoints that fit" limitation). - Placement → the block sits before
if use_seq_device_map:so themax_memorycap applies, and after it thebart/t5device_map = Noneoverrides can't collide.
Deferred with stated rationale (accepting as reasonable, not blocking):
has_pack_quantized_configstill hardcodesdevice_map="auto"and dropsmax_memory, so a pack-quantized DiffusionGemma checkpoint can still hit the meta-tensor crash. Author confirmed this is pre-existing (that branch already ignores--use_seq_device_mapfor every pack-quantized model) and that fixing it changes loading behavior for all such models, so it belongs in its own PR. The author also correctly flagged that CodeRabbit's "✅ Addressed in commit 6ef39d9" badge on that thread is wrong — good catch, and it removes the label/diff mismatch I raised last round.- Consolidating the DiffusionGemma predicate into
model_utils.pyand a tri-state escape-hatch flag (+ README note) are both tracked as follow-ups;CUDA_VISIBLE_DEVICES=0is a workable stopgap given the multi-GPU gate.
Nit, non-blocking: the test module docstring (lines 15-19) still claims the file covers only load_mtp_weights, though it now also covers get_original_hf_quant_method, _resolve_init_config, and two get_model selection paths — one-line fix if you touch the file again.
No licensing changes (existing headers only). No prompt-injection content in the PR text or comments.
DiffusionGemma ties weights between its encoder and decoder. Loading it
with device_map="auto" (balanced) can place the two sides of a tied pair
on different GPUs; the tie cannot then be honored and one side is left on
the meta device, so the pre-quantization preview fails with:
RuntimeError: Tensor.item() cannot be called on meta tensors
Detect DiffusionGemma configs in get_model and select
device_map="sequential", which keeps tied modules together. This mirrors
the existing per-model handling for bart and t5, where device_map="auto"
similarly mis-shards tied encoder/decoder weights.
Multi-GPU only; single-GPU runs were unaffected. Previously this required
passing --use_seq_device_map manually.
Fixes NVBug 6524370
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Signed-off-by: Juhi Mittal <juhim@nvidia.com>
6c28e5b to
e4d0214
Compare
|
2 participants
What does this PR do?
Type of change: Bug fix
Fixes NVBug 6524370.
DiffusionGemmaties weights between its encoder and decoder.get_modelloads withdevice_map="auto"(examples/hf_ptq/example_utils.py), and"auto"is an alias for"balanced"— accelerate splits the model evenly across all visible GPUs by size, with no awareness of tied parameters. On multi-GPU it can place the two sides of a tied pair on different devices; the tie then cannot be honored and one side is left on themetadevice.The pre-quantization preview in
pre_quantizethen reaches(input_ids == self.config.image_token_id).any()ingeneration_diffusion_gemma.pyand fails:This is multi-GPU-only by construction: with one visible GPU the balanced split is trivial, nothing is separated, and nothing lands on
meta.This PR detects DiffusionGemma configs in
get_modeland selectsdevice_map="sequential", which fills one GPU before spilling to the next and so keeps tied modules together. It mirrors the existing per-model handling forbartandt5, wheredevice_map="auto"similarly mis-shards tied encoder/decoder weights.Detection reads
model_typeandarchitecturesfrom the config and ignores underscores, since the family is spelleddiffusion_gemmain the Transformers module path andDiffusionGemmain the class name.Usage
No API change. Previously this needed the flag passed manually:
It is now selected automatically, and the model load logs:
Passing
--use_seq_device_mapexplicitly still works and is unaffected.Testing
diffusiongemma-26B-A4B-itand thenvfp4_experts_onlyrecipe;--use_seq_device_mapresolves the crash, confirming the device-mapping cause.diffusiongemma-26B-A4B-itloads correctly and the meta-tensor crash no longer reproduces.is_diffusion_gemmachecked against both config spellings,architectures=None,architectures=[], and agemma3negative to confirm no over-match —get_model_typealready ordersDiffusionGemmabeforeGemmafor exactly this substring-collision reason.pre-commit run --files examples/hf_ptq/example_utils.pypasses (ruff, ruff-format, mypy, bandit).Before your PR is "Ready for review"
CONTRIBUTING.md: N/Aget_modeldevice-map selection; happy to add a config-level test foris_diffusion_gemmaif wanted.Additional Information
NVBug 6524370. Same class of failure as the existing
t5workaround inget_model; a general "any tied encoder/decoder model" rule was considered but rejected, sincetie_word_embeddings=Trueholds for most decoder-only LLMs whereautois fine and forcing sequential would regress large-model runs.🤖 Generated with Claude Code
Summary by CodeRabbit