[None][bugfix] Guard e_score_correction_bias load for DeepSeek-V2-Lite

[waynehacking8]

Summary

• DeepseekV3Gate.load_weights accesses e_score_correction_bias unconditionally, but lite DeepSeek checkpoints (scoring_func=softmax, q_lora_rank=None, e.g. DeepSeek-V2-Lite) do not have this tensor — it is V3-specific (sigmoid routing with auxiliary-loss-free load balancing).
• The PyTorch DeepseekV3 model class already supports these lite checkpoints (the is_lite = q_lora_rank is None branches throughout modeling_deepseekv3.py), so a lite checkpoint served under a registered V3-family architecture reaches this gate and hits a KeyError on load (e.g. an NVFP4-quantized DeepSeek-V2-Lite).
• Fix: guard the .copy_() with a key-existence check, and zero-initialize the parameter (torch.empty → torch.zeros) so the missing-key path is a safe no-op — e_score_correction_bias is added to the routing scores (scores + e_score_correction_bias in noaux_tc), so zero means "no correction". The same optional-bias guard is used by sibling gates (modeling_laguna.py, modeling_minimaxm2.py).

Reproduction

# A lite (q_lora_rank=None, softmax-routed) checkpoint served via the DeepseekV3
# PyTorch class — e.g. an NVFP4-quantized DeepSeek-V2-Lite:
#   KeyError: 'e_score_correction_bias' in DeepseekV3Gate.load_weights

Test plan

• Verified on RTX PRO 6000 (SM120) with DeepSeek-V2-Lite NVFP4 (modelopt 0.37, kv_b_proj kept BF16) — warmup + generate produce coherent output after the fix
• Existing unit tests pass (pytest tests/unittest/)

PR Checklist

Please review the following before submitting your PR:

• PR description clearly explains what and why. If using CodeRabbit's summary, please make sure it makes sense.

• PR Follows TRT-LLM CODING GUIDELINES to the best of your knowledge.

• Test cases are provided for new code paths (see test instructions)

• If PR introduces API changes, an appropriate PR label is added - either api-compatible or api-breaking. For api-breaking, include BREAKING in the PR title.

• Any new dependencies have been scanned for license and vulnerabilities

• CODEOWNERS updated if ownership changes

• Documentation updated as needed

• Update tava architecture diagram if there is a significant design change in PR.

• The reviewers assigned automatically/manually are appropriate for the PR.

• Please check this after reviewing the above items as appropriate for this PR.

Summary by CodeRabbit

Release Notes

• Bug Fixes
  • Fixed an issue where loading DeepSeekV3 models could fail due to missing checkpoint weights, improving model loading robustness.

[coderabbitai]

📝 Walkthrough

Walkthrough

The pull request adds a defensive check to DeepseekV3Gate.load_weights that conditionally loads e_score_correction_bias only when the checkpoint contains that key, preventing load failures on checkpoints without this optional bias.

Changes

DeepseekV3Gate Conditional Bias Loading

Layer / File(s)	Summary
Conditional e_score_correction_bias loading tensorrt_llm/_torch/models/modeling_deepseekv3.py	DeepseekV3Gate.load_weights now checks for the presence of e_score_correction_bias in checkpoint weights before copying it, allowing compatibility with checkpoints that do not include this optional tensor.

Estimated code review effort

🎯 1 (Trivial) | ⏱️ ~3 minutes

🚥 Pre-merge checks | ✅ 4 | ❌ 1

❌ Failed checks (1 warning)

Check name	Status	Explanation	Resolution
Docstring Coverage	⚠️ Warning	Docstring coverage is 0.00% which is insufficient. The required threshold is 80.00%.	Write docstrings for the functions missing them to satisfy the coverage threshold.

✅ Passed checks (4 passed)

Check name	Status	Explanation
Title check	✅ Passed	The title clearly identifies the main change: guarding e_score_correction_bias load for DeepSeek-V2-Lite, which is the core fix addressing a KeyError when loading V2-Lite variants.
Linked Issues check	✅ Passed	Check skipped because no linked issues were found for this pull request.
Out of Scope Changes check	✅ Passed	Check skipped because no linked issues were found for this pull request.
Description check	✅ Passed	The pull request description comprehensively explains the issue, root cause, solution, reproduction steps, and testing. It includes proper context about DeepSeek-V2-Lite variants lacking the V3-specific e_score_correction_bias tensor, justifies the fix design (zero-initialization for safe no-op), and provides verification details.

_{✏️ Tip: You can configure your own custom pre-merge checks in the settings.}

✨ Finishing Touches

🧪 Generate unit tests (beta)

• Create PR with unit tests

---

_{Comment @coderabbitai help to get the list of available commands and usage tips.}

[coderabbitai]

Actionable comments posted: 1

🤖 Prompt for all review comments with AI agents

Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.

Inline comments:
In `@tensorrt_llm/_torch/models/modeling_deepseekv3.py`:
- Around line 899-902: The e_score_correction_bias parameter is currently
created with torch.empty (uninitialized) and left as garbage when the checkpoint
lacks the "e_score_correction_bias" key; update initialization so
e_score_correction_bias is created zero-initialized (use torch.zeros with the
same shape and dtype) in the model __init__ (the same place where torch.empty is
used) so that runtime access in the routing method (where
e_score_correction_bias is read) gets valid zeros if the checkpoint omits the
key; alternatively, if you prefer not to change __init__, ensure the load path
that checks weights[0] sets self.e_score_correction_bias.zero_() when the key is
missing—prefer the __init__ zero-init approach to match LagunaGate.

🪄 Autofix (Beta)

Fix all unresolved CodeRabbit comments on this PR:

• Push a commit to this branch (recommended)
• Create a new PR with the fixes

---

ℹ️ Review info

⚙️ Run configuration

Configuration used: Path: .coderabbit.yaml

Review profile: CHILL

Plan: Enterprise

Run ID: 194db519-88d0-4c16-b0bf-066c6b503360

📥 Commits

Reviewing files that changed from the base of the PR and between b852703 and 830d7e5.

📒 Files selected for processing (1)

• tensorrt_llm/_torch/models/modeling_deepseekv3.py

[waynehacking8]

Status check: this PR has been mergeable and review-ready since 06-09 (coderabbit pass addressed at open time). Could a maintainer trigger CI (/bot run) when convenient? Happy to rebase or adjust if anything in the recent refactors moved — the fix still applies cleanly to current main.

[waynehacking8]

Rebased onto current main (50ea42aff) — MERGEABLE again. Main's DeepseekV3Gate.load_weights gained allow_partial_loading since this PR was opened, and its strict-mode assert required e_score_correction_bias unconditionally — so the V2-Lite failure this PR fixes had survived as an AssertionError instead of a KeyError. The rebase folds the fix into the new structure: weight is still asserted in strict mode, but e_score_correction_bias stays optional in both modes (V2-Lite / scoring_func=softmax checkpoints legitimately omit it; the parameter is zero-initialized per the earlier CodeRabbit round). allow_partial_loading behavior is unchanged. Verified: py_compile clean; semantics identical to the originally GPU-tested fix (DeepSeek-V2-Lite NVFP4 on RTX PRO 6000, warmup + generate clean).