[None][fix] Clamp small non-zero sampling temperature

[chfeng-cs]

Summary by CodeRabbit

• Bug Fixes
  • Improved temperature validation to prevent very small nonzero values from causing unstable sampling behavior.
  • Very small temperatures are now automatically adjusted up to a safe minimum value.
  • Added coverage to verify the adjusted temperature is reflected correctly in sampling output.

Description

Fix #15715 .This PR clamps very small non-zero SamplingParams.temperature values to 1e-2 before they are passed to the sampling backend.

temperature=0 continues to represent greedy decoding, while positive values below the stability threshold are normalized to avoid excessive logit scaling from logits / temperature.

TensorRT-LLM currently validates that temperature >= 0, but values such as 1e-12 can still enter the sampling path and may lead to inf, nan, or unstable sampling behavior in edge cases.

This mirrors the numerical stability guard used by vLLM while preserving the existing greedy decoding semantics for temperature=0.

Changes

Added MIN_SAMPLING_TEMPERATURE = 1e-2.

Clamped 0 < temperature < MIN_SAMPLING_TEMPERATURE to MIN_SAMPLING_TEMPERATURE in SamplingParams._validate().

Added debug logging when a small non-zero temperature is clamped.

Added a unit test covering temperature=0, very small positive temperatures, the threshold value, and normal temperatures.

Test Coverage

pytest tests/unittest/llmapi/test_sampling_params.py -k temperature_clamps_small_nonzero_values

PR Checklist

Please review the following before submitting your PR:

• PR description clearly explains what and why. If using CodeRabbit's summary, please make sure it makes sense.

• PR Follows TRT-LLM CODING GUIDELINES to the best of your knowledge.

• Test cases are provided for new code paths (see test instructions)

• If PR introduces API changes, an appropriate PR label is added - either api-compatible or api-breaking. For api-breaking, include BREAKING in the PR title.

• Any new dependencies have been scanned for license and vulnerabilities

• CODEOWNERS updated if ownership changes

• Documentation updated as needed

• Update tava architecture diagram if there is a significant design change in PR.

• The reviewers assigned automatically/manually are appropriate for the PR.

• Please check this after reviewing the above items as appropriate for this PR.

[coderabbitai]

📝 Walkthrough

Walkthrough

Adds a MIN_SAMPLING_TEMPERATURE = 1e-2 constant to sampling_params.py and updates SamplingParams._validate() to clamp temperature values in the range (0, MIN_SAMPLING_TEMPERATURE) up to MIN_SAMPLING_TEMPERATURE with a debug log. A parametrized unit test verifies the clamping behavior.

Temperature Clamping

Layer / File(s)	Summary
Constant and clamping logic tensorrt_llm/sampling_params.py	Introduces MIN_SAMPLING_TEMPERATURE = 1e-2 and adds a clamping branch in _validate() for 0 < temperature < MIN_SAMPLING_TEMPERATURE, emitting a debug log.
Parametrized clamping test tests/unittest/llmapi/test_sampling_params.py	Adds test_temperature_clamps_small_nonzero_values with parametrized inputs verifying temperature and _get_sampling_config().temperature are clamped; also updates imports to include MIN_SAMPLING_TEMPERATURE.

Estimated code review effort

🎯 1 (Trivial) | ⏱️ ~5 minutes

🚥 Pre-merge checks | ✅ 4 | ❌ 1

❌ Failed checks (1 warning)

Check name	Status	Explanation	Resolution
Docstring Coverage	⚠️ Warning	Docstring coverage is 40.00% which is insufficient. The required threshold is 80.00%.	Write docstrings for the functions missing them to satisfy the coverage threshold.

✅ Passed checks (4 passed)

Check name	Status	Explanation
Title check	✅ Passed	The title is concise, specific, and accurately describes the main fix.
Description check	✅ Passed	The description includes the required sections and clearly explains the problem, fix, and test coverage.
Linked Issues check	✅ Passed	The changes implement the linked bug fix by clamping tiny non-zero temperatures while preserving temperature=0 behavior.
Out of Scope Changes check	✅ Passed	The diff is limited to the temperature clamp and its unit test, with no unrelated changes.

✨ Finishing Touches

🧪 Generate unit tests (beta)

• Create PR with unit tests

---

_{Comment @coderabbitai help to get the list of available commands.}

[coderabbitai]

Actionable comments posted: 1

🤖 Prompt for all review comments with AI agents

Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.

Inline comments:
In `@tensorrt_llm/sampling_params.py`:
- Around line 376-381: Update the public documentation for SamplingParams to
reflect the new temperature normalization behavior. In the SamplingParams class
docstring (and any temperature field description used by the user-facing API),
add that values greater than 0 but below MIN_SAMPLING_TEMPERATURE are clamped to
MIN_SAMPLING_TEMPERATURE, while 0 still means greedy decoding. Keep the wording
aligned with the existing temperature description in sampling_params.py so the
contract matches the behavior in the temperature handling logic.

🪄 Autofix (Beta)

Fix all unresolved CodeRabbit comments on this PR:

• Push a commit to this branch (recommended)
• Create a new PR with the fixes

---

ℹ️ Review info

⚙️ Run configuration

Configuration used: Path: .coderabbit.yaml

Review profile: CHILL

Plan: Enterprise

Run ID: 68c9e8a0-1271-425b-884b-35a8a0e8f402

📥 Commits

Reviewing files that changed from the base of the PR and between fde8e0a and 770deb1.

📒 Files selected for processing (2)

• tensorrt_llm/sampling_params.py
• tests/unittest/llmapi/test_sampling_params.py