[None][feat] BREAKING: add per-model transceiver runtime auto selection

[nv-xtf]

Description

Add per-model auto selection for the KV-cache transceiver runtime on the standard PyTorch model-loading path.

• Add auto as the default value of CacheTransceiverConfig.transceiver_runtime.
• Add get_preferred_transceiver_runtime() for models to declare their preferred runtime.
• Preserve explicit user settings (CPP, PYTHON, or None).
• Apply the Python runtime preference only when the effective backend is NIXL.
• Fall back to the C++ transceiver when the model does not provide a preference, the effective backend is incompatible, model resolution is unavailable, or the loading path does not use the standard PyTorch model loader.
• Preserve the original model preference when the resolved execution class is redirected, such as the MTP draft-model path.
• Prevent get_model_defaults() from implicitly creating or enabling cache_transceiver_config.
• Share DEFAULT backend resolution between runtime selection and transceiver creation.
• Update the telemetry manifest, generated documentation, and related tests.

No model opts into the Python transceiver in this PR, so the effective default remains the C++ transceiver until model-specific migrations are added.

Model Opt-in

To provide an automatic preference for a model, override get_preferred_transceiver_runtime() in its ForCausalLM implementation:

@classmethod
def get_preferred_transceiver_runtime(cls, pretrained_config=None):
    return "PYTHON"

Valid return values are "PYTHON", "CPP", or None (defer to the global default, i.e. the C++ transceiver). Shared implementation classes can inspect pretrained_config.architectures to return different preferences for different model architectures.

The user can always override the model preference explicitly:

cache_transceiver_config:
  backend: NIXL
  transceiver_runtime: CPP  # CPP, PYTHON, or auto

Resolution outcomes:

User setting	Model preference	Effective backend	Result
unset / auto	PYTHON	NIXL	Python transceiver
unset / auto	PYTHON	UCX/MPI/MOONCAKE	C++ transceiver (fallback, logged)
unset / auto	None	any	C++ transceiver
CPP / None	any	any	C++ transceiver (user wins)
PYTHON	any	NIXL	Python transceiver (user wins)
PYTHON	any	non-NIXL	Error (unchanged behavior)

auto only selects the runtime after the user enables cache_transceiver_config; it does not enable disaggregated serving by itself. On paths that skip the standard PyTorch model loader (e.g. AutoDeploy), auto falls back to the C++ transceiver — in mixed deployments, set transceiver_runtime explicitly on both sides to keep them aligned.

The preference must not be declared through get_model_defaults(), because model defaults must not create or enable a transceiver configuration (rejected at load time).

With TLLM_LOG_LEVEL=INFO, adoption is visible at startup: Resolved transceiver_runtime='auto' to 'PYTHON' for GptOssForCausalLM. followed by Using KvCacheTransceiverV2.

Breaking Change

The default value of CacheTransceiverConfig.transceiver_runtime changes from None (C++ runtime) to auto. Existing explicit values (CPP, PYTHON, and None) remain supported. No model opts into the Python runtime in this PR, so the effective runtime still falls back to C++ unless a model-specific preference is added later.

Test Coverage

LLM_MODELS_ROOT=/tmp python -m pytest -q tests/unittest/llmapi/test_llm_args.py::TestTransceiverRuntimeAutoResolution

python -m pytest -q tests/unittest/usage/test_llmapi_config_capture.py::test_collect_llm_api_config_captures_transceiver_runtime_categorical

LLM_MODELS_ROOT=<model-root> python -m pytest -q tests/unittest/llmapi/test_llm_args.py tests/unittest/usage/

PR Checklist

Please review the following before submitting your PR:

• PR description clearly explains what and why. If using CodeRabbit's summary, please make sure it makes sense.

• PR Follows TRT-LLM CODING GUIDELINES to the best of your knowledge.

• Test cases are provided for new code paths (see test instructions)

• If PR introduces API changes, an appropriate PR label is added - either api-compatible or api-breaking. For api-breaking, include BREAKING in the PR title.

• Any new dependencies have been scanned for license and vulnerabilities

• CODEOWNERS updated if ownership changes

• Documentation updated as needed

• Update tava architecture diagram if there is a significant design change in PR.

• The reviewers assigned automatically/manually are appropriate for the PR.

• Please check this after reviewing the above items as appropriate for this PR.

GitHub Bot Help

To see a list of available CI bot commands, please comment /bot help.

Summary by CodeRabbit

• New Features

  • Added an automatic option for transceiver runtime selection, letting the system choose the best runtime when not set explicitly.
  • Expanded support for an additional runtime value in configuration capture and validation.

• Bug Fixes

  • Improved configuration loading so automatic runtime settings are resolved consistently across model setups and checkpoint-based workflows.
  • Prevented model defaults from overriding transceiver configuration in unsupported ways.

• Documentation

  • Updated telemetry/configuration docs to reflect the new runtime option and updated field counts.

[coderabbitai]

📝 Walkthrough

Walkthrough

Adds an "auto" option for cache_transceiver_config.transceiver_runtime, resolved via a new model classmethod get_preferred_transceiver_runtime and backend compatibility checks in a new _resolve_transceiver_runtime_auto helper. Wires resolution into ModelLoader, refactors default backend selection off env-var scanning, and updates telemetry docs, golden manifest, and tests.

Changes

Transceiver Runtime Auto Resolution

Layer / File(s)	Summary
Config schema and default backend resolution tensorrt_llm/llmapi/llm_args.py	CacheTransceiverConfig.transceiver_runtime now accepts "auto" (new default) and gains _resolve_default_backend() to resolve DEFAULT backend via legacy env vars.
Model base preference API tensorrt_llm/_torch/models/modeling_utils.py	Adds get_preferred_transceiver_runtime classmethod (default None), clarifies get_model_defaults docs, reformats imports, and simplifies an error message.
Auto-resolution helper and guard tensorrt_llm/llmapi/llm_utils.py	apply_model_defaults_to_llm_args now rejects cache_transceiver_config in model defaults; new _resolve_transceiver_runtime_auto resolves "auto" using model preference and backend compatibility.
ModelLoader integration tensorrt_llm/_torch/pyexecutor/model_loader.py	Invokes the resolution helper early when no checkpoint loader is used, and derives preference_cls from checkpoint architectures to resolve runtime after model defaults are applied.
KV cache transceiver backend selection refactor tensorrt_llm/_torch/pyexecutor/kv_cache_transceiver.py	Normalizes "auto" to None, replaces local getenv scanning with _resolve_default_backend(), logging a warning on env override.
Tests tests/unittest/llmapi/test_llm_args.py, tests/unittest/usage/test_llmapi_config_capture.py	Adds TestTransceiverRuntimeAutoResolution suite and a telemetry capture test for transceiver_runtime categorical values.
Telemetry docs and golden manifest docs/source/developer-guide/telemetry.md, tensorrt_llm/usage/llm_args_golden_manifest.json	Updates field counts and categorical allowlists to include "auto" for transceiver_runtime.

Estimated code review effort: 3 (Moderate) | ~25 minutes

Possibly related PRs

• NVIDIA/TensorRT-LLM#14398: Extends the same telemetry capture/golden-manifest framework for CacheTransceiverConfig fields that this PR modifies.

Suggested reviewers: yuxianq, dongxuy04, pcastonguay

🚥 Pre-merge checks | ✅ 4 | ❌ 1

❌ Failed checks (1 warning)

Check name	Status	Explanation	Resolution
Docstring Coverage	⚠️ Warning	Docstring coverage is 58.82% which is insufficient. The required threshold is 80.00%.	Write docstrings for the functions missing them to satisfy the coverage threshold.

✅ Passed checks (4 passed)

Check name	Status	Explanation
Linked Issues check	✅ Passed	Check skipped because no linked issues were found for this pull request.
Out of Scope Changes check	✅ Passed	Check skipped because no linked issues were found for this pull request.
Title check	✅ Passed	The title is specific, concise, and matches the PR's main change, including the breaking API note.
Description check	✅ Passed	The description covers the change, opt-in behavior, breaking impact, test coverage, and checklist items required by the template.

✨ Finishing Touches

🧪 Generate unit tests (beta)

• Create PR with unit tests

---

_{Comment @coderabbitai help to get the list of available commands.}

[coderabbitai]

🧹 Nitpick comments (1)

tests/unittest/llmapi/test_llm_args.py (1)

3017-3309: 🎯 Functional Correctness | 🔵 Trivial | ⚡ Quick win

Coverage gap: no test for explicit "CPP" model preference adoption.

The suite thoroughly covers "PYTHON" adoption, fallback-to-None, and invalid-value rejection, but no fake model returns "CPP" from get_preferred_transceiver_runtime. Add a _PreferCppTransceiverModel fixture and a test_cpp_preference_adopted case (parallel to test_model_preference_adopted) to guard the direct-assignment branch in _resolve_transceiver_runtime_auto against future regressions.

As per path instructions, "Act as a QA engineer reviewing test changes and coverage for TensorRT-LLM... suggest concrete list file names and whether coverage is sufficient, insufficient, or needs follow-up outside the PR."

🤖 Prompt for AI Agents

Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.

In `@tests/unittest/llmapi/test_llm_args.py` around lines 3017 - 3309, The
transceiver-runtime auto-resolution tests are missing coverage for the explicit
"CPP" preference branch in _resolve_transceiver_runtime_auto. Add a
_PreferCppTransceiverModel fixture alongside _PreferPythonTransceiverModel, then
add a test_cpp_preference_adopted case parallel to test_model_preference_adopted
to verify that get_preferred_transceiver_runtime returning "CPP" is adopted
correctly when the config is auto. Keep the new test within
TestTransceiverRuntimeAutoResolution and use the same _disagg_args helper so the
branch is exercised consistently.

Source: Path instructions

🤖 Prompt for all review comments with AI agents

Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.

Nitpick comments:
In `@tests/unittest/llmapi/test_llm_args.py`:
- Around line 3017-3309: The transceiver-runtime auto-resolution tests are
missing coverage for the explicit "CPP" preference branch in
_resolve_transceiver_runtime_auto. Add a _PreferCppTransceiverModel fixture
alongside _PreferPythonTransceiverModel, then add a test_cpp_preference_adopted
case parallel to test_model_preference_adopted to verify that
get_preferred_transceiver_runtime returning "CPP" is adopted correctly when the
config is auto. Keep the new test within TestTransceiverRuntimeAutoResolution
and use the same _disagg_args helper so the branch is exercised consistently.

---

ℹ️ Review info

⚙️ Run configuration

Configuration used: Path: .coderabbit.yaml

Review profile: CHILL

Plan: Enterprise

Run ID: 1e30e36e-8e9e-4c45-9a6b-84e07a9dcb65

📥 Commits

Reviewing files that changed from the base of the PR and between 60583d2 and 64bcec3.

📒 Files selected for processing (9)

• docs/source/developer-guide/telemetry.md
• tensorrt_llm/_torch/models/modeling_utils.py
• tensorrt_llm/_torch/pyexecutor/kv_cache_transceiver.py
• tensorrt_llm/_torch/pyexecutor/model_loader.py
• tensorrt_llm/llmapi/llm_args.py
• tensorrt_llm/llmapi/llm_utils.py
• tensorrt_llm/usage/llm_args_golden_manifest.json
• tests/unittest/llmapi/test_llm_args.py
• tests/unittest/usage/test_llmapi_config_capture.py

[nv-xtf]

/bot run

[tensorrt-cicd]

PR_Github #58389 [ run ] triggered by Bot. Commit: 3c583b0 Link to invocation

[tensorrt-cicd]

PR_Github #58389 [ run ] completed with state SUCCESS. Commit: 3c583b0
/LLM/main/L0_MergeRequest_PR pipeline #47012 completed with status: 'FAILURE'

CI Report

⚠️ Action Required:

• Please check the failed tests and fix your PR
• If you cannot view the failures, ask the CI triggerer to share details
• Once fixed, request an NVIDIA team member to trigger CI again

CI Agent Failure Analysis

Link to invocation

[nv-xtf]

/bot run --disable-fail-fast

[tensorrt-cicd]

PR_Github #58445 [ run ] triggered by Bot. Commit: 3c583b0 Link to invocation