feat(turbomind): Derive composable TurboMind parallel configurations

[lzhangzz]

Summary

This PR replaces TurboMind's ad hoc parallelism calculations with a constraint-based derivation system for TP, CP, EP, and DP.

The new system supports configurations where expert parallelism and MLP tensor parallelism are both greater than 1. For example:

tp=4, ep=2 -> attn_tp=4, mlp_tp=2, ep=2

Parallel layout derivation

The new parallel_config.py derives per-module parallel sizes using these invariants:

device_num = outer_dp * attn_dp * attn_tp * cp
device_num = outer_dp * ep * mlp_tp
dp = outer_dp * attn_dp

attn_tp == tp or mlp_tp == tp
tp % attn_tp == 0
tp % mlp_tp == 0

The solver:

• Computes dp * tp * gcd(cp, ep) as the device-count upper bound.
• Validates an explicit device_num, or scans valid divisors within the available GPU count.
• Chooses the largest valid device count.
• Scans divisors of dp to derive outer_dp, attention DP/TP, and MLP TP.
• Rejects configurations that cannot satisfy the layout constraints.

This keeps the derivation independent of CUDA and compiled extensions, making it easier to test and reason about.

Integration changes

• Route TurboMind configuration through the new derivation system.
• Preserve explicit per-module parallel configurations.
• Include attn_cp_size when completing partial explicit configurations.
• Keep EP restricted to single-node execution.
• Allow mlp_tp_size > 1 together with ep > 1.
• Update the TurboMind smoke-test script to pass and report TP, CP, EP, and DP consistently.

Compatibility

Legacy configurations such as tp=1, ep=N no longer describe a valid layout. They must specify the global tensor-parallel width explicitly, for example:

tp=N, ep=N

This ensures the requested global parallelism is consistent with every derived attention and MLP parallel group.

[Copilot]

Pull request overview

This PR introduces a constraint-based derivation mechanism for TurboMind’s per-module parallel layout (attention TP/DP/CP, MLP TP, outer DP) from user-facing dp/tp/ep/cp, and wires that derivation into the TurboMind engine configuration path. It also updates the TurboMind smoke-test script to accept and report tp/cp/ep/dp consistently (plus communicator selection).

Changes:

• Add lmdeploy/turbomind/parallel_config.py to derive a valid parallel layout (and optionally device count) from dp/tp/ep/cp constraints.
• Route update_parallel_config() through the new derivation logic and ensure partial explicit per-module configs include attn_cp_size.
• Extend scripts/test_turbomind_model.py CLI to support --ep, --dp, and --communicator, with defaults of 1 for parallelism args.

Reviewed changes

Copilot reviewed 3 out of 3 changed files in this pull request and generated 2 comments.

File	Description
scripts/test_turbomind_model.py	Adds CLI/reporting for ep, dp, and communicator; makes parallelism args default to 1.
lmdeploy/turbomind/turbomind.py	Integrates the new constraint-based derivation into TurboMind config completion.
lmdeploy/turbomind/parallel_config.py	New stdlib-only parallel layout derivation/validation module.

---

💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.