[lhotse] Added support for re-weighting datasets with temperature on the fly.

[XuesongYang]

Summary

Add on-the-fly temperature-based dataset reweighting to the Lhotse dataloader, allowing users to control sampling distributions across nested input_cfg groups without manually recalculating weights each time datasets are added or removed.

The feature applies the formula w_i^τ / Σ w_j^τ at each nesting level, where τ is configurable per level. Validation is strict: reweight_temperature must be either a scalar (broadcast to all levels) or a list whose length exactly matches the input_cfg nesting depth.

ŵᵢ = wᵢ^τ / Σⱼ wⱼ^τ.

• τ = 1.0: Preserves original weight ratios (neutral)
• τ = 0.0: Equalizes all datasets regardless of original weights
• 0 < τ < 1.0: Over-samples smaller datasets relative to larger ones

Changes

New functionality (cutset.py)

• temperature_reweighting(weights, temperature): applies temperature scaling and normalizes weights.
• count_input_cfg_levels(config): computes maximum input_cfg nesting depth.
• Validation in read_dataset_config(): scalar is broadcast with a logged warning; list length mismatch raises ValueError.
• parse_and_combine_datasets() pops the current-level temperature and passes remaining values to child groups via propagate_attrs.

Config (dataloader.py)

• Added reweight_temperature field to LhotseDataLoadingConfig (default None)

Tests

• Unit tests for temperature_reweighting() and count_input_cfg_levels()
• Integration tests for dataloader with various temperature configurations
• Validation tests for scalar/list normalization behavior

Documentation (docs/source/audio/configs.rst)

• New "Dataset Reweighting with Temperature" section with formula, configuration examples (scalar and list), nesting depth explanation, and a multi-task-group walkthrough.

Example configs

• magpietts_lhotse.yaml / magpietts_lhotse_moe.yaml: added reweight_temperature: 1.0 (neutral default) with comment showing list alternative
• Example of preserving top-level ratios but equalize within sub-groups:

reweight_temperature: [1.0, 0.0]  # Level 1: preserve, Level 2: equalize

[Copilot]

Pull request overview

This PR introduces temperature-based dataset reweighting to the Lhotse dataloader, enabling dynamic adjustment of sampling probabilities across datasets through a configurable temperature parameter. This eliminates the need for manual weight recalculation when combining datasets with different sizes.

Key changes:

• New temperature_reweighting() function that applies temperature scaling to weights using the formula (w_i ^ temp) / sum(w_j ^ temp)
• New reweight_temperature configuration option that supports hierarchical temperature application across nested dataset groups
• Comprehensive test suite with 19 tests covering various input types and edge cases

Reviewed changes

Copilot reviewed 3 out of 3 changed files in this pull request and generated 4 comments.

File	Description
nemo/collections/common/data/lhotse/cutset.py	Adds the temperature_reweighting() function and integrates it into the dataset loading pipeline with temperature propagation through nested groups
examples/tts/conf/magpietts/magpietts_lhotse.yaml	Adds example configuration demonstrating hierarchical temperature usage with [1.0, 0.0]
tests/collections/common/test_lhotse_temperature_reweighting.py	Adds comprehensive unit and integration tests for the temperature reweighting functionality

---

💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.

[Copilot]

Pull request overview

Copilot reviewed 3 out of 3 changed files in this pull request and generated 2 comments.

---

💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.

[pzelasko]

I think you also need to add this to dataloader.py in LhotseDataLoadingConfig otherwise it won't get propagated to cutset.py functions. We should also have a test that uses this option through get_lhotse_dataloader_from_config to check it works.

[XuesongYang]

good point. indeed, my test using get_lhotse_dataloader_from_config failed to pass this new param to propagate_attrs.

Fixed now.