[RL][BugFix][Optimization] Support chunked part files loading and fix model path format in IPC snapshot strategy

[wikilsh]

Motivation

In RL training elastic recovery via IPC snapshot, two issues existed in _update_ipc_snapshot:

1. Memory spike: Loading a single large .pdparams file in one shot causes a significant memory spike, risking OOM during recovery.
2. Ambiguous file naming: The snapshot path used model_state.tp{rank}{id} without a separator between rank and id, causing naming ambiguity (e.g., rank=1, id=234 and rank=12, id=34 both produce tp1234). Additionally, the rank was previously hardcoded as tp0, causing all ranks to load rank-0's shard in multi-process scenarios.

Modifications

Refactored _update_ipc_snapshot in fastdeploy/rl/dynamic_weight_manager.py with a four-level loading priority:

1. Chunked part files (model_state.tp{rank}.{id}.part{N}.pdparams): Load multiple smaller shards sequentially in ascending numeric order, releasing memory between each chunk via gc.collect() to avoid memory spike.
2. Single full file (model_state.tp{rank}.{id}.pdparams): Standard single-file loading with corrected naming format (dot separator added).
3. Legacy format fallback (model_state.tp0{id}.pdparams): Backward-compatible fallback for checkpoints saved in the old format, ensuring smooth rolling upgrades.
4. Shared directory fallback (/shared_ipc_meta/...): Oldest legacy fallback path preserved for compatibility.

Key fixes:

• Add dot separator in snapshot file name: tp{rank}{id} → tp{rank}.{id} to eliminate naming ambiguity
• Replace hardcoded tp0 with paddle.distributed.get_rank() so each rank loads its own shard correctly
• Add legacy format (Priority 3) as a transitional fallback to avoid load failures on existing checkpoints
• Raise FileNotFoundError with clear message when no snapshot is found in any candidate path

Usage or Command

During RL elastic recovery, the trainer will automatically select the appropriate loading strategy based on available snapshot files. No additional configuration is needed.

To generate chunked part files on the trainer side, split the full state dict and save as:
model_state.tp{rank}.{id}.part0.pdparams
model_state.tp{rank}.{id}.part1.pdparams
...

Accuracy Tests

This PR only modifies the weight loading path and chunking strategy in _update_ipc_snapshot. It does not affect model forward computation or kernel logic. No accuracy regression is expected.

Checklist

• Add at least a tag in the PR title.
• Format your code, run pre-commit before commit.
• Add unit tests. Please write the reason in this PR if no unit tests.

  Added tests/rl/test_update_ipc_snapshot.py covering all 4 loading priority branches and the error path:

  • Priority 1: chunked part files loaded sequentially with gc.collect() after each part
  • Priority 1 sort: part files are loaded in ascending numeric order even when glob returns them unordered
  • Priority 2: single full .pdparams file with corrected tp{rank}.{id} naming
  • Priority 3: legacy tp0{id} format fallback for backward compatibility
  • Priority 4: /shared_ipc_meta/ shared directory fallback
  • Error path: FileNotFoundError raised when no snapshot is found in any candidate path
    All tests use unittest.mock and bypass __init__, requiring no filesystem or GPU dependencies.

• Provide accuracy results.
• This PR targets the develop branch.

[CLAassistant]

All committers have signed the CLA.

[paddle-bot]

Thanks for your contribution!

[codecov-commenter]

Codecov Report

❌ Patch coverage is 85.45455% with 8 lines in your changes missing coverage. Please review.
⚠️ Please upload report for BASE (develop@820eb60). Learn more about missing BASE report.

Files with missing lines	Patch %	Lines
fastdeploy/rl/dynamic_weight_manager.py	85.45%	6 Missing and 2 partials ⚠️

Additional details and impacted files

@@            Coverage Diff             @@
##             develop    #6852   +/-   ##
==========================================
  Coverage           ?   73.42%           
==========================================
  Files              ?      399           
  Lines              ?    55668           
  Branches           ?     8768           
==========================================
  Hits               ?    40876           
  Misses             ?    11868           
  Partials           ?     2924           

Flag	Coverage Δ
GPU	73.42% <85.45%> (?)

Flags with carried forward coverage won't be shown. Click here to find out more.

☔ View full report in Codecov by Sentry.
📢 Have feedback on the report? Share it here.

🚀 New features to boost your workflow:

• ❄️ Test Analytics: Detect flaky tests, report on failures, and find test suite problems.

[Copilot]

Pull request overview

该 PR 聚焦于 RL 训练的 IPC snapshot 弹性恢复路径：通过支持按 chunk 分片加载来降低大权重文件一次性加载导致的内存峰值，并修正 snapshot 文件命名格式以避免歧义与多进程加载错误。

Changes:

• 重构 DynamicWeightManager._update_ipc_snapshot：按“分片文件 → 单文件 → 旧格式 → shared 兜底”的优先级加载，并在分片之间 gc.collect()。
• 修复 snapshot 文件名格式：tp{rank}{id} → tp{rank}.{id}，并新增旧格式兼容分支与更清晰的 FileNotFoundError。
• 新增单测 tests/rl/test_update_ipc_snapshot.py 覆盖 4 条分支与错误路径，并验证分片排序行为。

Reviewed changes

Copilot reviewed 2 out of 2 changed files in this pull request and generated 3 comments.

File	Description
fastdeploy/rl/dynamic_weight_manager.py	实现 chunked part files 优先加载、文件名格式修复、兼容旧格式与兜底路径，并在找不到快照时抛出明确异常
tests/rl/test_update_ipc_snapshot.py	为 _update_ipc_snapshot 新增单测，覆盖 4 级优先级与异常分支，验证分片顺序与 gc.collect() 行为

---

You can also share your feedback on Copilot code review. Take the survey.