[Feature] Transform Module - ModuleTransform and Ray Service Refactor

[vmoens]

Stack from ghstack (oldest at bottom):

• [Quality] Fix flaky test #3211
• [Feature] Documentation #3192
• [Feature] SAC Trainer #3191
• [Feature] PPO Trainer Updates #3190
• [Feature] Trainer Algorithms - Configuration System #3189
• [Feature] Trainer Infrastructure - Timing and Utilities #3188
• [Feature] Collectors - Weight Sync Scheme Integration #3187
• [Feature] vLLM Weight Synchronization Schemes #3186
• [Feature] Weight Synchronization Schemes - Core Infrastructure #3185
• -> [Feature] Transform Module - ModuleTransform and Ray Service Refactor #3184
• [Feature] Storage Shared Initialization for Multiprocessing #3183

[pytorch-bot]

🔗 Helpful Links

🧪 See artifacts and rendered test results at hud.pytorch.org/pr/pytorch/rl/3184

• 📄 Preview Python docs built from this PR

Note: Links to docs will display an error until the docs builds have been completed.

❗ 1 Active SEVs

There are 1 currently active SEVs. If your PR is affected, please view them below:

• [ROCm][CI] Machines under the label linux.rocm.gpu.2 are undergoing maintenance.

❌ 5 New Failures, 2 Cancelled Jobs

As of commit d7685df with merge base 071d079 ():

NEW FAILURES - The following jobs have failed:

• Build Linux Wheels / pytorch/rl (pytorch/rl, test/smoke_test.py, torchrl, .github/scripts/pre-build-script.sh) / build-manywheel-py3_10-cuda12_9 (gh)
  Process completed with exit code 1.
• Build Linux Wheels / pytorch/rl (pytorch/rl, test/smoke_test.py, torchrl, .github/scripts/pre-build-script.sh) / upload / upload-manywheel-py3_10-cuda12_9 (gh)
  Unable to download artifact(s): Artifact not found for name: pytorch_rl__3.10_cu129_x86_64
• Continuous Benchmark (PR) / CPU Pytest benchmark (gh)
  Process completed with exit code 1.
• Habitat Tests on Linux / tests (3.9, 12.8) / linux-job (gh)
  RuntimeError: Command docker exec -t e4098028abb5cdbbe20c5e8e4f3aa1983a67c829d5ed1233c1b1360cdf3ec70e /exec failed with exit code 1
• Libs Tests on Linux / unittests-gym (3.9, 12.8) / linux-job (gh)
  RuntimeError: Command docker exec -t 477f000eb6095ad9fcf5e58bb5534184701d34c4eabfbe57d74db9d17a006489 /exec failed with exit code 1

CANCELLED JOBS - The following jobs were cancelled. Please retry:

• Continuous Benchmark (PR) / GPU Pytest benchmark (gh)
  ##[error]The operation was canceled.
• LLM Tests on Linux / unittests (3.9, 12.8) / linux-job (gh)

This comment was automatically generated by Dr. CI and updates every 15 minutes.