[core] Add PG + actor scheduling scalability benchmark for RL/post-training

[Yicheng-Lu-llll]

Description

This PR reviews all PG and actor related release tests and adds a PG + actor scheduling scalability benchmark for RL/post-training. If you only care about the new benchmark, just look at release/benchmarks/distributed/test_pg_actor_scalability.py and ignore the rest of the diff.

The benchmark has two modes:

• pg-per-node: one single bundle PG per worker node. Each bundle is a whole worker node; actors-per-node(default 4) actors are scheduled into each.
• pg-per-rack: one PG per rack, one bundle per node in the rack. Each bundle is a whole worker node; actors-per-node(default 4) actors are scheduled into each.

Both modes run at 2,000 and 10k node settings, and report PG creation throughput and actor scheduling throughput separately.

After this PR:

Pure Actor Scheduling Throughput release test

• E2E scheduling quality
• test_many_actors.py: 64 nodes, 156 actors/node, high density actor scheduling.
  • This should reflect the end to end overhead of placing an actor.
• whether throughput degrades as scale increases
  • many_nodes_tests/actor_test.py: 10 actors/node, tested at 1,000, 2,000, and 10k nodes.
    • This should surface the "stale resource view" issue or "GCS single thread bottleneck" and show how their impact grows as the number of nodes increases, basically, whether throughput degrades as scale increases.

Pure PG Scheduling release test

• E2E PG scheduling quality
  • Microbenchmark, 5 nodes:
    • num_pgs ∈ [10, 100, 200, 400, 800, 1600] (bundle=1), tests how scheduling scales with the number of PGs.
    • num_bundles ∈ [1, 10, 20, 40] (pgs=100), tests how scheduling scales with the number of bundles per PG.
  • test_placement_group.py: 5 nodes. 666 PGs / 3,330 bundles, measures the call latency of PG creation/deletion.
    • I actually feel like this test is quite meaningful now. not in its original intention, but its throughput will be impacted by the PG removal speed, which is a good and unique indicator.
• whether throughput degrades as scale increases
  • Large node scale PG throughput is tested in "PG + Actor among PG".

PG + Actor among PG release test

• The newly added test_pg_actor_scalability.py.
• test_many_pgs.py: 64 nodes. 1,000 small 3 bundle PGs (~25 PGs per node), then 3 actors packed into each PG. This test should be deleted, as it doesn't make much sense here and could be replaced by the new test. Will do in a separate PR.

[gemini-code-assist]

Code Review

This pull request introduces placement group and actor scheduling scalability benchmarks, along with several new cluster compute configurations (up to 10,000 nodes) and release test definitions. It also adds resource leak checks to the existing actor test. A critical issue was identified in the new benchmark script where bundle sizing is determined by the maximum CPU count across all nodes, including the head node. Since the head node has 64 CPUs and the worker nodes only have 2, the placement groups will require 64 CPUs and remain permanently pending on worker nodes, causing the benchmark to hang. To resolve this, the head node should be filtered out when calculating worker node CPU capacity.

[cursor]

Cursor Bugbot has reviewed your changes using default effort and found 1 potential issue.

^{Reviewed by Cursor Bugbot for commit 8040a8e. Configure here.}