Skip to content

[BugFix] Collector robustness & Async fixes#3325

Merged
vmoens merged 4 commits into
gh/vmoens/193/basefrom
gh/vmoens/193/head
Jan 13, 2026
Merged

[BugFix] Collector robustness & Async fixes#3325
vmoens merged 4 commits into
gh/vmoens/193/basefrom
gh/vmoens/193/head

Conversation

@vmoens

@vmoens vmoens commented Jan 12, 2026 •
edited
Loading

Copy link
Copy Markdown
Collaborator

Stack from ghstack (oldest at bottom):

Add worker health checks and timeout logging to MultiAsyncCollector.
Add _recv_and_check to MultiCollector for safer pipe communication.
Improved pause() and start() in MultiCollector.
Fix init_random_frames support for .start() mode by using
replay_buffer.write_count when available.

@vmoens
Update
eab980c 
[ghstack-poisoned]
@pytorch-bot

pytorch-bot Bot commented Jan 12, 2026 •
edited
Loading

Copy link
Copy Markdown

🔗 Helpful Links

🧪 See artifacts and rendered test results at hud.pytorch.org/pr/pytorch/rl/3325

Note: Links to docs will display an error until the docs builds have been completed.

❌ 3 New Failures, 4 Unrelated Failures

As of commit d04f8bf with merge base 9d34dbe (image):

NEW FAILURES - The following jobs have failed:

FLAKY - The following jobs failed but were likely due to flakiness present on trunk:

BROKEN TRUNK - The following jobs failed but were present on the merge base:

👉 Rebase onto the `viable/strict` branch to avoid these failures

This comment was automatically generated by Dr. CI and updates every 15 minutes.

@meta-cla meta-cla Bot added the CLA Signed This label is managed by the Facebook bot. Authors need to sign the CLA before a PR can be reviewed. label Jan 12, 2026
This was referenced Jan 12, 2026
Merged
Merged
Merged
Merged
Merged
Merged
Merged
Merged
Merged
Merged
@github-actions

github-actions Bot commented Jan 12, 2026 •
edited
Loading

Copy link
Copy Markdown
Contributor

$\color{#D29922}\textsf{\Large⚠\kern{0.2cm}\normalsize Warning}$ Result of CPU Benchmark Tests

Total Benchmarks: 164. Improved: $\large\color{#35bf28}47$. Worsened: $\large\color{#d91a1a}6$.

Expand to view detailed results
Name Max Mean Ops Ops on Repo HEAD Change
test_tensor_to_bytestream_speed[pickle] 80.2285μs 79.3672μs 12.5997 KOps/s 12.4588 KOps/s $\color{#35bf28}+1.13\%$
test_tensor_to_bytestream_speed[torch.save] 0.1372ms 0.1364ms 7.3312 KOps/s 7.2518 KOps/s $\color{#35bf28}+1.09\%$
test_tensor_to_bytestream_speed[untyped_storage] 0.1122s 0.1119s 8.9367 Ops/s 7.9253 Ops/s $\textbf{\color{#35bf28}+12.76\%}$
test_tensor_to_bytestream_speed[numpy] 2.6168μs 2.6104μs 383.0768 KOps/s 375.7650 KOps/s $\color{#35bf28}+1.95\%$
test_tensor_to_bytestream_speed[safetensors] 37.0362μs 36.8897μs 27.1079 KOps/s 27.1059 KOps/s $+0.01\%$
test_simple 0.5334s 0.5326s 1.8778 Ops/s 1.7539 Ops/s $\textbf{\color{#35bf28}+7.06\%}$
test_transformed 1.0915s 1.0901s 0.9173 Ops/s 0.8814 Ops/s $\color{#35bf28}+4.07\%$
test_serial 1.6308s 1.6279s 0.6143 Ops/s 0.5893 Ops/s $\color{#35bf28}+4.24\%$
test_parallel 1.1968s 1.1003s 0.9088 Ops/s 0.8406 Ops/s $\textbf{\color{#35bf28}+8.12\%}$
test_step_mdp_speed[True-True-True-True-True] 0.3093ms 43.7715μs 22.8459 KOps/s 22.4976 KOps/s $\color{#35bf28}+1.55\%$
test_step_mdp_speed[True-True-True-True-False] 52.9720μs 24.6148μs 40.6259 KOps/s 39.2360 KOps/s $\color{#35bf28}+3.54\%$
test_step_mdp_speed[True-True-True-False-True] 55.0010μs 24.1511μs 41.4060 KOps/s 40.2069 KOps/s $\color{#35bf28}+2.98\%$
test_step_mdp_speed[True-True-True-False-False] 41.0110μs 13.4500μs 74.3496 KOps/s 71.4788 KOps/s $\color{#35bf28}+4.02\%$
test_step_mdp_speed[True-True-False-True-True] 76.7010μs 46.8717μs 21.3348 KOps/s 20.5172 KOps/s $\color{#35bf28}+3.98\%$
test_step_mdp_speed[True-True-False-True-False] 53.9710μs 27.2025μs 36.7613 KOps/s 35.2676 KOps/s $\color{#35bf28}+4.24\%$
test_step_mdp_speed[True-True-False-False-True] 68.8620μs 27.3462μs 36.5681 KOps/s 34.8041 KOps/s $\textbf{\color{#35bf28}+5.07\%}$
test_step_mdp_speed[True-True-False-False-False] 48.9110μs 16.1593μs 61.8840 KOps/s 59.3284 KOps/s $\color{#35bf28}+4.31\%$
test_step_mdp_speed[True-False-True-True-True] 0.1033ms 49.9444μs 20.0223 KOps/s 19.5157 KOps/s $\color{#35bf28}+2.60\%$
test_step_mdp_speed[True-False-True-True-False] 62.4710μs 29.8787μs 33.4687 KOps/s 31.7528 KOps/s $\textbf{\color{#35bf28}+5.40\%}$
test_step_mdp_speed[True-False-True-False-True] 57.5710μs 26.8336μs 37.2667 KOps/s 35.3314 KOps/s $\textbf{\color{#35bf28}+5.48\%}$
test_step_mdp_speed[True-False-True-False-False] 43.7910μs 15.9477μs 62.7048 KOps/s 59.0114 KOps/s $\textbf{\color{#35bf28}+6.26\%}$
test_step_mdp_speed[True-False-False-True-True] 86.0210μs 51.9053μs 19.2659 KOps/s 18.3600 KOps/s $\color{#35bf28}+4.93\%$
test_step_mdp_speed[True-False-False-True-False] 68.5620μs 32.7492μs 30.5351 KOps/s 29.3839 KOps/s $\color{#35bf28}+3.92\%$
test_step_mdp_speed[True-False-False-False-True] 64.3120μs 29.2784μs 34.1548 KOps/s 31.7792 KOps/s $\textbf{\color{#35bf28}+7.48\%}$
test_step_mdp_speed[True-False-False-False-False] 55.7210μs 18.4443μs 54.2172 KOps/s 50.7841 KOps/s $\textbf{\color{#35bf28}+6.76\%}$
test_step_mdp_speed[False-True-True-True-True] 81.5010μs 48.2641μs 20.7193 KOps/s 19.4253 KOps/s $\textbf{\color{#35bf28}+6.66\%}$
test_step_mdp_speed[False-True-True-True-False] 64.0220μs 29.6785μs 33.6944 KOps/s 31.4047 KOps/s $\textbf{\color{#35bf28}+7.29\%}$
test_step_mdp_speed[False-True-True-False-True] 2.3464ms 31.0547μs 32.2012 KOps/s 30.7771 KOps/s $\color{#35bf28}+4.63\%$
test_step_mdp_speed[False-True-True-False-False] 49.8310μs 17.7946μs 56.1969 KOps/s 53.6865 KOps/s $\color{#35bf28}+4.68\%$
test_step_mdp_speed[False-True-False-True-True] 82.8720μs 51.5722μs 19.3903 KOps/s 18.5113 KOps/s $\color{#35bf28}+4.75\%$
test_step_mdp_speed[False-True-False-True-False] 65.4020μs 32.2145μs 31.0420 KOps/s 29.2851 KOps/s $\textbf{\color{#35bf28}+6.00\%}$
test_step_mdp_speed[False-True-False-False-True] 62.2710μs 33.5701μs 29.7885 KOps/s 28.6559 KOps/s $\color{#35bf28}+3.95\%$
test_step_mdp_speed[False-True-False-False-False] 64.6710μs 20.5895μs 48.5684 KOps/s 46.4745 KOps/s $\color{#35bf28}+4.51\%$
test_step_mdp_speed[False-False-True-True-True] 90.7310μs 54.7595μs 18.2617 KOps/s 17.2897 KOps/s $\textbf{\color{#35bf28}+5.62\%}$
test_step_mdp_speed[False-False-True-True-False] 67.2810μs 34.8800μs 28.6697 KOps/s 26.7554 KOps/s $\textbf{\color{#35bf28}+7.15\%}$
test_step_mdp_speed[False-False-True-False-True] 65.9610μs 33.7375μs 29.6406 KOps/s 27.9635 KOps/s $\textbf{\color{#35bf28}+6.00\%}$
test_step_mdp_speed[False-False-True-False-False] 49.2510μs 20.2444μs 49.3964 KOps/s 46.2920 KOps/s $\textbf{\color{#35bf28}+6.71\%}$
test_step_mdp_speed[False-False-False-True-True] 87.4220μs 56.4991μs 17.6994 KOps/s 16.5352 KOps/s $\textbf{\color{#35bf28}+7.04\%}$
test_step_mdp_speed[False-False-False-True-False] 77.8010μs 37.1843μs 26.8931 KOps/s 24.6338 KOps/s $\textbf{\color{#35bf28}+9.17\%}$
test_step_mdp_speed[False-False-False-False-True] 67.0320μs 35.1228μs 28.4716 KOps/s 26.2677 KOps/s $\textbf{\color{#35bf28}+8.39\%}$
test_step_mdp_speed[False-False-False-False-False] 45.9010μs 23.0207μs 43.4391 KOps/s 41.3533 KOps/s $\textbf{\color{#35bf28}+5.04\%}$
test_non_tensor_env_rollout_speed[1000-single-True] 0.8575s 0.7590s 1.3175 Ops/s 1.3037 Ops/s $\color{#35bf28}+1.06\%$
test_non_tensor_env_rollout_speed[1000-single-False] 0.7304s 0.6252s 1.5994 Ops/s 1.5856 Ops/s $\color{#35bf28}+0.87\%$
test_non_tensor_env_rollout_speed[1000-serial-no-buffers-True] 1.7375s 1.6464s 0.6074 Ops/s 0.5987 Ops/s $\color{#35bf28}+1.45\%$
test_non_tensor_env_rollout_speed[1000-serial-no-buffers-False] 1.4965s 1.4164s 0.7060 Ops/s 0.6888 Ops/s $\color{#35bf28}+2.51\%$
test_non_tensor_env_rollout_speed[1000-serial-buffers-True] 1.9544s 1.8706s 0.5346 Ops/s 0.5210 Ops/s $\color{#35bf28}+2.61\%$
test_non_tensor_env_rollout_speed[1000-serial-buffers-False] 1.7398s 1.6561s 0.6038 Ops/s 0.5866 Ops/s $\color{#35bf28}+2.94\%$
test_non_tensor_env_rollout_speed[1000-parallel-no-buffers-True] 4.7485s 4.6086s 0.2170 Ops/s 0.2144 Ops/s $\color{#35bf28}+1.21\%$
test_non_tensor_env_rollout_speed[1000-parallel-no-buffers-False] 4.5900s 4.4772s 0.2234 Ops/s 0.2228 Ops/s $\color{#35bf28}+0.24\%$
test_non_tensor_env_rollout_speed[1000-parallel-buffers-True] 1.9877s 1.9162s 0.5219 Ops/s 0.5147 Ops/s $\color{#35bf28}+1.40\%$
test_non_tensor_env_rollout_speed[1000-parallel-buffers-False] 1.8932s 1.6907s 0.5915 Ops/s 0.5974 Ops/s $\color{#d91a1a}-1.00\%$
test_values[generalized_advantage_estimate-True-True] 11.1331ms 10.3792ms 96.3470 Ops/s 92.1373 Ops/s $\color{#35bf28}+4.57\%$
test_values[vec_generalized_advantage_estimate-True-True] 14.9827ms 11.1171ms 89.9515 Ops/s 56.8993 Ops/s $\textbf{\color{#35bf28}+58.09\%}$
test_values[td0_return_estimate-False-False] 0.2261ms 0.1341ms 7.4551 KOps/s 7.9424 KOps/s $\textbf{\color{#d91a1a}-6.14\%}$
test_values[td1_return_estimate-False-False] 29.9783ms 28.8091ms 34.7113 Ops/s 34.3520 Ops/s $\color{#35bf28}+1.05\%$
test_values[vec_td1_return_estimate-False-False] 11.9018ms 11.1139ms 89.9771 Ops/s 56.4510 Ops/s $\textbf{\color{#35bf28}+59.39\%}$
test_values[td_lambda_return_estimate-True-False] 41.6779ms 41.0809ms 24.3422 Ops/s 23.1496 Ops/s $\textbf{\color{#35bf28}+5.15\%}$
test_values[vec_td_lambda_return_estimate-True-False] 11.8654ms 11.0553ms 90.4541 Ops/s 56.9067 Ops/s $\textbf{\color{#35bf28}+58.95\%}$
test_gae_speed[generalized_advantage_estimate-False-1-512] 8.9975ms 8.9019ms 112.3357 Ops/s 103.0340 Ops/s $\textbf{\color{#35bf28}+9.03\%}$
test_gae_speed[vec_generalized_advantage_estimate-True-1-512] 1.9090ms 1.4992ms 667.0141 Ops/s 661.0507 Ops/s $\color{#35bf28}+0.90\%$
test_gae_speed[vec_generalized_advantage_estimate-False-1-512] 0.6579ms 0.4202ms 2.3798 KOps/s 2.3333 KOps/s $\color{#35bf28}+1.99\%$
test_gae_speed[vec_generalized_advantage_estimate-True-32-512] 29.8772ms 29.4069ms 34.0056 Ops/s 28.9703 Ops/s $\textbf{\color{#35bf28}+17.38\%}$
test_gae_speed[vec_generalized_advantage_estimate-False-32-512] 1.8606ms 1.7054ms 586.3661 Ops/s 585.1474 Ops/s $\color{#35bf28}+0.21\%$
test_dqn_speed[False-None] 2.1591ms 1.4105ms 708.9931 Ops/s 710.0001 Ops/s $\color{#d91a1a}-0.14\%$
test_dqn_speed[False-backward] 1.9943ms 1.9153ms 522.1093 Ops/s 510.7391 Ops/s $\color{#35bf28}+2.23\%$
test_dqn_speed[True-None] 0.6823ms 0.5365ms 1.8639 KOps/s 1.8199 KOps/s $\color{#35bf28}+2.42\%$
test_dqn_speed[True-backward] 1.0033ms 0.9706ms 1.0303 KOps/s 993.7569 Ops/s $\color{#35bf28}+3.67\%$
test_dqn_speed[reduce-overhead-None] 0.6892ms 0.5262ms 1.9005 KOps/s 1.8602 KOps/s $\color{#35bf28}+2.16\%$
test_dqn_speed[reduce-overhead-backward] 0.9977ms 0.9653ms 1.0359 KOps/s 933.1397 Ops/s $\textbf{\color{#35bf28}+11.01\%}$
test_ddpg_speed[False-None] 3.1250ms 2.8017ms 356.9263 Ops/s 343.5199 Ops/s $\color{#35bf28}+3.90\%$
test_ddpg_speed[False-backward] 4.1081ms 4.0088ms 249.4508 Ops/s 242.6783 Ops/s $\color{#35bf28}+2.79\%$
test_ddpg_speed[True-None] 1.4458ms 1.3750ms 727.2709 Ops/s 713.3816 Ops/s $\color{#35bf28}+1.95\%$
test_ddpg_speed[True-backward] 2.3930ms 2.3397ms 427.4078 Ops/s 381.0436 Ops/s $\textbf{\color{#35bf28}+12.17\%}$
test_ddpg_speed[reduce-overhead-None] 1.7558ms 1.3738ms 727.9238 Ops/s 719.1866 Ops/s $\color{#35bf28}+1.21\%$
test_ddpg_speed[reduce-overhead-backward] 2.4246ms 2.3487ms 425.7647 Ops/s 416.6549 Ops/s $\color{#35bf28}+2.19\%$
test_sac_speed[False-None] 8.5175ms 7.9550ms 125.7075 Ops/s 123.2150 Ops/s $\color{#35bf28}+2.02\%$
test_sac_speed[False-backward] 11.6739ms 11.1384ms 89.7796 Ops/s 87.2011 Ops/s $\color{#35bf28}+2.96\%$
test_sac_speed[True-None] 2.5399ms 2.1225ms 471.1523 Ops/s 462.4360 Ops/s $\color{#35bf28}+1.88\%$
test_sac_speed[True-backward] 4.0801ms 3.9712ms 251.8103 Ops/s 215.4269 Ops/s $\textbf{\color{#35bf28}+16.89\%}$
test_sac_speed[reduce-overhead-None] 2.4945ms 2.1104ms 473.8345 Ops/s 460.2514 Ops/s $\color{#35bf28}+2.95\%$
test_sac_speed[reduce-overhead-backward] 4.1933ms 4.0570ms 246.4848 Ops/s 208.1292 Ops/s $\textbf{\color{#35bf28}+18.43\%}$
test_redq_speed[False-None] 11.1840ms 10.3017ms 97.0715 Ops/s 66.7442 Ops/s $\textbf{\color{#35bf28}+45.44\%}$
test_redq_speed[False-backward] 18.7510ms 17.8915ms 55.8925 Ops/s 54.9479 Ops/s $\color{#35bf28}+1.72\%$
test_redq_speed[True-None] 4.8474ms 4.3661ms 229.0393 Ops/s 216.3184 Ops/s $\textbf{\color{#35bf28}+5.88\%}$
test_redq_speed[True-backward] 10.0611ms 9.7633ms 102.4246 Ops/s 103.1741 Ops/s $\color{#d91a1a}-0.73\%$
test_redq_speed[reduce-overhead-None] 4.6258ms 4.3442ms 230.1897 Ops/s 219.2179 Ops/s $\textbf{\color{#35bf28}+5.00\%}$
test_redq_speed[reduce-overhead-backward] 10.2233ms 9.9604ms 100.3973 Ops/s 99.7626 Ops/s $\color{#35bf28}+0.64\%$
test_redq_deprec_speed[False-None] 11.5232ms 11.0823ms 90.2340 Ops/s 88.4937 Ops/s $\color{#35bf28}+1.97\%$
test_redq_deprec_speed[False-backward] 16.2542ms 15.9227ms 62.8033 Ops/s 62.3859 Ops/s $\color{#35bf28}+0.67\%$
test_redq_deprec_speed[True-None] 3.8711ms 3.6824ms 271.5609 Ops/s 269.4404 Ops/s $\color{#35bf28}+0.79\%$
test_redq_deprec_speed[True-backward] 7.8066ms 7.5772ms 131.9752 Ops/s 108.0694 Ops/s $\textbf{\color{#35bf28}+22.12\%}$
test_redq_deprec_speed[reduce-overhead-None] 3.7490ms 3.5689ms 280.1955 Ops/s 272.9766 Ops/s $\color{#35bf28}+2.64\%$
test_redq_deprec_speed[reduce-overhead-backward] 7.7780ms 7.5880ms 131.7864 Ops/s 121.2582 Ops/s $\textbf{\color{#35bf28}+8.68\%}$
test_td3_speed[False-None] 8.1671ms 7.8892ms 126.7557 Ops/s 123.0094 Ops/s $\color{#35bf28}+3.05\%$
test_td3_speed[False-backward] 11.1929ms 10.7383ms 93.1242 Ops/s 91.1568 Ops/s $\color{#35bf28}+2.16\%$
test_td3_speed[True-None] 1.8397ms 1.7953ms 557.0091 Ops/s 534.4066 Ops/s $\color{#35bf28}+4.23\%$
test_td3_speed[True-backward] 3.9540ms 3.6206ms 276.1986 Ops/s 272.9305 Ops/s $\color{#35bf28}+1.20\%$
test_td3_speed[reduce-overhead-None] 1.8457ms 1.7915ms 558.1823 Ops/s 546.2414 Ops/s $\color{#35bf28}+2.19\%$
test_td3_speed[reduce-overhead-backward] 3.8080ms 3.6403ms 274.7047 Ops/s 245.2612 Ops/s $\textbf{\color{#35bf28}+12.00\%}$
test_cql_speed[False-None] 29.4179ms 25.9320ms 38.5623 Ops/s 37.9396 Ops/s $\color{#35bf28}+1.64\%$
test_cql_speed[False-backward] 37.2629ms 34.9598ms 28.6042 Ops/s 27.8004 Ops/s $\color{#35bf28}+2.89\%$
test_cql_speed[True-None] 12.8735ms 12.3315ms 81.0933 Ops/s 81.4568 Ops/s $\color{#d91a1a}-0.45\%$
test_cql_speed[True-backward] 19.0401ms 18.4216ms 54.2842 Ops/s 53.7302 Ops/s $\color{#35bf28}+1.03\%$
test_cql_speed[reduce-overhead-None] 13.5435ms 12.4769ms 80.1483 Ops/s 79.5887 Ops/s $\color{#35bf28}+0.70\%$
test_cql_speed[reduce-overhead-backward] 18.9907ms 18.4362ms 54.2410 Ops/s 53.4073 Ops/s $\color{#35bf28}+1.56\%$
test_a2c_speed[False-None] 5.7450ms 5.3454ms 187.0766 Ops/s 183.9965 Ops/s $\color{#35bf28}+1.67\%$
test_a2c_speed[False-backward] 12.0830ms 11.8434ms 84.4349 Ops/s 85.4014 Ops/s $\color{#d91a1a}-1.13\%$
test_a2c_speed[True-None] 4.0793ms 3.7140ms 269.2531 Ops/s 267.4639 Ops/s $\color{#35bf28}+0.67\%$
test_a2c_speed[True-backward] 8.8536ms 8.5778ms 116.5804 Ops/s 110.8769 Ops/s $\textbf{\color{#35bf28}+5.14\%}$
test_a2c_speed[reduce-overhead-None] 4.0472ms 3.6805ms 271.7056 Ops/s 270.5011 Ops/s $\color{#35bf28}+0.45\%$
test_a2c_speed[reduce-overhead-backward] 8.9661ms 8.6672ms 115.3772 Ops/s 114.4858 Ops/s $\color{#35bf28}+0.78\%$
test_ppo_speed[False-None] 6.0453ms 5.8500ms 170.9395 Ops/s 166.4896 Ops/s $\color{#35bf28}+2.67\%$
test_ppo_speed[False-backward] 12.7175ms 12.5101ms 79.9355 Ops/s 80.4438 Ops/s $\color{#d91a1a}-0.63\%$
test_ppo_speed[True-None] 3.9247ms 3.5688ms 280.2024 Ops/s 272.9088 Ops/s $\color{#35bf28}+2.67\%$
test_ppo_speed[True-backward] 8.9082ms 8.4161ms 118.8206 Ops/s 117.4163 Ops/s $\color{#35bf28}+1.20\%$
test_ppo_speed[reduce-overhead-None] 3.9276ms 3.5586ms 281.0098 Ops/s 277.1649 Ops/s $\color{#35bf28}+1.39\%$
test_ppo_speed[reduce-overhead-backward] 8.9468ms 8.6599ms 115.4743 Ops/s 112.7907 Ops/s $\color{#35bf28}+2.38\%$
test_reinforce_speed[False-None] 4.8783ms 4.5009ms 222.1783 Ops/s 218.0700 Ops/s $\color{#35bf28}+1.88\%$
test_reinforce_speed[False-backward] 7.6072ms 7.3338ms 136.3547 Ops/s 135.1429 Ops/s $\color{#35bf28}+0.90\%$
test_reinforce_speed[True-None] 3.1142ms 2.8549ms 350.2784 Ops/s 337.9409 Ops/s $\color{#35bf28}+3.65\%$
test_reinforce_speed[True-backward] 8.0182ms 7.6611ms 130.5299 Ops/s 119.6723 Ops/s $\textbf{\color{#35bf28}+9.07\%}$
test_reinforce_speed[reduce-overhead-None] 3.0047ms 2.8353ms 352.6993 Ops/s 338.6486 Ops/s $\color{#35bf28}+4.15\%$
test_reinforce_speed[reduce-overhead-backward] 8.2316ms 7.8803ms 126.8988 Ops/s 125.0844 Ops/s $\color{#35bf28}+1.45\%$
test_iql_speed[False-None] 24.9890ms 20.2245ms 49.4451 Ops/s 49.8117 Ops/s $\color{#d91a1a}-0.74\%$
test_iql_speed[False-backward] 35.1998ms 30.5426ms 32.7412 Ops/s 32.8120 Ops/s $\color{#d91a1a}-0.22\%$
test_iql_speed[True-None] 8.6964ms 8.4467ms 118.3889 Ops/s 111.6641 Ops/s $\textbf{\color{#35bf28}+6.02\%}$
test_iql_speed[True-backward] 17.0303ms 16.6522ms 60.0522 Ops/s 57.9433 Ops/s $\color{#35bf28}+3.64\%$
test_iql_speed[reduce-overhead-None] 8.8927ms 8.5105ms 117.5024 Ops/s 113.4056 Ops/s $\color{#35bf28}+3.61\%$
test_iql_speed[reduce-overhead-backward] 17.4543ms 17.0846ms 58.5323 Ops/s 58.9065 Ops/s $\color{#d91a1a}-0.64\%$
test_rb_sample[TensorDictReplayBuffer-ListStorage-RandomSampler-4000] 7.3001ms 5.8856ms 169.9053 Ops/s 167.6380 Ops/s $\color{#35bf28}+1.35\%$
test_rb_sample[TensorDictReplayBuffer-LazyMemmapStorage-RandomSampler-10000] 0.5151ms 0.2819ms 3.5475 KOps/s 3.4995 KOps/s $\color{#35bf28}+1.37\%$
test_rb_sample[TensorDictReplayBuffer-LazyTensorStorage-RandomSampler-10000] 0.4789ms 0.2586ms 3.8666 KOps/s 3.8042 KOps/s $\color{#35bf28}+1.64\%$
test_rb_sample[TensorDictReplayBuffer-ListStorage-SamplerWithoutReplacement-4000] 5.8691ms 5.6463ms 177.1063 Ops/s 174.8864 Ops/s $\color{#35bf28}+1.27\%$
test_rb_sample[TensorDictReplayBuffer-LazyMemmapStorage-SamplerWithoutReplacement-10000] 1.8922ms 0.2760ms 3.6230 KOps/s 3.6097 KOps/s $\color{#35bf28}+0.37\%$
test_rb_sample[TensorDictReplayBuffer-LazyTensorStorage-SamplerWithoutReplacement-10000] 0.6637ms 0.2620ms 3.8168 KOps/s 3.8873 KOps/s $\color{#d91a1a}-1.81\%$
test_rb_sample[TensorDictReplayBuffer-LazyMemmapStorage-sampler6-10000] 1.5156ms 1.2884ms 776.1735 Ops/s 781.3695 Ops/s $\color{#d91a1a}-0.66\%$
test_rb_sample[TensorDictReplayBuffer-LazyTensorStorage-sampler7-10000] 1.5468ms 1.1792ms 848.0451 Ops/s 832.2204 Ops/s $\color{#35bf28}+1.90\%$
test_rb_sample[TensorDictPrioritizedReplayBuffer-ListStorage-None-4000] 9.9393ms 5.9920ms 166.8881 Ops/s 169.5251 Ops/s $\color{#d91a1a}-1.56\%$
test_rb_sample[TensorDictPrioritizedReplayBuffer-LazyMemmapStorage-None-10000] 1.9373ms 0.5226ms 1.9136 KOps/s 2.1725 KOps/s $\textbf{\color{#d91a1a}-11.92\%}$
test_rb_sample[TensorDictPrioritizedReplayBuffer-LazyTensorStorage-None-10000] 0.6547ms 0.4773ms 2.0950 KOps/s 2.3641 KOps/s $\textbf{\color{#d91a1a}-11.38\%}$
test_rb_iterate[TensorDictReplayBuffer-ListStorage-RandomSampler-4000] 5.7876ms 5.6699ms 176.3707 Ops/s 172.5319 Ops/s $\color{#35bf28}+2.22\%$
test_rb_iterate[TensorDictReplayBuffer-LazyMemmapStorage-RandomSampler-10000] 2.0024ms 0.3685ms 2.7138 KOps/s 2.7592 KOps/s $\color{#d91a1a}-1.65\%$
test_rb_iterate[TensorDictReplayBuffer-LazyTensorStorage-RandomSampler-10000] 0.5970ms 0.3538ms 2.8268 KOps/s 3.1155 KOps/s $\textbf{\color{#d91a1a}-9.26\%}$
test_rb_iterate[TensorDictReplayBuffer-ListStorage-SamplerWithoutReplacement-4000] 5.7943ms 5.5833ms 179.1066 Ops/s 174.9349 Ops/s $\color{#35bf28}+2.38\%$
test_rb_iterate[TensorDictReplayBuffer-LazyMemmapStorage-SamplerWithoutReplacement-10000] 1.9619ms 0.3827ms 2.6130 KOps/s 2.9215 KOps/s $\textbf{\color{#d91a1a}-10.56\%}$
test_rb_iterate[TensorDictReplayBuffer-LazyTensorStorage-SamplerWithoutReplacement-10000] 0.4575ms 0.2613ms 3.8275 KOps/s 3.3322 KOps/s $\textbf{\color{#35bf28}+14.86\%}$
test_rb_iterate[TensorDictPrioritizedReplayBuffer-ListStorage-None-4000] 6.3771ms 5.8103ms 172.1084 Ops/s 169.9381 Ops/s $\color{#35bf28}+1.28\%$
test_rb_iterate[TensorDictPrioritizedReplayBuffer-LazyMemmapStorage-None-10000] 1.1842ms 0.4414ms 2.2657 KOps/s 1.9790 KOps/s $\textbf{\color{#35bf28}+14.49\%}$
test_rb_iterate[TensorDictPrioritizedReplayBuffer-LazyTensorStorage-None-10000] 0.6220ms 0.4171ms 2.3977 KOps/s 2.3449 KOps/s $\color{#35bf28}+2.25\%$
test_rb_populate[TensorDictReplayBuffer-ListStorage-RandomSampler-400] 0.6408s 17.6412ms 56.6855 Ops/s 48.9643 Ops/s $\textbf{\color{#35bf28}+15.77\%}$
test_rb_populate[TensorDictReplayBuffer-LazyMemmapStorage-RandomSampler-400] 11.4723ms 1.7981ms 556.1397 Ops/s 530.5802 Ops/s $\color{#35bf28}+4.82\%$
test_rb_populate[TensorDictReplayBuffer-LazyTensorStorage-RandomSampler-400] 1.9875ms 0.9116ms 1.0970 KOps/s 761.8989 Ops/s $\textbf{\color{#35bf28}+43.98\%}$
test_rb_populate[TensorDictReplayBuffer-ListStorage-SamplerWithoutReplacement-400] 6.4078ms 4.9180ms 203.3336 Ops/s 191.0757 Ops/s $\textbf{\color{#35bf28}+6.42\%}$
test_rb_populate[TensorDictReplayBuffer-LazyMemmapStorage-SamplerWithoutReplacement-400] 12.8859ms 1.9843ms 503.9594 Ops/s 518.8312 Ops/s $\color{#d91a1a}-2.87\%$
test_rb_populate[TensorDictReplayBuffer-LazyTensorStorage-SamplerWithoutReplacement-400] 6.1214ms 1.1774ms 849.3430 Ops/s 805.1568 Ops/s $\textbf{\color{#35bf28}+5.49\%}$
test_rb_populate[TensorDictPrioritizedReplayBuffer-ListStorage-None-400] 0.5944s 16.9345ms 59.0510 Ops/s 185.0975 Ops/s $\textbf{\color{#d91a1a}-68.10\%}$
test_rb_populate[TensorDictPrioritizedReplayBuffer-LazyMemmapStorage-None-400] 4.0135ms 1.8656ms 536.0079 Ops/s 502.8131 Ops/s $\textbf{\color{#35bf28}+6.60\%}$
test_rb_populate[TensorDictPrioritizedReplayBuffer-LazyTensorStorage-None-400] 2.1714ms 1.2413ms 805.5867 Ops/s 670.3944 Ops/s $\textbf{\color{#35bf28}+20.17\%}$
test_rb_extend_sample[ReplayBuffer-LazyTensorStorage-RandomSampler-10000-10000-100-True] 38.2229ms 35.4507ms 28.2082 Ops/s 27.7521 Ops/s $\color{#35bf28}+1.64\%$
test_rb_extend_sample[ReplayBuffer-LazyTensorStorage-RandomSampler-10000-10000-100-False] 19.5808ms 17.5458ms 56.9937 Ops/s 56.5260 Ops/s $\color{#35bf28}+0.83\%$
test_rb_extend_sample[ReplayBuffer-LazyTensorStorage-RandomSampler-100000-10000-100-True] 39.8276ms 36.8463ms 27.1398 Ops/s 26.8378 Ops/s $\color{#35bf28}+1.13\%$
test_rb_extend_sample[ReplayBuffer-LazyTensorStorage-RandomSampler-100000-10000-100-False] 19.8382ms 18.0421ms 55.4259 Ops/s 54.7076 Ops/s $\color{#35bf28}+1.31\%$
test_rb_extend_sample[ReplayBuffer-LazyTensorStorage-RandomSampler-1000000-10000-100-True] 40.4727ms 37.9650ms 26.3400 Ops/s 25.7477 Ops/s $\color{#35bf28}+2.30\%$
test_rb_extend_sample[ReplayBuffer-LazyTensorStorage-RandomSampler-1000000-10000-100-False] 20.9982ms 19.3678ms 51.6321 Ops/s 50.9415 Ops/s $\color{#35bf28}+1.36\%$

vmoens added 3 commits January 12, 2026 17:37
@vmoens
Update
2349e22 
[ghstack-poisoned]
@vmoens
Update
0c2bb49 
[ghstack-poisoned]
@vmoens
Update
d04f8bf 
[ghstack-poisoned]
vmoens added a commit that referenced this pull request Jan 13, 2026
@vmoens
[BugFix] Collector robustness & Async fixes
cadf23b 
Add worker health checks and timeout logging to `MultiAsyncCollector`.
Add `_recv_and_check` to `MultiCollector` for safer pipe communication.
Improved `pause()` and `start()` in `MultiCollector`.
Fix `init_random_frames` support for `.start()` mode by using
`replay_buffer.write_count` when available.


ghstack-source-id: acc531b
Pull-Request: #3325
@vmoens vmoens merged commit d04f8bf into gh/vmoens/193/base Jan 13, 2026
99 of 106 checks passed
@vmoens vmoens deleted the gh/vmoens/193/head branch January 13, 2026 18:50
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Reviewers

No reviews

Assignees

No one assigned

Labels

CLA Signed This label is managed by the Facebook bot. Authors need to sign the CLA before a PR can be reviewed.

Projects

None yet

Milestone

No milestone

Development

Successfully merging this pull request may close these issues.

1 participant

@vmoens