Skip to content

[Feature] Weight loss outputs when using prioritized sampler #3235

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom
Merged
vmoens merged 3 commits into from
Nov 15, 2025
Merged

[Feature] Weight loss outputs when using prioritized sampler #3235

vmoens merged 3 commits into from
Nov 15, 2025

Conversation

@vmoens
Copy link
Collaborator

@vmoens vmoens commented Nov 15, 2025 •
edited
Loading

@pytorch-bot
Copy link

pytorch-bot bot commented Nov 15, 2025 •
edited
Loading

🔗 Helpful Links

🧪 See artifacts and rendered test results at hud.pytorch.org/pr/pytorch/rl/3235

Note: Links to docs will display an error until the docs builds have been completed.

This comment was automatically generated by Dr. CI and updates every 15 minutes.

@meta-cla meta-cla bot added the CLA Signed This label is managed by the Facebook bot. Authors need to sign the CLA before a PR can be reviewed. label Nov 15, 2025
@vmoens vmoens added bug Something isn't working enhancement New feature or request labels Nov 15, 2025
@github-actions
Copy link

github-actions bot commented Nov 15, 2025 •
edited
Loading

$\color{#D29922}\textsf{\Large⚠\kern{0.2cm}\normalsize Warning}$ Result of CPU Benchmark Tests

Total Benchmarks: 154. Improved: $\large\color{#35bf28}16$. Worsened: $\large\color{#d91a1a}12$.

Expand to view detailed results
Name Max Mean Ops Ops on Repo HEAD Change
test_tensor_to_bytestream_speed[pickle] 88.8883μs 87.3322μs 11.4505 KOps/s 12.1212 KOps/s $\textbf{\color{#d91a1a}-5.53\%}$
test_tensor_to_bytestream_speed[torch.save] 0.1417ms 0.1407ms 7.1056 KOps/s 7.1186 KOps/s $\color{#d91a1a}-0.18\%$
test_tensor_to_bytestream_speed[untyped_storage] 0.1250s 0.1243s 8.0422 Ops/s 8.5374 Ops/s $\textbf{\color{#d91a1a}-5.80\%}$
test_tensor_to_bytestream_speed[numpy] 2.9356μs 2.9265μs 341.6996 KOps/s 334.4705 KOps/s $\color{#35bf28}+2.16\%$
test_tensor_to_bytestream_speed[safetensors] 44.6507μs 44.3047μs 22.5710 KOps/s 23.9885 KOps/s $\textbf{\color{#d91a1a}-5.91\%}$
test_simple 0.6625s 0.5730s 1.7452 Ops/s 1.7345 Ops/s $\color{#35bf28}+0.62\%$
test_transformed 1.2309s 1.1385s 0.8783 Ops/s 0.8689 Ops/s $\color{#35bf28}+1.08\%$
test_serial 1.6779s 1.6700s 0.5988 Ops/s 0.5924 Ops/s $\color{#35bf28}+1.07\%$
test_parallel 1.1743s 1.0923s 0.9155 Ops/s 0.9076 Ops/s $\color{#35bf28}+0.87\%$
test_step_mdp_speed[True-True-True-True-True] 0.4433ms 44.4520μs 22.4962 KOps/s 21.8041 KOps/s $\color{#35bf28}+3.17\%$
test_step_mdp_speed[True-True-True-True-False] 0.4249ms 25.7287μs 38.8671 KOps/s 39.5965 KOps/s $\color{#d91a1a}-1.84\%$
test_step_mdp_speed[True-True-True-False-True] 56.5630μs 25.5618μs 39.1209 KOps/s 39.4808 KOps/s $\color{#d91a1a}-0.91\%$
test_step_mdp_speed[True-True-True-False-False] 0.4117ms 14.0764μs 71.0409 KOps/s 71.4175 KOps/s $\color{#d91a1a}-0.53\%$
test_step_mdp_speed[True-True-False-True-True] 0.4627ms 48.6309μs 20.5630 KOps/s 20.5646 KOps/s $-0.01\%$
test_step_mdp_speed[True-True-False-True-False] 0.4345ms 28.2452μs 35.4042 KOps/s 35.5116 KOps/s $\color{#d91a1a}-0.30\%$
test_step_mdp_speed[True-True-False-False-True] 61.4240μs 28.1416μs 35.5345 KOps/s 34.6481 KOps/s $\color{#35bf28}+2.56\%$
test_step_mdp_speed[True-True-False-False-False] 0.4268ms 16.7840μs 59.5804 KOps/s 59.2251 KOps/s $\color{#35bf28}+0.60\%$
test_step_mdp_speed[True-False-True-True-True] 0.4656ms 50.8232μs 19.6761 KOps/s 19.0945 KOps/s $\color{#35bf28}+3.05\%$
test_step_mdp_speed[True-False-True-True-False] 65.1530μs 31.3855μs 31.8619 KOps/s 32.2578 KOps/s $\color{#d91a1a}-1.23\%$
test_step_mdp_speed[True-False-True-False-True] 0.4354ms 28.4547μs 35.1436 KOps/s 35.3768 KOps/s $\color{#d91a1a}-0.66\%$
test_step_mdp_speed[True-False-True-False-False] 0.4297ms 16.8973μs 59.1812 KOps/s 59.9225 KOps/s $\color{#d91a1a}-1.24\%$
test_step_mdp_speed[True-False-False-True-True] 0.4586ms 53.3870μs 18.7311 KOps/s 18.6107 KOps/s $\color{#35bf28}+0.65\%$
test_step_mdp_speed[True-False-False-True-False] 68.1140μs 33.7536μs 29.6264 KOps/s 29.6725 KOps/s $\color{#d91a1a}-0.16\%$
test_step_mdp_speed[True-False-False-False-True] 0.4385ms 30.9391μs 32.3216 KOps/s 32.4170 KOps/s $\color{#d91a1a}-0.29\%$
test_step_mdp_speed[True-False-False-False-False] 0.4234ms 19.3674μs 51.6332 KOps/s 51.4030 KOps/s $\color{#35bf28}+0.45\%$
test_step_mdp_speed[False-True-True-True-True] 0.4535ms 51.1661μs 19.5442 KOps/s 19.4907 KOps/s $\color{#35bf28}+0.27\%$
test_step_mdp_speed[False-True-True-True-False] 70.0940μs 31.0125μs 32.2450 KOps/s 32.3054 KOps/s $\color{#d91a1a}-0.19\%$
test_step_mdp_speed[False-True-True-False-True] 2.5055ms 33.1742μs 30.1439 KOps/s 30.6134 KOps/s $\color{#d91a1a}-1.53\%$
test_step_mdp_speed[False-True-True-False-False] 0.4379ms 18.6975μs 53.4831 KOps/s 54.0413 KOps/s $\color{#d91a1a}-1.03\%$
test_step_mdp_speed[False-True-False-True-True] 0.4644ms 54.2064μs 18.4480 KOps/s 18.3516 KOps/s $\color{#35bf28}+0.53\%$
test_step_mdp_speed[False-True-False-True-False] 0.4328ms 33.7253μs 29.6513 KOps/s 29.6196 KOps/s $\color{#35bf28}+0.11\%$
test_step_mdp_speed[False-True-False-False-True] 72.8640μs 34.4906μs 28.9934 KOps/s 28.9302 KOps/s $\color{#35bf28}+0.22\%$
test_step_mdp_speed[False-True-False-False-False] 0.4335ms 21.0089μs 47.5988 KOps/s 47.2532 KOps/s $\color{#35bf28}+0.73\%$
test_step_mdp_speed[False-False-True-True-True] 0.4644ms 56.0035μs 17.8560 KOps/s 17.7587 KOps/s $\color{#35bf28}+0.55\%$
test_step_mdp_speed[False-False-True-True-False] 0.4469ms 36.3895μs 27.4804 KOps/s 27.3273 KOps/s $\color{#35bf28}+0.56\%$
test_step_mdp_speed[False-False-True-False-True] 69.4740μs 34.7074μs 28.8123 KOps/s 28.7197 KOps/s $\color{#35bf28}+0.32\%$
test_step_mdp_speed[False-False-True-False-False] 0.4301ms 21.4039μs 46.7205 KOps/s 46.8146 KOps/s $\color{#d91a1a}-0.20\%$
test_step_mdp_speed[False-False-False-True-True] 0.4691ms 58.8097μs 17.0040 KOps/s 16.8284 KOps/s $\color{#35bf28}+1.04\%$
test_step_mdp_speed[False-False-False-True-False] 0.4437ms 38.7753μs 25.7896 KOps/s 25.6196 KOps/s $\color{#35bf28}+0.66\%$
test_step_mdp_speed[False-False-False-False-True] 0.1081ms 36.8159μs 27.1622 KOps/s 27.2700 KOps/s $\color{#d91a1a}-0.40\%$
test_step_mdp_speed[False-False-False-False-False] 0.4282ms 24.0223μs 41.6279 KOps/s 42.3187 KOps/s $\color{#d91a1a}-1.63\%$
test_values[generalized_advantage_estimate-True-True] 9.8760ms 9.7016ms 103.0753 Ops/s 104.5969 Ops/s $\color{#d91a1a}-1.45\%$
test_values[vec_generalized_advantage_estimate-True-True] 13.6766ms 11.1648ms 89.5673 Ops/s 56.9487 Ops/s $\textbf{\color{#35bf28}+57.28\%}$
test_values[td0_return_estimate-False-False] 0.2231ms 0.1281ms 7.8059 KOps/s 7.8512 KOps/s $\color{#d91a1a}-0.58\%$
test_values[td1_return_estimate-False-False] 27.4004ms 27.0216ms 37.0074 Ops/s 37.1717 Ops/s $\color{#d91a1a}-0.44\%$
test_values[vec_td1_return_estimate-False-False] 12.4968ms 11.2292ms 89.0531 Ops/s 56.6763 Ops/s $\textbf{\color{#35bf28}+57.13\%}$
test_values[td_lambda_return_estimate-True-False] 41.8877ms 40.3581ms 24.7782 Ops/s 24.9881 Ops/s $\color{#d91a1a}-0.84\%$
test_values[vec_td_lambda_return_estimate-True-False] 12.4698ms 11.2434ms 88.9414 Ops/s 56.5643 Ops/s $\textbf{\color{#35bf28}+57.24\%}$
test_gae_speed[generalized_advantage_estimate-False-1-512] 8.9849ms 8.5717ms 116.6631 Ops/s 119.2525 Ops/s $\color{#d91a1a}-2.17\%$
test_gae_speed[vec_generalized_advantage_estimate-True-1-512] 1.8134ms 1.4992ms 667.0025 Ops/s 667.6510 Ops/s $\color{#d91a1a}-0.10\%$
test_gae_speed[vec_generalized_advantage_estimate-False-1-512] 0.5391ms 0.4141ms 2.4150 KOps/s 2.4383 KOps/s $\color{#d91a1a}-0.96\%$
test_gae_speed[vec_generalized_advantage_estimate-True-32-512] 28.9166ms 23.3208ms 42.8802 Ops/s 28.8925 Ops/s $\textbf{\color{#35bf28}+48.41\%}$
test_gae_speed[vec_generalized_advantage_estimate-False-32-512] 1.8417ms 1.7241ms 580.0117 Ops/s 583.9356 Ops/s $\color{#d91a1a}-0.67\%$
test_dqn_speed[False-None] 6.4021ms 1.4261ms 701.2173 Ops/s 706.1133 Ops/s $\color{#d91a1a}-0.69\%$
test_dqn_speed[False-backward] 2.0371ms 1.9534ms 511.9377 Ops/s 519.2727 Ops/s $\color{#d91a1a}-1.41\%$
test_dqn_speed[True-None] 0.8999ms 0.5110ms 1.9568 KOps/s 1.8722 KOps/s $\color{#35bf28}+4.52\%$
test_dqn_speed[True-backward] 1.0020ms 0.9708ms 1.0301 KOps/s 1.0104 KOps/s $\color{#35bf28}+1.95\%$
test_dqn_speed[reduce-overhead-None] 0.9109ms 0.5062ms 1.9757 KOps/s 1.9060 KOps/s $\color{#35bf28}+3.65\%$
test_dqn_speed[reduce-overhead-backward] 0.9902ms 0.9578ms 1.0441 KOps/s 867.2470 Ops/s $\textbf{\color{#35bf28}+20.39\%}$
test_ddpg_speed[False-None] 3.1487ms 2.8644ms 349.1169 Ops/s 326.3105 Ops/s $\textbf{\color{#35bf28}+6.99\%}$
test_ddpg_speed[False-backward] 4.2863ms 4.1316ms 242.0378 Ops/s 244.1740 Ops/s $\color{#d91a1a}-0.87\%$
test_ddpg_speed[True-None] 1.7852ms 1.3776ms 725.9032 Ops/s 721.1186 Ops/s $\color{#35bf28}+0.66\%$
test_ddpg_speed[True-backward] 2.5746ms 2.3795ms 420.2508 Ops/s 369.5964 Ops/s $\textbf{\color{#35bf28}+13.71\%}$
test_ddpg_speed[reduce-overhead-None] 1.7860ms 1.3730ms 728.3318 Ops/s 719.5347 Ops/s $\color{#35bf28}+1.22\%$
test_ddpg_speed[reduce-overhead-backward] 2.4017ms 2.3642ms 422.9791 Ops/s 425.1303 Ops/s $\color{#d91a1a}-0.51\%$
test_sac_speed[False-None] 8.5047ms 7.9985ms 125.0237 Ops/s 124.5365 Ops/s $\color{#35bf28}+0.39\%$
test_sac_speed[False-backward] 11.9497ms 11.5022ms 86.9397 Ops/s 87.2007 Ops/s $\color{#d91a1a}-0.30\%$
test_sac_speed[True-None] 2.4431ms 2.2854ms 437.5685 Ops/s 410.3406 Ops/s $\textbf{\color{#35bf28}+6.64\%}$
test_sac_speed[True-backward] 4.3666ms 4.2486ms 235.3720 Ops/s 236.2887 Ops/s $\color{#d91a1a}-0.39\%$
test_sac_speed[reduce-overhead-None] 2.4517ms 2.2854ms 437.5636 Ops/s 425.6708 Ops/s $\color{#35bf28}+2.79\%$
test_sac_speed[reduce-overhead-backward] 4.8295ms 4.3077ms 232.1408 Ops/s 222.9769 Ops/s $\color{#35bf28}+4.11\%$
test_redq_speed[False-None] 10.7521ms 10.3433ms 96.6808 Ops/s 97.5414 Ops/s $\color{#d91a1a}-0.88\%$
test_redq_speed[False-backward] 18.3076ms 17.8274ms 56.0935 Ops/s 55.4280 Ops/s $\color{#35bf28}+1.20\%$
test_redq_speed[True-None] 4.8462ms 4.3543ms 229.6556 Ops/s 229.5229 Ops/s $\color{#35bf28}+0.06\%$
test_redq_speed[True-backward] 10.1812ms 9.8974ms 101.0367 Ops/s 102.1075 Ops/s $\color{#d91a1a}-1.05\%$
test_redq_speed[reduce-overhead-None] 4.7271ms 4.3904ms 227.7716 Ops/s 220.7577 Ops/s $\color{#35bf28}+3.18\%$
test_redq_speed[reduce-overhead-backward] 10.4260ms 10.1346ms 98.6715 Ops/s 99.0819 Ops/s $\color{#d91a1a}-0.41\%$
test_redq_deprec_speed[False-None] 11.6431ms 11.1160ms 89.9603 Ops/s 90.4766 Ops/s $\color{#d91a1a}-0.57\%$
test_redq_deprec_speed[False-backward] 16.3172ms 15.9758ms 62.5945 Ops/s 62.5931 Ops/s $+0.00\%$
test_redq_deprec_speed[True-None] 4.7482ms 4.3267ms 231.1251 Ops/s 235.6695 Ops/s $\color{#d91a1a}-1.93\%$
test_redq_deprec_speed[True-backward] 8.6779ms 8.4269ms 118.6682 Ops/s 121.5128 Ops/s $\color{#d91a1a}-2.34\%$
test_redq_deprec_speed[reduce-overhead-None] 4.6445ms 4.2966ms 232.7418 Ops/s 224.3861 Ops/s $\color{#35bf28}+3.72\%$
test_redq_deprec_speed[reduce-overhead-backward] 8.6613ms 8.3796ms 119.3368 Ops/s 116.5487 Ops/s $\color{#35bf28}+2.39\%$
test_td3_speed[False-None] 7.9971ms 7.9498ms 125.7892 Ops/s 127.1817 Ops/s $\color{#d91a1a}-1.09\%$
test_td3_speed[False-backward] 11.4454ms 10.9174ms 91.5973 Ops/s 93.6736 Ops/s $\color{#d91a1a}-2.22\%$
test_td3_speed[True-None] 1.8497ms 1.7961ms 556.7603 Ops/s 554.2340 Ops/s $\color{#35bf28}+0.46\%$
test_td3_speed[True-backward] 3.7720ms 3.6552ms 273.5793 Ops/s 253.4298 Ops/s $\textbf{\color{#35bf28}+7.95\%}$
test_td3_speed[reduce-overhead-None] 1.8434ms 1.7806ms 561.6218 Ops/s 563.6677 Ops/s $\color{#d91a1a}-0.36\%$
test_td3_speed[reduce-overhead-backward] 3.7319ms 3.6560ms 273.5195 Ops/s 235.2135 Ops/s $\textbf{\color{#35bf28}+16.29\%}$
test_cql_speed[False-None] 29.2916ms 26.2396ms 38.1104 Ops/s 39.1194 Ops/s $\color{#d91a1a}-2.58\%$
test_cql_speed[False-backward] 41.8409ms 36.5615ms 27.3512 Ops/s 28.3336 Ops/s $\color{#d91a1a}-3.47\%$
test_cql_speed[True-None] 12.7439ms 12.3499ms 80.9724 Ops/s 77.0588 Ops/s $\textbf{\color{#35bf28}+5.08\%}$
test_cql_speed[True-backward] 18.9533ms 18.5246ms 53.9823 Ops/s 55.5648 Ops/s $\color{#d91a1a}-2.85\%$
test_cql_speed[reduce-overhead-None] 13.0071ms 12.4817ms 80.1175 Ops/s 78.6038 Ops/s $\color{#35bf28}+1.93\%$
test_cql_speed[reduce-overhead-backward] 18.8848ms 18.5032ms 54.0448 Ops/s 54.4177 Ops/s $\color{#d91a1a}-0.69\%$
test_a2c_speed[False-None] 5.7292ms 5.4745ms 182.6648 Ops/s 182.3256 Ops/s $\color{#35bf28}+0.19\%$
test_a2c_speed[False-backward] 12.2628ms 11.9887ms 83.4117 Ops/s 84.3464 Ops/s $\color{#d91a1a}-1.11\%$
test_a2c_speed[True-None] 4.0775ms 3.7418ms 267.2479 Ops/s 268.0144 Ops/s $\color{#d91a1a}-0.29\%$
test_a2c_speed[True-backward] 9.2114ms 8.5741ms 116.6306 Ops/s 115.4972 Ops/s $\color{#35bf28}+0.98\%$
test_a2c_speed[reduce-overhead-None] 4.0909ms 3.7082ms 269.6723 Ops/s 271.2359 Ops/s $\color{#d91a1a}-0.58\%$
test_a2c_speed[reduce-overhead-backward] 9.2399ms 8.8421ms 113.0958 Ops/s 112.4614 Ops/s $\color{#35bf28}+0.56\%$
test_ppo_speed[False-None] 6.2865ms 5.9690ms 167.5311 Ops/s 169.7607 Ops/s $\color{#d91a1a}-1.31\%$
test_ppo_speed[False-backward] 12.8982ms 12.5526ms 79.6648 Ops/s 79.8426 Ops/s $\color{#d91a1a}-0.22\%$
test_ppo_speed[True-None] 4.1112ms 3.6760ms 272.0315 Ops/s 268.6608 Ops/s $\color{#35bf28}+1.25\%$
test_ppo_speed[True-backward] 8.8456ms 8.5076ms 117.5423 Ops/s 108.5082 Ops/s $\textbf{\color{#35bf28}+8.33\%}$
test_ppo_speed[reduce-overhead-None] 3.8614ms 3.6067ms 277.2592 Ops/s 275.5639 Ops/s $\color{#35bf28}+0.62\%$
test_ppo_speed[reduce-overhead-backward] 8.9481ms 8.7251ms 114.6125 Ops/s 114.6639 Ops/s $\color{#d91a1a}-0.04\%$
test_reinforce_speed[False-None] 4.7480ms 4.5836ms 218.1710 Ops/s 219.9046 Ops/s $\color{#d91a1a}-0.79\%$
test_reinforce_speed[False-backward] 7.8403ms 7.4212ms 134.7496 Ops/s 135.3799 Ops/s $\color{#d91a1a}-0.47\%$
test_reinforce_speed[True-None] 3.2883ms 2.8736ms 347.9949 Ops/s 347.5654 Ops/s $\color{#35bf28}+0.12\%$
test_reinforce_speed[True-backward] 7.8879ms 7.6972ms 129.9168 Ops/s 130.3334 Ops/s $\color{#d91a1a}-0.32\%$
test_reinforce_speed[reduce-overhead-None] 3.2546ms 2.8442ms 351.5938 Ops/s 340.4700 Ops/s $\color{#35bf28}+3.27\%$
test_reinforce_speed[reduce-overhead-backward] 8.1850ms 7.9349ms 126.0257 Ops/s 125.6603 Ops/s $\color{#35bf28}+0.29\%$
test_iql_speed[False-None] 24.5745ms 20.3627ms 49.1094 Ops/s 49.6517 Ops/s $\color{#d91a1a}-1.09\%$
test_iql_speed[False-backward] 36.9910ms 31.4136ms 31.8333 Ops/s 32.2438 Ops/s $\color{#d91a1a}-1.27\%$
test_iql_speed[True-None] 8.9262ms 8.6012ms 116.2625 Ops/s 114.2343 Ops/s $\color{#35bf28}+1.78\%$
test_iql_speed[True-backward] 17.7028ms 17.2135ms 58.0939 Ops/s 57.7608 Ops/s $\color{#35bf28}+0.58\%$
test_iql_speed[reduce-overhead-None] 9.3640ms 8.7246ms 114.6184 Ops/s 115.7552 Ops/s $\color{#d91a1a}-0.98\%$
test_iql_speed[reduce-overhead-backward] 18.1997ms 17.7842ms 56.2297 Ops/s 57.0339 Ops/s $\color{#d91a1a}-1.41\%$
test_rb_sample[TensorDictReplayBuffer-ListStorage-RandomSampler-4000] 7.4999ms 6.0293ms 165.8577 Ops/s 165.7627 Ops/s $\color{#35bf28}+0.06\%$
test_rb_sample[TensorDictReplayBuffer-LazyMemmapStorage-RandomSampler-10000] 0.5711ms 0.3313ms 3.0183 KOps/s 3.5259 KOps/s $\textbf{\color{#d91a1a}-14.40\%}$
test_rb_sample[TensorDictReplayBuffer-LazyTensorStorage-RandomSampler-10000] 0.5688ms 0.3153ms 3.1719 KOps/s 3.1068 KOps/s $\color{#35bf28}+2.09\%$
test_rb_sample[TensorDictReplayBuffer-ListStorage-SamplerWithoutReplacement-4000] 6.0091ms 5.7378ms 174.2814 Ops/s 173.5633 Ops/s $\color{#35bf28}+0.41\%$
test_rb_sample[TensorDictReplayBuffer-LazyMemmapStorage-SamplerWithoutReplacement-10000] 0.6133ms 0.3388ms 2.9513 KOps/s 3.4852 KOps/s $\textbf{\color{#d91a1a}-15.32\%}$
test_rb_sample[TensorDictReplayBuffer-LazyTensorStorage-SamplerWithoutReplacement-10000] 0.5358ms 0.3243ms 3.0831 KOps/s 3.2080 KOps/s $\color{#d91a1a}-3.89\%$
test_rb_sample[TensorDictReplayBuffer-LazyMemmapStorage-sampler6-10000] 1.6049ms 1.3232ms 755.7167 Ops/s 790.7197 Ops/s $\color{#d91a1a}-4.43\%$
test_rb_sample[TensorDictReplayBuffer-LazyTensorStorage-sampler7-10000] 1.4501ms 1.2481ms 801.2321 Ops/s 832.3162 Ops/s $\color{#d91a1a}-3.73\%$
test_rb_sample[TensorDictPrioritizedReplayBuffer-ListStorage-None-4000] 6.0315ms 5.8570ms 170.7366 Ops/s 167.9490 Ops/s $\color{#35bf28}+1.66\%$
test_rb_sample[TensorDictPrioritizedReplayBuffer-LazyMemmapStorage-None-10000] 1.1242ms 0.4759ms 2.1012 KOps/s 2.2688 KOps/s $\textbf{\color{#d91a1a}-7.39\%}$
test_rb_sample[TensorDictPrioritizedReplayBuffer-LazyTensorStorage-None-10000] 0.6430ms 0.4578ms 2.1843 KOps/s 2.2984 KOps/s $\color{#d91a1a}-4.96\%$
test_rb_iterate[TensorDictReplayBuffer-ListStorage-RandomSampler-4000] 5.8631ms 5.7012ms 175.4010 Ops/s 170.6595 Ops/s $\color{#35bf28}+2.78\%$
test_rb_iterate[TensorDictReplayBuffer-LazyMemmapStorage-RandomSampler-10000] 2.0919ms 0.3460ms 2.8903 KOps/s 2.7563 KOps/s $\color{#35bf28}+4.86\%$
test_rb_iterate[TensorDictReplayBuffer-LazyTensorStorage-RandomSampler-10000] 0.5734ms 0.3311ms 3.0205 KOps/s 2.8577 KOps/s $\textbf{\color{#35bf28}+5.70\%}$
test_rb_iterate[TensorDictReplayBuffer-ListStorage-SamplerWithoutReplacement-4000] 5.9248ms 5.6755ms 176.1957 Ops/s 174.0272 Ops/s $\color{#35bf28}+1.25\%$
test_rb_iterate[TensorDictReplayBuffer-LazyMemmapStorage-SamplerWithoutReplacement-10000] 0.9064ms 0.3119ms 3.2060 KOps/s 3.5774 KOps/s $\textbf{\color{#d91a1a}-10.38\%}$
test_rb_iterate[TensorDictReplayBuffer-LazyTensorStorage-SamplerWithoutReplacement-10000] 0.5614ms 0.3276ms 3.0525 KOps/s 3.7902 KOps/s $\textbf{\color{#d91a1a}-19.46\%}$
test_rb_iterate[TensorDictPrioritizedReplayBuffer-ListStorage-None-4000] 9.7044ms 6.0283ms 165.8853 Ops/s 168.2339 Ops/s $\color{#d91a1a}-1.40\%$
test_rb_iterate[TensorDictPrioritizedReplayBuffer-LazyMemmapStorage-None-10000] 1.1333ms 0.4724ms 2.1171 KOps/s 2.2826 KOps/s $\textbf{\color{#d91a1a}-7.25\%}$
test_rb_iterate[TensorDictPrioritizedReplayBuffer-LazyTensorStorage-None-10000] 0.7281ms 0.4589ms 2.1792 KOps/s 2.3908 KOps/s $\textbf{\color{#d91a1a}-8.85\%}$
test_rb_populate[TensorDictReplayBuffer-ListStorage-RandomSampler-400] 0.4703s 14.3977ms 69.4555 Ops/s 191.7859 Ops/s $\textbf{\color{#d91a1a}-63.78\%}$
test_rb_populate[TensorDictReplayBuffer-LazyMemmapStorage-RandomSampler-400] 9.4950ms 2.0577ms 485.9699 Ops/s 436.8541 Ops/s $\textbf{\color{#35bf28}+11.24\%}$
test_rb_populate[TensorDictReplayBuffer-LazyTensorStorage-RandomSampler-400] 7.2939ms 1.1653ms 858.1434 Ops/s 847.3380 Ops/s $\color{#35bf28}+1.28\%$
test_rb_populate[TensorDictReplayBuffer-ListStorage-SamplerWithoutReplacement-400] 6.6781ms 5.0908ms 196.4334 Ops/s 59.0707 Ops/s $\textbf{\color{#35bf28}+232.54\%}$
test_rb_populate[TensorDictReplayBuffer-LazyMemmapStorage-SamplerWithoutReplacement-400] 9.4602ms 2.0539ms 486.8812 Ops/s 509.0606 Ops/s $\color{#d91a1a}-4.36\%$
test_rb_populate[TensorDictReplayBuffer-LazyTensorStorage-SamplerWithoutReplacement-400] 6.4211ms 1.1759ms 850.4251 Ops/s 844.3964 Ops/s $\color{#35bf28}+0.71\%$
test_rb_populate[TensorDictPrioritizedReplayBuffer-ListStorage-None-400] 0.4327s 13.8411ms 72.2487 Ops/s 184.5374 Ops/s $\textbf{\color{#d91a1a}-60.85\%}$
test_rb_populate[TensorDictPrioritizedReplayBuffer-LazyMemmapStorage-None-400] 11.7173ms 2.2236ms 449.7193 Ops/s 448.7909 Ops/s $\color{#35bf28}+0.21\%$
test_rb_populate[TensorDictPrioritizedReplayBuffer-LazyTensorStorage-None-400] 1.3206ms 1.0103ms 989.8449 Ops/s 741.0869 Ops/s $\textbf{\color{#35bf28}+33.57\%}$
test_rb_extend_sample[ReplayBuffer-LazyTensorStorage-RandomSampler-10000-10000-100-True] 35.6414ms 33.5820ms 29.7778 Ops/s 29.3669 Ops/s $\color{#35bf28}+1.40\%$
test_rb_extend_sample[ReplayBuffer-LazyTensorStorage-RandomSampler-10000-10000-100-False] 18.9738ms 17.5971ms 56.8275 Ops/s 57.7369 Ops/s $\color{#d91a1a}-1.58\%$
test_rb_extend_sample[ReplayBuffer-LazyTensorStorage-RandomSampler-100000-10000-100-True] 36.2787ms 34.3363ms 29.1237 Ops/s 28.6711 Ops/s $\color{#35bf28}+1.58\%$
test_rb_extend_sample[ReplayBuffer-LazyTensorStorage-RandomSampler-100000-10000-100-False] 19.1354ms 17.7931ms 56.2016 Ops/s 56.6494 Ops/s $\color{#d91a1a}-0.79\%$
test_rb_extend_sample[ReplayBuffer-LazyTensorStorage-RandomSampler-1000000-10000-100-True] 41.3324ms 36.2416ms 27.5926 Ops/s 27.4009 Ops/s $\color{#35bf28}+0.70\%$
test_rb_extend_sample[ReplayBuffer-LazyTensorStorage-RandomSampler-1000000-10000-100-False] 20.6917ms 19.2354ms 51.9876 Ops/s 52.4797 Ops/s $\color{#d91a1a}-0.94\%$

@vmoens vmoens merged commit 8570c25 into main Nov 15, 2025
60 of 81 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Reviewers

No reviews

Assignees

No one assigned

Labels

bug Something isn't working CLA Signed This label is managed by the Facebook bot. Authors need to sign the CLA before a PR can be reviewed. enhancement New feature or request

Projects

None yet

Milestone

No milestone

Development

Successfully merging this pull request may close these issues.

2 participants

@vmoens