Skip to content

[Refactor] Make env creator optional for Ray #3227

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom
Merged
merged 1 commit into from
Oct 30, 2025
Merged

[Refactor] Make env creator optional for Ray #3227

merged 1 commit into from
Oct 30, 2025

Conversation

@vmoens
Copy link
Collaborator

@vmoens vmoens commented Oct 29, 2025 •
edited
Loading

@vmoens
Update
9441d6a 
[ghstack-poisoned]
@pytorch-bot
Copy link

pytorch-bot bot commented Oct 29, 2025 •
edited
Loading

🔗 Helpful Links

🧪 See artifacts and rendered test results at hud.pytorch.org/pr/pytorch/rl/3227

Note: Links to docs will display an error until the docs builds have been completed.

❌ 8 New Failures, 6 Unrelated Failures

As of commit 9441d6a with merge base 13434eb (image):

NEW FAILURES - The following jobs have failed:

BROKEN TRUNK - The following jobs failed but were present on the merge base:

👉 Rebase onto the `viable/strict` branch to avoid these failures

This comment was automatically generated by Dr. CI and updates every 15 minutes.

@meta-cla meta-cla bot added the CLA Signed This label is managed by the Facebook bot. Authors need to sign the CLA before a PR can be reviewed. label Oct 29, 2025
@vmoens vmoens mentioned this pull request Oct 29, 2025
Merged
@github-actions
Copy link

github-actions bot commented Oct 29, 2025

$\color{#D29922}\textsf{\Large⚠\kern{0.2cm}\normalsize Warning}$ Result of CPU Benchmark Tests

Total Benchmarks: 154. Improved: $\large\color{#35bf28}17$. Worsened: $\large\color{#d91a1a}14$.

Expand to view detailed results
Name Max Mean Ops Ops on Repo HEAD Change
test_tensor_to_bytestream_speed[pickle] 83.4335μs 81.7365μs 12.2344 KOps/s 12.0648 KOps/s $\color{#35bf28}+1.41\%$
test_tensor_to_bytestream_speed[torch.save] 0.1434ms 0.1419ms 7.0464 KOps/s 7.0389 KOps/s $\color{#35bf28}+0.11\%$
test_tensor_to_bytestream_speed[untyped_storage] 0.1291s 0.1290s 7.7490 Ops/s 7.6857 Ops/s $\color{#35bf28}+0.82\%$
test_tensor_to_bytestream_speed[numpy] 3.1096μs 3.1014μs 322.4339 KOps/s 330.7436 KOps/s $\color{#d91a1a}-2.51\%$
test_tensor_to_bytestream_speed[safetensors] 43.5986μs 43.3903μs 23.0466 KOps/s 21.9958 KOps/s $\color{#35bf28}+4.78\%$
test_simple 0.5603s 0.5574s 1.7939 Ops/s 1.7227 Ops/s $\color{#35bf28}+4.13\%$
test_transformed 1.2386s 1.1464s 0.8723 Ops/s 0.8704 Ops/s $\color{#35bf28}+0.22\%$
test_serial 1.6870s 1.6793s 0.5955 Ops/s 0.5822 Ops/s $\color{#35bf28}+2.29\%$
test_parallel 1.1858s 1.1052s 0.9048 Ops/s 0.8894 Ops/s $\color{#35bf28}+1.73\%$
test_step_mdp_speed[True-True-True-True-True] 0.1387ms 45.3597μs 22.0460 KOps/s 22.6566 KOps/s $\color{#d91a1a}-2.70\%$
test_step_mdp_speed[True-True-True-True-False] 52.5510μs 25.1551μs 39.7534 KOps/s 40.1749 KOps/s $\color{#d91a1a}-1.05\%$
test_step_mdp_speed[True-True-True-False-True] 52.3510μs 25.1077μs 39.8284 KOps/s 39.9456 KOps/s $\color{#d91a1a}-0.29\%$
test_step_mdp_speed[True-True-True-False-False] 79.2410μs 13.9255μs 71.8105 KOps/s 72.1896 KOps/s $\color{#d91a1a}-0.53\%$
test_step_mdp_speed[True-True-False-True-True] 84.3910μs 47.5494μs 21.0307 KOps/s 20.9716 KOps/s $\color{#35bf28}+0.28\%$
test_step_mdp_speed[True-True-False-True-False] 55.6910μs 27.2165μs 36.7424 KOps/s 35.0061 KOps/s $\color{#35bf28}+4.96\%$
test_step_mdp_speed[True-True-False-False-True] 0.1032ms 28.2431μs 35.4068 KOps/s 36.2263 KOps/s $\color{#d91a1a}-2.26\%$
test_step_mdp_speed[True-True-False-False-False] 44.6900μs 16.6867μs 59.9281 KOps/s 59.6607 KOps/s $\color{#35bf28}+0.45\%$
test_step_mdp_speed[True-False-True-True-True] 83.7510μs 50.2057μs 19.9181 KOps/s 19.7306 KOps/s $\color{#35bf28}+0.95\%$
test_step_mdp_speed[True-False-True-True-False] 73.6610μs 30.2629μs 33.0438 KOps/s 32.1188 KOps/s $\color{#35bf28}+2.88\%$
test_step_mdp_speed[True-False-True-False-True] 62.2710μs 28.1519μs 35.5216 KOps/s 35.2811 KOps/s $\color{#35bf28}+0.68\%$
test_step_mdp_speed[True-False-True-False-False] 48.3810μs 16.5016μs 60.6003 KOps/s 59.6012 KOps/s $\color{#35bf28}+1.68\%$
test_step_mdp_speed[True-False-False-True-True] 86.0220μs 52.7683μs 18.9508 KOps/s 18.9280 KOps/s $\color{#35bf28}+0.12\%$
test_step_mdp_speed[True-False-False-True-False] 62.5510μs 32.8191μs 30.4700 KOps/s 29.9449 KOps/s $\color{#35bf28}+1.75\%$
test_step_mdp_speed[True-False-False-False-True] 65.6210μs 30.4358μs 32.8560 KOps/s 32.8314 KOps/s $\color{#35bf28}+0.07\%$
test_step_mdp_speed[True-False-False-False-False] 52.8810μs 18.8722μs 52.9880 KOps/s 52.3675 KOps/s $\color{#35bf28}+1.19\%$
test_step_mdp_speed[False-True-True-True-True] 88.7020μs 50.6504μs 19.7432 KOps/s 19.8458 KOps/s $\color{#d91a1a}-0.52\%$
test_step_mdp_speed[False-True-True-True-False] 61.2810μs 30.3700μs 32.9272 KOps/s 32.5726 KOps/s $\color{#35bf28}+1.09\%$
test_step_mdp_speed[False-True-True-False-True] 2.4563ms 32.0531μs 31.1982 KOps/s 31.0651 KOps/s $\color{#35bf28}+0.43\%$
test_step_mdp_speed[False-True-True-False-False] 55.4910μs 18.6072μs 53.7425 KOps/s 54.1833 KOps/s $\color{#d91a1a}-0.81\%$
test_step_mdp_speed[False-True-False-True-True] 87.7120μs 53.0283μs 18.8579 KOps/s 18.8927 KOps/s $\color{#d91a1a}-0.18\%$
test_step_mdp_speed[False-True-False-True-False] 62.6710μs 32.8607μs 30.4315 KOps/s 29.9407 KOps/s $\color{#35bf28}+1.64\%$
test_step_mdp_speed[False-True-False-False-True] 65.5820μs 33.9991μs 29.4126 KOps/s 28.9905 KOps/s $\color{#35bf28}+1.46\%$
test_step_mdp_speed[False-True-False-False-False] 49.6410μs 21.0808μs 47.4365 KOps/s 47.4997 KOps/s $\color{#d91a1a}-0.13\%$
test_step_mdp_speed[False-False-True-True-True] 97.9320μs 55.5277μs 18.0090 KOps/s 17.8524 KOps/s $\color{#35bf28}+0.88\%$
test_step_mdp_speed[False-False-True-True-False] 69.0810μs 35.5661μs 28.1167 KOps/s 27.5947 KOps/s $\color{#35bf28}+1.89\%$
test_step_mdp_speed[False-False-True-False-True] 80.7020μs 34.2801μs 29.1714 KOps/s 29.2343 KOps/s $\color{#d91a1a}-0.22\%$
test_step_mdp_speed[False-False-True-False-False] 52.4110μs 20.8438μs 47.9760 KOps/s 47.5080 KOps/s $\color{#35bf28}+0.99\%$
test_step_mdp_speed[False-False-False-True-True] 95.8410μs 57.3739μs 17.4295 KOps/s 17.1226 KOps/s $\color{#35bf28}+1.79\%$
test_step_mdp_speed[False-False-False-True-False] 69.8420μs 38.3509μs 26.0750 KOps/s 25.9685 KOps/s $\color{#35bf28}+0.41\%$
test_step_mdp_speed[False-False-False-False-True] 67.9710μs 36.8766μs 27.1175 KOps/s 27.4376 KOps/s $\color{#d91a1a}-1.17\%$
test_step_mdp_speed[False-False-False-False-False] 50.6200μs 23.3059μs 42.9077 KOps/s 42.4158 KOps/s $\color{#35bf28}+1.16\%$
test_values[generalized_advantage_estimate-True-True] 10.4598ms 10.1848ms 98.1858 Ops/s 97.9152 Ops/s $\color{#35bf28}+0.28\%$
test_values[vec_generalized_advantage_estimate-True-True] 20.3798ms 17.4753ms 57.2235 Ops/s 90.6251 Ops/s $\textbf{\color{#d91a1a}-36.86\%}$
test_values[td0_return_estimate-False-False] 0.2245ms 0.1298ms 7.7026 KOps/s 7.4446 KOps/s $\color{#35bf28}+3.47\%$
test_values[td1_return_estimate-False-False] 29.2298ms 27.9765ms 35.7443 Ops/s 35.1523 Ops/s $\color{#35bf28}+1.68\%$
test_values[vec_td1_return_estimate-False-False] 18.2084ms 17.5284ms 57.0503 Ops/s 90.1361 Ops/s $\textbf{\color{#d91a1a}-36.71\%}$
test_values[td_lambda_return_estimate-True-False] 42.6087ms 42.0904ms 23.7584 Ops/s 23.6807 Ops/s $\color{#35bf28}+0.33\%$
test_values[vec_td_lambda_return_estimate-True-False] 18.1545ms 17.5447ms 56.9974 Ops/s 89.9525 Ops/s $\textbf{\color{#d91a1a}-36.64\%}$
test_gae_speed[generalized_advantage_estimate-False-1-512] 8.9892ms 8.8966ms 112.4020 Ops/s 111.7337 Ops/s $\color{#35bf28}+0.60\%$
test_gae_speed[vec_generalized_advantage_estimate-True-1-512] 1.7238ms 1.5517ms 644.4565 Ops/s 659.8729 Ops/s $\color{#d91a1a}-2.34\%$
test_gae_speed[vec_generalized_advantage_estimate-False-1-512] 0.5152ms 0.4168ms 2.3992 KOps/s 2.3683 KOps/s $\color{#35bf28}+1.31\%$
test_gae_speed[vec_generalized_advantage_estimate-True-32-512] 34.9231ms 34.3952ms 29.0738 Ops/s 36.2518 Ops/s $\textbf{\color{#d91a1a}-19.80\%}$
test_gae_speed[vec_generalized_advantage_estimate-False-32-512] 2.1029ms 1.7138ms 583.4832 Ops/s 584.5407 Ops/s $\color{#d91a1a}-0.18\%$
test_dqn_speed[False-None] 1.5207ms 1.4206ms 703.9113 Ops/s 697.7180 Ops/s $\color{#35bf28}+0.89\%$
test_dqn_speed[False-backward] 2.0060ms 1.9372ms 516.2212 Ops/s 512.2893 Ops/s $\color{#35bf28}+0.77\%$
test_dqn_speed[True-None] 0.9253ms 0.5325ms 1.8778 KOps/s 1.9110 KOps/s $\color{#d91a1a}-1.73\%$
test_dqn_speed[True-backward] 1.0123ms 0.9663ms 1.0348 KOps/s 912.4023 Ops/s $\textbf{\color{#35bf28}+13.42\%}$
test_dqn_speed[reduce-overhead-None] 0.6836ms 0.5145ms 1.9436 KOps/s 1.9048 KOps/s $\color{#35bf28}+2.04\%$
test_dqn_speed[reduce-overhead-backward] 1.0140ms 0.9496ms 1.0531 KOps/s 909.8158 Ops/s $\textbf{\color{#35bf28}+15.75\%}$
test_ddpg_speed[False-None] 3.2813ms 2.9167ms 342.8483 Ops/s 342.4578 Ops/s $\color{#35bf28}+0.11\%$
test_ddpg_speed[False-backward] 4.2383ms 4.1294ms 242.1642 Ops/s 241.2104 Ops/s $\color{#35bf28}+0.40\%$
test_ddpg_speed[True-None] 1.7973ms 1.4074ms 710.5339 Ops/s 681.7638 Ops/s $\color{#35bf28}+4.22\%$
test_ddpg_speed[True-backward] 2.4732ms 2.3805ms 420.0773 Ops/s 347.3742 Ops/s $\textbf{\color{#35bf28}+20.93\%}$
test_ddpg_speed[reduce-overhead-None] 1.8049ms 1.3866ms 721.1683 Ops/s 717.4694 Ops/s $\color{#35bf28}+0.52\%$
test_ddpg_speed[reduce-overhead-backward] 2.4075ms 2.3609ms 423.5590 Ops/s 415.1013 Ops/s $\color{#35bf28}+2.04\%$
test_sac_speed[False-None] 8.3950ms 7.9602ms 125.6250 Ops/s 123.8185 Ops/s $\color{#35bf28}+1.46\%$
test_sac_speed[False-backward] 11.7810ms 11.2095ms 89.2097 Ops/s 87.7716 Ops/s $\color{#35bf28}+1.64\%$
test_sac_speed[True-None] 2.3320ms 2.1341ms 468.5741 Ops/s 455.7966 Ops/s $\color{#35bf28}+2.80\%$
test_sac_speed[True-backward] 4.4603ms 4.0749ms 245.4047 Ops/s 239.1546 Ops/s $\color{#35bf28}+2.61\%$
test_sac_speed[reduce-overhead-None] 2.4668ms 2.1303ms 469.4282 Ops/s 453.2342 Ops/s $\color{#35bf28}+3.57\%$
test_sac_speed[reduce-overhead-backward] 4.1093ms 4.0306ms 248.1011 Ops/s 232.4221 Ops/s $\textbf{\color{#35bf28}+6.75\%}$
test_redq_speed[False-None] 14.1017ms 10.2829ms 97.2485 Ops/s 94.3178 Ops/s $\color{#35bf28}+3.11\%$
test_redq_speed[False-backward] 19.7910ms 17.8662ms 55.9715 Ops/s 55.8188 Ops/s $\color{#35bf28}+0.27\%$
test_redq_speed[True-None] 4.7903ms 4.4552ms 224.4577 Ops/s 221.4825 Ops/s $\color{#35bf28}+1.34\%$
test_redq_speed[True-backward] 10.2226ms 9.8548ms 101.4732 Ops/s 96.3277 Ops/s $\textbf{\color{#35bf28}+5.34\%}$
test_redq_speed[reduce-overhead-None] 4.7645ms 4.4232ms 226.0788 Ops/s 226.5141 Ops/s $\color{#d91a1a}-0.19\%$
test_redq_speed[reduce-overhead-backward] 10.2190ms 9.9375ms 100.6286 Ops/s 92.4629 Ops/s $\textbf{\color{#35bf28}+8.83\%}$
test_redq_deprec_speed[False-None] 11.4970ms 11.1021ms 90.0729 Ops/s 89.0660 Ops/s $\color{#35bf28}+1.13\%$
test_redq_deprec_speed[False-backward] 16.4478ms 16.0328ms 62.3723 Ops/s 61.9679 Ops/s $\color{#35bf28}+0.65\%$
test_redq_deprec_speed[True-None] 4.0600ms 3.6701ms 272.4731 Ops/s 269.2839 Ops/s $\color{#35bf28}+1.18\%$
test_redq_deprec_speed[True-backward] 8.1335ms 7.7846ms 128.4593 Ops/s 125.1595 Ops/s $\color{#35bf28}+2.64\%$
test_redq_deprec_speed[reduce-overhead-None] 4.0780ms 3.6029ms 277.5565 Ops/s 272.3855 Ops/s $\color{#35bf28}+1.90\%$
test_redq_deprec_speed[reduce-overhead-backward] 7.8673ms 7.6694ms 130.3885 Ops/s 115.0070 Ops/s $\textbf{\color{#35bf28}+13.37\%}$
test_td3_speed[False-None] 8.1361ms 8.0134ms 124.7907 Ops/s 116.4504 Ops/s $\textbf{\color{#35bf28}+7.16\%}$
test_td3_speed[False-backward] 11.4380ms 10.9062ms 91.6908 Ops/s 89.5560 Ops/s $\color{#35bf28}+2.38\%$
test_td3_speed[True-None] 2.2339ms 1.8367ms 544.4414 Ops/s 551.5702 Ops/s $\color{#d91a1a}-1.29\%$
test_td3_speed[True-backward] 3.6919ms 3.5809ms 279.2608 Ops/s 269.2370 Ops/s $\color{#35bf28}+3.72\%$
test_td3_speed[reduce-overhead-None] 1.8324ms 1.7687ms 565.3817 Ops/s 562.8444 Ops/s $\color{#35bf28}+0.45\%$
test_td3_speed[reduce-overhead-backward] 3.7402ms 3.6099ms 277.0126 Ops/s 273.3507 Ops/s $\color{#35bf28}+1.34\%$
test_cql_speed[False-None] 26.7278ms 25.9263ms 38.5709 Ops/s 38.5829 Ops/s $\color{#d91a1a}-0.03\%$
test_cql_speed[False-backward] 36.1190ms 35.3923ms 28.2547 Ops/s 27.8441 Ops/s $\color{#35bf28}+1.47\%$
test_cql_speed[True-None] 12.6658ms 12.3558ms 80.9338 Ops/s 80.1690 Ops/s $\color{#35bf28}+0.95\%$
test_cql_speed[True-backward] 18.7766ms 18.1562ms 55.0775 Ops/s 55.2115 Ops/s $\color{#d91a1a}-0.24\%$
test_cql_speed[reduce-overhead-None] 12.7871ms 12.4883ms 80.0748 Ops/s 79.4234 Ops/s $\color{#35bf28}+0.82\%$
test_cql_speed[reduce-overhead-backward] 18.4355ms 18.0044ms 55.5419 Ops/s 55.4606 Ops/s $\color{#35bf28}+0.15\%$
test_a2c_speed[False-None] 5.9133ms 5.5724ms 179.4566 Ops/s 177.3868 Ops/s $\color{#35bf28}+1.17\%$
test_a2c_speed[False-backward] 12.3795ms 12.0102ms 83.2623 Ops/s 82.0145 Ops/s $\color{#35bf28}+1.52\%$
test_a2c_speed[True-None] 4.5557ms 3.7580ms 266.1024 Ops/s 264.0357 Ops/s $\color{#35bf28}+0.78\%$
test_a2c_speed[True-backward] 8.7986ms 8.5890ms 116.4274 Ops/s 113.8282 Ops/s $\color{#35bf28}+2.28\%$
test_a2c_speed[reduce-overhead-None] 3.9209ms 3.7446ms 267.0548 Ops/s 265.6478 Ops/s $\color{#35bf28}+0.53\%$
test_a2c_speed[reduce-overhead-backward] 9.0611ms 8.8928ms 112.4502 Ops/s 106.8102 Ops/s $\textbf{\color{#35bf28}+5.28\%}$
test_ppo_speed[False-None] 6.3842ms 6.0417ms 165.5168 Ops/s 163.1323 Ops/s $\color{#35bf28}+1.46\%$
test_ppo_speed[False-backward] 13.1063ms 12.7614ms 78.3614 Ops/s 77.8903 Ops/s $\color{#35bf28}+0.60\%$
test_ppo_speed[True-None] 3.8023ms 3.6495ms 274.0065 Ops/s 267.6352 Ops/s $\color{#35bf28}+2.38\%$
test_ppo_speed[True-backward] 8.6328ms 8.4807ms 117.9141 Ops/s 114.5960 Ops/s $\color{#35bf28}+2.90\%$
test_ppo_speed[reduce-overhead-None] 4.0779ms 3.6750ms 272.1057 Ops/s 268.2935 Ops/s $\color{#35bf28}+1.42\%$
test_ppo_speed[reduce-overhead-backward] 9.3203ms 8.7374ms 114.4512 Ops/s 113.1803 Ops/s $\color{#35bf28}+1.12\%$
test_reinforce_speed[False-None] 5.0626ms 4.6333ms 215.8307 Ops/s 213.1076 Ops/s $\color{#35bf28}+1.28\%$
test_reinforce_speed[False-backward] 7.8778ms 7.5083ms 133.1866 Ops/s 132.5259 Ops/s $\color{#35bf28}+0.50\%$
test_reinforce_speed[True-None] 3.3872ms 2.9249ms 341.8899 Ops/s 334.0331 Ops/s $\color{#35bf28}+2.35\%$
test_reinforce_speed[True-backward] 7.8964ms 7.7402ms 129.1960 Ops/s 128.6131 Ops/s $\color{#35bf28}+0.45\%$
test_reinforce_speed[reduce-overhead-None] 3.0487ms 2.8896ms 346.0639 Ops/s 342.7538 Ops/s $\color{#35bf28}+0.97\%$
test_reinforce_speed[reduce-overhead-backward] 8.1237ms 7.9409ms 125.9296 Ops/s 107.7735 Ops/s $\textbf{\color{#35bf28}+16.85\%}$
test_iql_speed[False-None] 25.4065ms 20.3144ms 49.2262 Ops/s 49.9061 Ops/s $\color{#d91a1a}-1.36\%$
test_iql_speed[False-backward] 36.2619ms 30.6792ms 32.5954 Ops/s 32.4252 Ops/s $\color{#35bf28}+0.52\%$
test_iql_speed[True-None] 9.2841ms 8.5296ms 117.2388 Ops/s 108.4524 Ops/s $\textbf{\color{#35bf28}+8.10\%}$
test_iql_speed[True-backward] 17.2080ms 16.7042ms 59.8651 Ops/s 58.7565 Ops/s $\color{#35bf28}+1.89\%$
test_iql_speed[reduce-overhead-None] 8.9419ms 8.5836ms 116.5010 Ops/s 112.9510 Ops/s $\color{#35bf28}+3.14\%$
test_iql_speed[reduce-overhead-backward] 17.4891ms 17.1909ms 58.1703 Ops/s 56.7160 Ops/s $\color{#35bf28}+2.56\%$
test_rb_sample[TensorDictReplayBuffer-ListStorage-RandomSampler-4000] 7.4421ms 6.0323ms 165.7731 Ops/s 166.7630 Ops/s $\color{#d91a1a}-0.59\%$
test_rb_sample[TensorDictReplayBuffer-LazyMemmapStorage-RandomSampler-10000] 0.5430ms 0.3130ms 3.1952 KOps/s 3.5000 KOps/s $\textbf{\color{#d91a1a}-8.71\%}$
test_rb_sample[TensorDictReplayBuffer-LazyTensorStorage-RandomSampler-10000] 0.5836ms 0.3103ms 3.2226 KOps/s 3.2223 KOps/s $+0.01\%$
test_rb_sample[TensorDictReplayBuffer-ListStorage-SamplerWithoutReplacement-4000] 6.0367ms 5.7432ms 174.1181 Ops/s 176.5318 Ops/s $\color{#d91a1a}-1.37\%$
test_rb_sample[TensorDictReplayBuffer-LazyMemmapStorage-SamplerWithoutReplacement-10000] 2.1806ms 0.3428ms 2.9169 KOps/s 3.0838 KOps/s $\textbf{\color{#d91a1a}-5.41\%}$
test_rb_sample[TensorDictReplayBuffer-LazyTensorStorage-SamplerWithoutReplacement-10000] 0.7227ms 0.3090ms 3.2360 KOps/s 3.5214 KOps/s $\textbf{\color{#d91a1a}-8.10\%}$
test_rb_sample[TensorDictReplayBuffer-LazyMemmapStorage-sampler6-10000] 1.7274ms 1.4240ms 702.2500 Ops/s 791.1288 Ops/s $\textbf{\color{#d91a1a}-11.23\%}$
test_rb_sample[TensorDictReplayBuffer-LazyTensorStorage-sampler7-10000] 1.6021ms 1.3375ms 747.6740 Ops/s 845.8679 Ops/s $\textbf{\color{#d91a1a}-11.61\%}$
test_rb_sample[TensorDictPrioritizedReplayBuffer-ListStorage-None-4000] 9.8832ms 6.0416ms 165.5180 Ops/s 171.7002 Ops/s $\color{#d91a1a}-3.60\%$
test_rb_sample[TensorDictPrioritizedReplayBuffer-LazyMemmapStorage-None-10000] 1.1339ms 0.4945ms 2.0222 KOps/s 2.2369 KOps/s $\textbf{\color{#d91a1a}-9.60\%}$
test_rb_sample[TensorDictPrioritizedReplayBuffer-LazyTensorStorage-None-10000] 0.8467ms 0.4347ms 2.3002 KOps/s 2.3418 KOps/s $\color{#d91a1a}-1.78\%$
test_rb_iterate[TensorDictReplayBuffer-ListStorage-RandomSampler-4000] 5.9968ms 5.7510ms 173.8840 Ops/s 176.9090 Ops/s $\color{#d91a1a}-1.71\%$
test_rb_iterate[TensorDictReplayBuffer-LazyMemmapStorage-RandomSampler-10000] 2.2564ms 0.3422ms 2.9223 KOps/s 822.3674 Ops/s $\textbf{\color{#35bf28}+255.35\%}$
test_rb_iterate[TensorDictReplayBuffer-LazyTensorStorage-RandomSampler-10000] 0.6341ms 0.3106ms 3.2194 KOps/s 2.9940 KOps/s $\textbf{\color{#35bf28}+7.53\%}$
test_rb_iterate[TensorDictReplayBuffer-ListStorage-SamplerWithoutReplacement-4000] 5.9301ms 5.6946ms 175.6060 Ops/s 174.0017 Ops/s $\color{#35bf28}+0.92\%$
test_rb_iterate[TensorDictReplayBuffer-LazyMemmapStorage-SamplerWithoutReplacement-10000] 0.8008ms 0.3362ms 2.9744 KOps/s 3.2899 KOps/s $\textbf{\color{#d91a1a}-9.59\%}$
test_rb_iterate[TensorDictReplayBuffer-LazyTensorStorage-SamplerWithoutReplacement-10000] 0.5556ms 0.3195ms 3.1299 KOps/s 3.5386 KOps/s $\textbf{\color{#d91a1a}-11.55\%}$
test_rb_iterate[TensorDictPrioritizedReplayBuffer-ListStorage-None-4000] 5.9339ms 5.8707ms 170.3375 Ops/s 169.4030 Ops/s $\color{#35bf28}+0.55\%$
test_rb_iterate[TensorDictPrioritizedReplayBuffer-LazyMemmapStorage-None-10000] 1.0105ms 0.4867ms 2.0546 KOps/s 2.1234 KOps/s $\color{#d91a1a}-3.24\%$
test_rb_iterate[TensorDictPrioritizedReplayBuffer-LazyTensorStorage-None-10000] 0.6842ms 0.4683ms 2.1353 KOps/s 2.2240 KOps/s $\color{#d91a1a}-3.99\%$
test_rb_populate[TensorDictReplayBuffer-ListStorage-RandomSampler-400] 7.5297ms 5.1245ms 195.1416 Ops/s 193.7390 Ops/s $\color{#35bf28}+0.72\%$
test_rb_populate[TensorDictReplayBuffer-LazyMemmapStorage-RandomSampler-400] 8.6139ms 2.5678ms 389.4391 Ops/s 435.0811 Ops/s $\textbf{\color{#d91a1a}-10.49\%}$
test_rb_populate[TensorDictReplayBuffer-LazyTensorStorage-RandomSampler-400] 7.3738ms 1.3205ms 757.3039 Ops/s 765.4588 Ops/s $\color{#d91a1a}-1.07\%$
test_rb_populate[TensorDictReplayBuffer-ListStorage-SamplerWithoutReplacement-400] 0.5211s 15.7270ms 63.5849 Ops/s 53.6200 Ops/s $\textbf{\color{#35bf28}+18.58\%}$
test_rb_populate[TensorDictReplayBuffer-LazyMemmapStorage-SamplerWithoutReplacement-400] 10.5920ms 2.0835ms 479.9667 Ops/s 509.0023 Ops/s $\textbf{\color{#d91a1a}-5.70\%}$
test_rb_populate[TensorDictReplayBuffer-LazyTensorStorage-SamplerWithoutReplacement-400] 5.7132ms 1.1124ms 898.9391 Ops/s 809.3748 Ops/s $\textbf{\color{#35bf28}+11.07\%}$
test_rb_populate[TensorDictPrioritizedReplayBuffer-ListStorage-None-400] 6.7562ms 5.2066ms 192.0632 Ops/s 182.8822 Ops/s $\textbf{\color{#35bf28}+5.02\%}$
test_rb_populate[TensorDictPrioritizedReplayBuffer-LazyMemmapStorage-None-400] 6.0970ms 2.1484ms 465.4581 Ops/s 442.6980 Ops/s $\textbf{\color{#35bf28}+5.14\%}$
test_rb_populate[TensorDictPrioritizedReplayBuffer-LazyTensorStorage-None-400] 10.6606ms 1.3769ms 726.2733 Ops/s 732.0154 Ops/s $\color{#d91a1a}-0.78\%$
test_rb_extend_sample[ReplayBuffer-LazyTensorStorage-RandomSampler-10000-10000-100-True] 35.3354ms 33.4616ms 29.8850 Ops/s 29.4414 Ops/s $\color{#35bf28}+1.51\%$
test_rb_extend_sample[ReplayBuffer-LazyTensorStorage-RandomSampler-10000-10000-100-False] 19.3880ms 17.7487ms 56.3422 Ops/s 54.3916 Ops/s $\color{#35bf28}+3.59\%$
test_rb_extend_sample[ReplayBuffer-LazyTensorStorage-RandomSampler-100000-10000-100-True] 34.7849ms 33.8504ms 29.5418 Ops/s 28.3894 Ops/s $\color{#35bf28}+4.06\%$
test_rb_extend_sample[ReplayBuffer-LazyTensorStorage-RandomSampler-100000-10000-100-False] 19.5156ms 17.8782ms 55.9340 Ops/s 54.4636 Ops/s $\color{#35bf28}+2.70\%$
test_rb_extend_sample[ReplayBuffer-LazyTensorStorage-RandomSampler-1000000-10000-100-True] 38.7755ms 36.2775ms 27.5653 Ops/s 27.6013 Ops/s $\color{#d91a1a}-0.13\%$
test_rb_extend_sample[ReplayBuffer-LazyTensorStorage-RandomSampler-1000000-10000-100-False] 20.0157ms 19.0365ms 52.5307 Ops/s 51.0171 Ops/s $\color{#35bf28}+2.97\%$

vmoens added a commit that referenced this pull request Oct 30, 2025
@vmoens
[Refactor] Make env creator optional for Ray
b599d9b 
ghstack-source-id: 6444d2c
Pull-Request: #3227
@vmoens vmoens merged commit 9441d6a into gh/vmoens/172/base Oct 30, 2025
87 of 101 checks passed
@vmoens vmoens deleted the gh/vmoens/172/head branch October 30, 2025 18:16
vmoens added a commit that referenced this pull request Oct 30, 2025
@vmoens
[Refactor] Make env creator optional for Ray
4a58d57 
ghstack-source-id: 6444d2c
Pull-Request: #3227
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Reviewers

No reviews

Assignees

No one assigned

Labels

CLA Signed This label is managed by the Facebook bot. Authors need to sign the CLA before a PR can be reviewed.

Projects

None yet

Milestone

No milestone

Development

Successfully merging this pull request may close these issues.

2 participants

@vmoens