Skip to content

[Feature] Auto-batching inference server: Monarch transport#3496

Open
vmoens wants to merge 4 commits intogh/vmoens/238/basefrom
gh/vmoens/238/head
Open

[Feature] Auto-batching inference server: Monarch transport#3496
vmoens wants to merge 4 commits intogh/vmoens/238/basefrom
gh/vmoens/238/head

Conversation

@vmoens
Copy link
Collaborator

@vmoens vmoens commented Feb 11, 2026 •
edited
Loading

Stack from ghstack (oldest at bottom):

Adds MonarchTransport for distributed inference on GPU clusters using
Monarch's actor model and RDMA channels. Monarch is imported lazily
at instantiation time.

Co-authored-by: Cursor cursoragent@cursor.com

@vmoens
Update
113917d 
[ghstack-poisoned]
@pytorch-bot
Copy link

pytorch-bot bot commented Feb 11, 2026 •
edited
Loading

🔗 Helpful Links

🧪 See artifacts and rendered test results at hud.pytorch.org/pr/pytorch/rl/3496

Note: Links to docs will display an error until the docs builds have been completed.

❌ 4 New Failures

As of commit afcd83c with merge base 266e4aa (image):

NEW FAILURES - The following jobs have failed:

This comment was automatically generated by Dr. CI and updates every 15 minutes.

@vmoens vmoens mentioned this pull request Feb 11, 2026
Open
@vmoens vmoens mentioned this pull request Feb 11, 2026
Open
@github-actions github-actions bot added the Feature New feature label Feb 11, 2026
This was referenced Feb 11, 2026
Open
Open
Open
@meta-cla meta-cla bot added the CLA Signed This label is managed by the Facebook bot. Authors need to sign the CLA before a PR can be reviewed. label Feb 11, 2026
@github-actions
Copy link
Contributor

github-actions bot commented Feb 11, 2026 •
edited
Loading

$\color{#D29922}\textsf{\Large⚠\kern{0.2cm}\normalsize Warning}$ Result of CPU Benchmark Tests

Total Benchmarks: 173. Improved: $\large\color{#35bf28}16$. Worsened: $\large\color{#d91a1a}9$.

Expand to view detailed results
Name Max Mean Ops Ops on Repo HEAD Change
test_tensor_to_bytestream_speed[pickle] 79.1073μs 78.3351μs 12.7657 KOps/s 12.5799 KOps/s $\color{#35bf28}+1.48\%$
test_tensor_to_bytestream_speed[torch.save] 0.1364ms 0.1355ms 7.3814 KOps/s 7.2570 KOps/s $\color{#35bf28}+1.71\%$
test_tensor_to_bytestream_speed[untyped_storage] 0.1004s 99.9200ms 10.0080 Ops/s 9.9547 Ops/s $\color{#35bf28}+0.54\%$
test_tensor_to_bytestream_speed[numpy] 2.3939μs 2.3827μs 419.6911 KOps/s 419.5212 KOps/s $\color{#35bf28}+0.04\%$
test_tensor_to_bytestream_speed[safetensors] 38.6377μs 38.3305μs 26.0889 KOps/s 26.0055 KOps/s $\color{#35bf28}+0.32\%$
test_simple 0.5293s 0.5282s 1.8934 Ops/s 1.8025 Ops/s $\textbf{\color{#35bf28}+5.04\%}$
test_transformed 1.0581s 1.0545s 0.9483 Ops/s 0.9347 Ops/s $\color{#35bf28}+1.45\%$
test_serial 1.6240s 1.6124s 0.6202 Ops/s 0.6104 Ops/s $\color{#35bf28}+1.61\%$
test_parallel 1.1037s 1.0127s 0.9875 Ops/s 0.9852 Ops/s $\color{#35bf28}+0.23\%$
test_step_mdp_speed[True-True-True-True-True] 0.1764ms 41.6180μs 24.0281 KOps/s 24.3212 KOps/s $\color{#d91a1a}-1.21\%$
test_step_mdp_speed[True-True-True-True-False] 55.5710μs 22.8869μs 43.6930 KOps/s 43.1974 KOps/s $\color{#35bf28}+1.15\%$
test_step_mdp_speed[True-True-True-False-True] 56.2610μs 23.6663μs 42.2542 KOps/s 42.3721 KOps/s $\color{#d91a1a}-0.28\%$
test_step_mdp_speed[True-True-True-False-False] 42.7800μs 13.0366μs 76.7070 KOps/s 77.0425 KOps/s $\color{#d91a1a}-0.44\%$
test_step_mdp_speed[True-True-False-True-True] 77.6310μs 44.9967μs 22.2238 KOps/s 22.6882 KOps/s $\color{#d91a1a}-2.05\%$
test_step_mdp_speed[True-True-False-True-False] 50.9510μs 25.4987μs 39.2177 KOps/s 38.9354 KOps/s $\color{#35bf28}+0.73\%$
test_step_mdp_speed[True-True-False-False-True] 61.7410μs 26.6905μs 37.4665 KOps/s 38.8234 KOps/s $\color{#d91a1a}-3.50\%$
test_step_mdp_speed[True-True-False-False-False] 47.2410μs 15.6826μs 63.7648 KOps/s 65.1403 KOps/s $\color{#d91a1a}-2.11\%$
test_step_mdp_speed[True-False-True-True-True] 85.2310μs 47.7273μs 20.9524 KOps/s 21.3317 KOps/s $\color{#d91a1a}-1.78\%$
test_step_mdp_speed[True-False-True-True-False] 59.3210μs 28.6064μs 34.9572 KOps/s 34.9750 KOps/s $\color{#d91a1a}-0.05\%$
test_step_mdp_speed[True-False-True-False-True] 56.4310μs 26.4875μs 37.7537 KOps/s 39.1539 KOps/s $\color{#d91a1a}-3.58\%$
test_step_mdp_speed[True-False-True-False-False] 47.0610μs 15.3436μs 65.1737 KOps/s 65.2486 KOps/s $\color{#d91a1a}-0.11\%$
test_step_mdp_speed[True-False-False-True-True] 85.8410μs 50.3587μs 19.8575 KOps/s 20.2698 KOps/s $\color{#d91a1a}-2.03\%$
test_step_mdp_speed[True-False-False-True-False] 62.0510μs 30.9334μs 32.3276 KOps/s 32.6436 KOps/s $\color{#d91a1a}-0.97\%$
test_step_mdp_speed[True-False-False-False-True] 62.8110μs 28.8080μs 34.7125 KOps/s 36.0489 KOps/s $\color{#d91a1a}-3.71\%$
test_step_mdp_speed[True-False-False-False-False] 47.5310μs 18.0708μs 55.3378 KOps/s 55.8231 KOps/s $\color{#d91a1a}-0.87\%$
test_step_mdp_speed[False-True-True-True-True] 85.0710μs 46.9523μs 21.2982 KOps/s 21.6781 KOps/s $\color{#d91a1a}-1.75\%$
test_step_mdp_speed[False-True-True-True-False] 56.9810μs 28.2852μs 35.3542 KOps/s 34.5516 KOps/s $\color{#35bf28}+2.32\%$
test_step_mdp_speed[False-True-True-False-True] 2.4604ms 30.3240μs 32.9772 KOps/s 33.2902 KOps/s $\color{#d91a1a}-0.94\%$
test_step_mdp_speed[False-True-True-False-False] 47.3600μs 17.2058μs 58.1198 KOps/s 57.1094 KOps/s $\color{#35bf28}+1.77\%$
test_step_mdp_speed[False-True-False-True-True] 94.8810μs 49.5527μs 20.1806 KOps/s 20.2716 KOps/s $\color{#d91a1a}-0.45\%$
test_step_mdp_speed[False-True-False-True-False] 61.6600μs 30.7238μs 32.5481 KOps/s 32.5573 KOps/s $\color{#d91a1a}-0.03\%$
test_step_mdp_speed[False-True-False-False-True] 65.0010μs 32.0272μs 31.2235 KOps/s 31.2457 KOps/s $\color{#d91a1a}-0.07\%$
test_step_mdp_speed[False-True-False-False-False] 57.9810μs 19.5352μs 51.1896 KOps/s 50.9237 KOps/s $\color{#35bf28}+0.52\%$
test_step_mdp_speed[False-False-True-True-True] 88.5020μs 52.2940μs 19.1226 KOps/s 19.2926 KOps/s $\color{#d91a1a}-0.88\%$
test_step_mdp_speed[False-False-True-True-False] 82.8710μs 33.2392μs 30.0850 KOps/s 29.6607 KOps/s $\color{#35bf28}+1.43\%$
test_step_mdp_speed[False-False-True-False-True] 67.0310μs 32.4154μs 30.8495 KOps/s 31.1540 KOps/s $\color{#d91a1a}-0.98\%$
test_step_mdp_speed[False-False-True-False-False] 46.8210μs 19.6756μs 50.8243 KOps/s 50.8378 KOps/s $\color{#d91a1a}-0.03\%$
test_step_mdp_speed[False-False-False-True-True] 87.8610μs 54.2650μs 18.4281 KOps/s 18.7530 KOps/s $\color{#d91a1a}-1.73\%$
test_step_mdp_speed[False-False-False-True-False] 65.8310μs 35.9172μs 27.8418 KOps/s 28.1103 KOps/s $\color{#d91a1a}-0.96\%$
test_step_mdp_speed[False-False-False-False-True] 74.1410μs 33.9388μs 29.4648 KOps/s 29.3852 KOps/s $\color{#35bf28}+0.27\%$
test_step_mdp_speed[False-False-False-False-False] 66.2710μs 21.4776μs 46.5602 KOps/s 45.5494 KOps/s $\color{#35bf28}+2.22\%$
test_non_tensor_env_rollout_speed[1000-single-True] 0.7047s 0.7018s 1.4250 Ops/s 1.3866 Ops/s $\color{#35bf28}+2.77\%$
test_non_tensor_env_rollout_speed[1000-single-False] 0.6886s 0.5922s 1.6885 Ops/s 1.7031 Ops/s $\color{#d91a1a}-0.85\%$
test_non_tensor_env_rollout_speed[1000-serial-no-buffers-True] 1.6775s 1.5980s 0.6258 Ops/s 0.6271 Ops/s $\color{#d91a1a}-0.22\%$
test_non_tensor_env_rollout_speed[1000-serial-no-buffers-False] 1.4546s 1.3742s 0.7277 Ops/s 0.7265 Ops/s $\color{#35bf28}+0.16\%$
test_non_tensor_env_rollout_speed[1000-serial-buffers-True] 1.9131s 1.8313s 0.5460 Ops/s 0.5475 Ops/s $\color{#d91a1a}-0.27\%$
test_non_tensor_env_rollout_speed[1000-serial-buffers-False] 1.6901s 1.6092s 0.6214 Ops/s 0.6231 Ops/s $\color{#d91a1a}-0.28\%$
test_non_tensor_env_rollout_speed[1000-parallel-no-buffers-True] 4.6711s 4.5208s 0.2212 Ops/s 0.2218 Ops/s $\color{#d91a1a}-0.29\%$
test_non_tensor_env_rollout_speed[1000-parallel-no-buffers-False] 4.3604s 4.2957s 0.2328 Ops/s 0.2307 Ops/s $\color{#35bf28}+0.89\%$
test_non_tensor_env_rollout_speed[1000-parallel-buffers-True] 1.8940s 1.8212s 0.5491 Ops/s 0.5250 Ops/s $\color{#35bf28}+4.60\%$
test_non_tensor_env_rollout_speed[1000-parallel-buffers-False] 1.6232s 1.5406s 0.6491 Ops/s 0.6542 Ops/s $\color{#d91a1a}-0.77\%$
test_values[generalized_advantage_estimate-True-True] 9.7803ms 9.6008ms 104.1575 Ops/s 104.3828 Ops/s $\color{#d91a1a}-0.22\%$
test_values[vec_generalized_advantage_estimate-True-True] 19.4372ms 17.5167ms 57.0883 Ops/s 54.6831 Ops/s $\color{#35bf28}+4.40\%$
test_values[td0_return_estimate-False-False] 0.2414ms 0.1281ms 7.8051 KOps/s 7.7865 KOps/s $\color{#35bf28}+0.24\%$
test_values[td1_return_estimate-False-False] 26.1156ms 25.8645ms 38.6631 Ops/s 38.8286 Ops/s $\color{#d91a1a}-0.43\%$
test_values[vec_td1_return_estimate-False-False] 18.3005ms 17.6524ms 56.6495 Ops/s 53.6661 Ops/s $\textbf{\color{#35bf28}+5.56\%}$
test_values[td_lambda_return_estimate-True-False] 40.7508ms 38.4963ms 25.9765 Ops/s 26.2079 Ops/s $\color{#d91a1a}-0.88\%$
test_values[vec_td_lambda_return_estimate-True-False] 18.8348ms 17.6460ms 56.6702 Ops/s 54.1704 Ops/s $\color{#35bf28}+4.61\%$
test_gae_speed[generalized_advantage_estimate-False-1-512] 8.5086ms 8.4436ms 118.4333 Ops/s 117.7006 Ops/s $\color{#35bf28}+0.62\%$
test_gae_speed[vec_generalized_advantage_estimate-True-1-512] 1.6813ms 1.4884ms 671.8814 Ops/s 655.7469 Ops/s $\color{#35bf28}+2.46\%$
test_gae_speed[vec_generalized_advantage_estimate-False-1-512] 0.5646ms 0.3965ms 2.5219 KOps/s 2.5162 KOps/s $\color{#35bf28}+0.23\%$
test_gae_speed[vec_generalized_advantage_estimate-True-32-512] 35.3274ms 34.8063ms 28.7304 Ops/s 28.8324 Ops/s $\color{#d91a1a}-0.35\%$
test_gae_speed[vec_generalized_advantage_estimate-False-32-512] 1.9628ms 1.7201ms 581.3575 Ops/s 581.7121 Ops/s $\color{#d91a1a}-0.06\%$
test_dqn_speed[False-None] 1.7383ms 1.3438ms 744.1352 Ops/s 740.4481 Ops/s $\color{#35bf28}+0.50\%$
test_dqn_speed[False-backward] 1.8814ms 1.8419ms 542.9229 Ops/s 538.9086 Ops/s $\color{#35bf28}+0.74\%$
test_dqn_speed[True-None] 0.7314ms 0.5883ms 1.6998 KOps/s 1.8295 KOps/s $\textbf{\color{#d91a1a}-7.09\%}$
test_dqn_speed[True-backward] 1.0468ms 0.9904ms 1.0097 KOps/s 843.6236 Ops/s $\textbf{\color{#35bf28}+19.69\%}$
test_dqn_speed[reduce-overhead-None] 0.9471ms 0.5329ms 1.8766 KOps/s 1.7728 KOps/s $\textbf{\color{#35bf28}+5.86\%}$
test_ddpg_speed[False-None] 3.2432ms 2.7757ms 360.2672 Ops/s 359.3202 Ops/s $\color{#35bf28}+0.26\%$
test_ddpg_speed[False-backward] 4.1784ms 3.9814ms 251.1677 Ops/s 254.2580 Ops/s $\color{#d91a1a}-1.22\%$
test_ddpg_speed[True-None] 1.7903ms 1.3878ms 720.5470 Ops/s 702.1077 Ops/s $\color{#35bf28}+2.63\%$
test_ddpg_speed[True-backward] 2.3921ms 2.3454ms 426.3754 Ops/s 383.6957 Ops/s $\textbf{\color{#35bf28}+11.12\%}$
test_ddpg_speed[reduce-overhead-None] 2.1108ms 1.4012ms 713.6991 Ops/s 730.9596 Ops/s $\color{#d91a1a}-2.36\%$
test_sac_speed[False-None] 8.5964ms 7.7079ms 129.7371 Ops/s 131.2115 Ops/s $\color{#d91a1a}-1.12\%$
test_sac_speed[False-backward] 11.0227ms 10.8008ms 92.5856 Ops/s 92.8045 Ops/s $\color{#d91a1a}-0.24\%$
test_sac_speed[True-None] 2.5009ms 2.1165ms 472.4797 Ops/s 463.0426 Ops/s $\color{#35bf28}+2.04\%$
test_sac_speed[True-backward] 4.1342ms 4.0090ms 249.4372 Ops/s 228.0679 Ops/s $\textbf{\color{#35bf28}+9.37\%}$
test_sac_speed[reduce-overhead-None] 2.5169ms 2.1241ms 470.7896 Ops/s 465.1369 Ops/s $\color{#35bf28}+1.22\%$
test_redq_speed[False-None] 13.4611ms 10.4351ms 95.8308 Ops/s 98.1135 Ops/s $\color{#d91a1a}-2.33\%$
test_redq_speed[False-backward] 18.3752ms 17.5385ms 57.0175 Ops/s 58.6387 Ops/s $\color{#d91a1a}-2.76\%$
test_redq_speed[True-None] 4.8199ms 4.4147ms 226.5167 Ops/s 218.1683 Ops/s $\color{#35bf28}+3.83\%$
test_redq_speed[True-backward] 10.3064ms 9.6446ms 103.6849 Ops/s 107.5496 Ops/s $\color{#d91a1a}-3.59\%$
test_redq_speed[reduce-overhead-None] 4.8135ms 4.4094ms 226.7899 Ops/s 227.8350 Ops/s $\color{#d91a1a}-0.46\%$
test_redq_deprec_speed[False-None] 11.1514ms 10.6562ms 93.8417 Ops/s 92.3321 Ops/s $\color{#35bf28}+1.63\%$
test_redq_deprec_speed[False-backward] 15.8239ms 15.3271ms 65.2437 Ops/s 63.6433 Ops/s $\color{#35bf28}+2.51\%$
test_redq_deprec_speed[True-None] 3.9934ms 3.6188ms 276.3324 Ops/s 278.0852 Ops/s $\color{#d91a1a}-0.63\%$
test_redq_deprec_speed[True-backward] 7.6345ms 7.3436ms 136.1734 Ops/s 140.1071 Ops/s $\color{#d91a1a}-2.81\%$
test_redq_deprec_speed[reduce-overhead-None] 3.9504ms 3.5208ms 284.0250 Ops/s 277.6237 Ops/s $\color{#35bf28}+2.31\%$
test_td3_speed[False-None] 7.8209ms 7.6793ms 130.2196 Ops/s 129.9196 Ops/s $\color{#35bf28}+0.23\%$
test_td3_speed[False-backward] 10.9868ms 10.5169ms 95.0855 Ops/s 94.8861 Ops/s $\color{#35bf28}+0.21\%$
test_td3_speed[True-None] 1.9145ms 1.8449ms 542.0257 Ops/s 542.5123 Ops/s $\color{#d91a1a}-0.09\%$
test_td3_speed[True-backward] 3.7088ms 3.5940ms 278.2425 Ops/s 276.2918 Ops/s $\color{#35bf28}+0.71\%$
test_td3_speed[reduce-overhead-None] 1.8244ms 1.7882ms 559.2253 Ops/s 562.2002 Ops/s $\color{#d91a1a}-0.53\%$
test_cql_speed[False-None] 27.7039ms 25.2477ms 39.6076 Ops/s 40.5776 Ops/s $\color{#d91a1a}-2.39\%$
test_cql_speed[False-backward] 39.6198ms 35.1964ms 28.4120 Ops/s 29.4069 Ops/s $\color{#d91a1a}-3.38\%$
test_cql_speed[True-None] 12.8122ms 12.0514ms 82.9782 Ops/s 86.0700 Ops/s $\color{#d91a1a}-3.59\%$
test_cql_speed[True-backward] 17.8688ms 17.4689ms 57.2446 Ops/s 54.4752 Ops/s $\textbf{\color{#35bf28}+5.08\%}$
test_cql_speed[reduce-overhead-None] 12.6690ms 12.2093ms 81.9049 Ops/s 80.9236 Ops/s $\color{#35bf28}+1.21\%$
test_a2c_speed[False-None] 7.3104ms 5.3841ms 185.7327 Ops/s 187.0257 Ops/s $\color{#d91a1a}-0.69\%$
test_a2c_speed[False-backward] 12.1733ms 11.7252ms 85.2862 Ops/s 84.7869 Ops/s $\color{#35bf28}+0.59\%$
test_a2c_speed[True-None] 4.0268ms 3.6597ms 273.2427 Ops/s 260.3005 Ops/s $\color{#35bf28}+4.97\%$
test_a2c_speed[True-backward] 9.0432ms 8.5824ms 116.5175 Ops/s 114.7628 Ops/s $\color{#35bf28}+1.53\%$
test_a2c_speed[reduce-overhead-None] 4.0061ms 3.6665ms 272.7419 Ops/s 266.4232 Ops/s $\color{#35bf28}+2.37\%$
test_ppo_speed[False-None] 6.2808ms 5.8019ms 172.3564 Ops/s 168.6916 Ops/s $\color{#35bf28}+2.17\%$
test_ppo_speed[False-backward] 12.9735ms 12.2308ms 81.7605 Ops/s 80.5185 Ops/s $\color{#35bf28}+1.54\%$
test_ppo_speed[True-None] 5.0627ms 3.6848ms 271.3833 Ops/s 267.1032 Ops/s $\color{#35bf28}+1.60\%$
test_ppo_speed[True-backward] 8.9277ms 8.5187ms 117.3888 Ops/s 114.8908 Ops/s $\color{#35bf28}+2.17\%$
test_ppo_speed[reduce-overhead-None] 4.0028ms 3.5980ms 277.9319 Ops/s 274.0400 Ops/s $\color{#35bf28}+1.42\%$
test_reinforce_speed[False-None] 4.8474ms 4.5047ms 221.9921 Ops/s 213.7225 Ops/s $\color{#35bf28}+3.87\%$
test_reinforce_speed[False-backward] 7.6607ms 7.3510ms 136.0355 Ops/s 135.1959 Ops/s $\color{#35bf28}+0.62\%$
test_reinforce_speed[True-None] 3.3457ms 2.9036ms 344.3953 Ops/s 343.5182 Ops/s $\color{#35bf28}+0.26\%$
test_reinforce_speed[True-backward] 8.0202ms 7.7457ms 129.1041 Ops/s 124.4511 Ops/s $\color{#35bf28}+3.74\%$
test_reinforce_speed[reduce-overhead-None] 3.2478ms 2.8831ms 346.8504 Ops/s 349.7981 Ops/s $\color{#d91a1a}-0.84\%$
test_iql_speed[False-None] 24.4922ms 19.7970ms 50.5126 Ops/s 50.0389 Ops/s $\color{#35bf28}+0.95\%$
test_iql_speed[False-backward] 36.7798ms 30.5457ms 32.7378 Ops/s 32.7472 Ops/s $\color{#d91a1a}-0.03\%$
test_iql_speed[True-None] 11.0912ms 8.5166ms 117.4179 Ops/s 116.2120 Ops/s $\color{#35bf28}+1.04\%$
test_iql_speed[True-backward] 16.8223ms 16.4734ms 60.7039 Ops/s 60.3185 Ops/s $\color{#35bf28}+0.64\%$
test_iql_speed[reduce-overhead-None] 8.7676ms 8.5123ms 117.4771 Ops/s 115.8096 Ops/s $\color{#35bf28}+1.44\%$
test_rb_sample[TensorDictReplayBuffer-ListStorage-RandomSampler-4000] 6.2254ms 5.8200ms 171.8227 Ops/s 169.3567 Ops/s $\color{#35bf28}+1.46\%$
test_rb_sample[TensorDictReplayBuffer-LazyMemmapStorage-RandomSampler-10000] 2.1413ms 0.3055ms 3.2733 KOps/s 3.1952 KOps/s $\color{#35bf28}+2.44\%$
test_rb_sample[TensorDictReplayBuffer-LazyTensorStorage-RandomSampler-10000] 0.7973ms 0.3314ms 3.0171 KOps/s 3.2470 KOps/s $\textbf{\color{#d91a1a}-7.08\%}$
test_rb_sample[TensorDictReplayBuffer-ListStorage-SamplerWithoutReplacement-4000] 6.2080ms 5.6711ms 176.3328 Ops/s 177.2553 Ops/s $\color{#d91a1a}-0.52\%$
test_rb_sample[TensorDictReplayBuffer-LazyMemmapStorage-SamplerWithoutReplacement-10000] 1.6565ms 0.2810ms 3.5583 KOps/s 3.0208 KOps/s $\textbf{\color{#35bf28}+17.79\%}$
test_rb_sample[TensorDictReplayBuffer-LazyTensorStorage-SamplerWithoutReplacement-10000] 0.8572ms 0.3265ms 3.0632 KOps/s 3.2946 KOps/s $\textbf{\color{#d91a1a}-7.02\%}$
test_rb_sample[TensorDictReplayBuffer-LazyMemmapStorage-sampler6-10000] 1.6288ms 1.2079ms 827.8955 Ops/s 777.2609 Ops/s $\textbf{\color{#35bf28}+6.51\%}$
test_rb_sample[TensorDictReplayBuffer-LazyTensorStorage-sampler7-10000] 1.3290ms 1.1312ms 883.9804 Ops/s 823.7465 Ops/s $\textbf{\color{#35bf28}+7.31\%}$
test_rb_sample[TensorDictPrioritizedReplayBuffer-ListStorage-None-4000] 9.1630ms 5.9727ms 167.4284 Ops/s 172.2936 Ops/s $\color{#d91a1a}-2.82\%$
test_rb_sample[TensorDictPrioritizedReplayBuffer-LazyMemmapStorage-None-10000] 1.7697ms 0.4266ms 2.3438 KOps/s 2.0227 KOps/s $\textbf{\color{#35bf28}+15.88\%}$
test_rb_sample[TensorDictPrioritizedReplayBuffer-LazyTensorStorage-None-10000] 0.8804ms 0.4404ms 2.2709 KOps/s 2.1116 KOps/s $\textbf{\color{#35bf28}+7.54\%}$
test_rb_iterate[TensorDictReplayBuffer-ListStorage-RandomSampler-4000] 6.1274ms 5.6989ms 175.4734 Ops/s 175.9968 Ops/s $\color{#d91a1a}-0.30\%$
test_rb_iterate[TensorDictReplayBuffer-LazyMemmapStorage-RandomSampler-10000] 1.7651ms 0.3456ms 2.8935 KOps/s 2.8238 KOps/s $\color{#35bf28}+2.47\%$
test_rb_iterate[TensorDictReplayBuffer-LazyTensorStorage-RandomSampler-10000] 0.5927ms 0.3252ms 3.0746 KOps/s 2.9453 KOps/s $\color{#35bf28}+4.39\%$
test_rb_iterate[TensorDictReplayBuffer-ListStorage-SamplerWithoutReplacement-4000] 6.2721ms 5.6846ms 175.9150 Ops/s 178.1180 Ops/s $\color{#d91a1a}-1.24\%$
test_rb_iterate[TensorDictReplayBuffer-LazyMemmapStorage-SamplerWithoutReplacement-10000] 1.0830ms 0.3421ms 2.9234 KOps/s 2.8407 KOps/s $\color{#35bf28}+2.91\%$
test_rb_iterate[TensorDictReplayBuffer-LazyTensorStorage-SamplerWithoutReplacement-10000] 0.5305ms 0.3296ms 3.0341 KOps/s 3.0048 KOps/s $\color{#35bf28}+0.97\%$
test_rb_iterate[TensorDictPrioritizedReplayBuffer-ListStorage-None-4000] 6.3108ms 5.8447ms 171.0942 Ops/s 173.0704 Ops/s $\color{#d91a1a}-1.14\%$
test_rb_iterate[TensorDictPrioritizedReplayBuffer-LazyMemmapStorage-None-10000] 0.8982ms 0.4552ms 2.1966 KOps/s 2.1271 KOps/s $\color{#35bf28}+3.27\%$
test_rb_iterate[TensorDictPrioritizedReplayBuffer-LazyTensorStorage-None-10000] 0.8624ms 0.4429ms 2.2579 KOps/s 2.1151 KOps/s $\textbf{\color{#35bf28}+6.75\%}$
test_rb_populate[TensorDictReplayBuffer-ListStorage-RandomSampler-400] 6.3317ms 4.9190ms 203.2933 Ops/s 58.2498 Ops/s $\textbf{\color{#35bf28}+249.00\%}$
test_rb_populate[TensorDictReplayBuffer-LazyMemmapStorage-RandomSampler-400] 3.5374ms 1.9249ms 519.4992 Ops/s 518.5107 Ops/s $\color{#35bf28}+0.19\%$
test_rb_populate[TensorDictReplayBuffer-LazyTensorStorage-RandomSampler-400] 2.2155ms 1.1402ms 877.0580 Ops/s 1.1548 KOps/s $\textbf{\color{#d91a1a}-24.05\%}$
test_rb_populate[TensorDictReplayBuffer-ListStorage-SamplerWithoutReplacement-400] 0.5480s 15.8220ms 63.2029 Ops/s 197.7694 Ops/s $\textbf{\color{#d91a1a}-68.04\%}$
test_rb_populate[TensorDictReplayBuffer-LazyMemmapStorage-SamplerWithoutReplacement-400] 8.2273ms 1.8713ms 534.3889 Ops/s 546.6288 Ops/s $\color{#d91a1a}-2.24\%$
test_rb_populate[TensorDictReplayBuffer-LazyTensorStorage-SamplerWithoutReplacement-400] 10.3433ms 1.1905ms 839.9835 Ops/s 1.1479 KOps/s $\textbf{\color{#d91a1a}-26.83\%}$
test_rb_populate[TensorDictPrioritizedReplayBuffer-ListStorage-None-400] 8.1628ms 5.1540ms 194.0250 Ops/s 193.1750 Ops/s $\color{#35bf28}+0.44\%$
test_rb_populate[TensorDictPrioritizedReplayBuffer-LazyMemmapStorage-None-400] 10.0636ms 2.0350ms 491.3889 Ops/s 526.9559 Ops/s $\textbf{\color{#d91a1a}-6.75\%}$
test_rb_populate[TensorDictPrioritizedReplayBuffer-LazyTensorStorage-None-400] 3.6901ms 1.0547ms 948.1580 Ops/s 945.2307 Ops/s $\color{#35bf28}+0.31\%$
test_rb_extend_sample[ReplayBuffer-LazyTensorStorage-RandomSampler-10000-10000-100-True] 36.9332ms 34.8268ms 28.7135 Ops/s 27.7726 Ops/s $\color{#35bf28}+3.39\%$
test_rb_extend_sample[ReplayBuffer-LazyTensorStorage-RandomSampler-10000-10000-100-False] 18.8214ms 17.3486ms 57.6416 Ops/s 33.7095 Ops/s $\textbf{\color{#35bf28}+71.00\%}$
test_rb_extend_sample[ReplayBuffer-LazyTensorStorage-RandomSampler-100000-10000-100-True] 39.7906ms 35.9939ms 27.7825 Ops/s 27.0374 Ops/s $\color{#35bf28}+2.76\%$
test_rb_extend_sample[ReplayBuffer-LazyTensorStorage-RandomSampler-100000-10000-100-False] 19.3915ms 17.8108ms 56.1456 Ops/s 53.9865 Ops/s $\color{#35bf28}+4.00\%$
test_rb_extend_sample[ReplayBuffer-LazyTensorStorage-RandomSampler-1000000-10000-100-True] 41.1202ms 37.9408ms 26.3568 Ops/s 25.8977 Ops/s $\color{#35bf28}+1.77\%$
test_rb_extend_sample[ReplayBuffer-LazyTensorStorage-RandomSampler-1000000-10000-100-False] 20.9063ms 19.1113ms 52.3251 Ops/s 48.8668 Ops/s $\textbf{\color{#35bf28}+7.08\%}$
test_storage_write_lazystack[50-img_shape0-small] 0.8570ms 0.2132ms 4.6912 KOps/s 4.4900 KOps/s $\color{#35bf28}+4.48\%$
test_storage_write_lazystack[100-img_shape1-atari] 1.6392ms 1.3626ms 733.9021 Ops/s 715.2888 Ops/s $\color{#35bf28}+2.60\%$
test_storage_write_lazystack[100-img_shape2-large_img] 2.5899ms 2.3823ms 419.7605 Ops/s 435.7678 Ops/s $\color{#d91a1a}-3.67\%$
test_storage_write_lazystack[200-img_shape3-large_batch] 3.3548ms 2.8991ms 344.9290 Ops/s 342.9313 Ops/s $\color{#35bf28}+0.58\%$
test_storage_write_contiguous[50-img_shape0-small] 0.2499ms 0.1303ms 7.6738 KOps/s 7.7549 KOps/s $\color{#d91a1a}-1.05\%$
test_storage_write_contiguous[100-img_shape1-atari] 0.6572ms 0.1847ms 5.4139 KOps/s 5.6820 KOps/s $\color{#d91a1a}-4.72\%$
test_storage_write_contiguous[100-img_shape2-large_img] 1.9063ms 1.7947ms 557.1954 Ops/s 572.8000 Ops/s $\color{#d91a1a}-2.72\%$
test_storage_write_contiguous[200-img_shape3-large_batch] 1.4412ms 1.3045ms 766.5770 Ops/s 763.9825 Ops/s $\color{#35bf28}+0.34\%$
test_collector_stack_then_write[50-img_shape0-small] 1.5235ms 1.0947ms 913.4836 Ops/s 925.0883 Ops/s $\color{#d91a1a}-1.25\%$
test_collector_stack_then_write[100-img_shape1-atari] 4.1252ms 3.5406ms 282.4344 Ops/s 287.7729 Ops/s $\color{#d91a1a}-1.86\%$
test_collector_stack_then_write[100-img_shape2-large_img] 5.9526ms 5.5733ms 179.4273 Ops/s 179.1551 Ops/s $\color{#35bf28}+0.15\%$
test_collector_stack_then_write[200-img_shape3-large_batch] 7.2342ms 6.8587ms 145.7997 Ops/s 147.2607 Ops/s $\color{#d91a1a}-0.99\%$
test_collector_lazystack_then_write[50-img_shape0-small] 0.7201ms 0.2883ms 3.4686 KOps/s 3.7144 KOps/s $\textbf{\color{#d91a1a}-6.62\%}$
test_collector_lazystack_then_write[100-img_shape1-atari] 2.0195ms 1.5203ms 657.7824 Ops/s 661.0849 Ops/s $\color{#d91a1a}-0.50\%$
test_collector_lazystack_then_write[100-img_shape2-large_img] 2.9942ms 2.5462ms 392.7378 Ops/s 413.5260 Ops/s $\textbf{\color{#d91a1a}-5.03\%}$
test_collector_lazystack_then_write[200-img_shape3-large_batch] 3.5947ms 3.1613ms 316.3238 Ops/s 319.6581 Ops/s $\color{#d91a1a}-1.04\%$
test_collector_without_rb[100-img_shape0-atari] 33.4064ms 32.6009ms 30.6740 Ops/s 31.0374 Ops/s $\color{#d91a1a}-1.17\%$
test_collector_without_rb[200-img_shape1-large_batch] 64.9357ms 64.2082ms 15.5743 Ops/s 15.7920 Ops/s $\color{#d91a1a}-1.38\%$
test_collector_with_rb[100-img_shape0-atari] 38.4280ms 37.3986ms 26.7390 Ops/s 27.3890 Ops/s $\color{#d91a1a}-2.37\%$
test_collector_with_rb[200-img_shape1-large_batch] 73.8607ms 72.9566ms 13.7068 Ops/s 13.8644 Ops/s $\color{#d91a1a}-1.14\%$

@github-actions
Copy link
Contributor

github-actions bot commented Feb 11, 2026 •
edited
Loading

Result of GPU Benchmark Tests

Expand to view detailed results
Name Max Mean Ops
test_tensor_to_bytestream_speed[pickle] 86.6365μs 85.2476μs 11.7305 KOps/s
test_tensor_to_bytestream_speed[torch.save] 0.1480ms 0.1442ms 6.9342 KOps/s
test_tensor_to_bytestream_speed[untyped_storage] 0.1130s 0.1129s 8.8607 Ops/s
test_tensor_to_bytestream_speed[numpy] 2.6771μs 2.6729μs 374.1262 KOps/s
test_tensor_to_bytestream_speed[safetensors] 42.7479μs 42.4377μs 23.5639 KOps/s
test_simple 0.7831s 0.7805s 1.2812 Ops/s
test_transformed 1.4994s 1.4063s 0.7111 Ops/s
test_serial 2.3934s 2.3089s 0.4331 Ops/s
test_parallel 1.9028s 1.8049s 0.5540 Ops/s
test_step_mdp_speed[True-True-True-True-True] 0.2475ms 41.6019μs 24.0374 KOps/s
test_step_mdp_speed[True-True-True-True-False] 0.4406ms 23.5203μs 42.5165 KOps/s
test_step_mdp_speed[True-True-True-False-True] 0.4437ms 23.2984μs 42.9214 KOps/s
test_step_mdp_speed[True-True-True-False-False] 39.5810μs 12.9244μs 77.3731 KOps/s
test_step_mdp_speed[True-True-False-True-True] 0.4646ms 44.7630μs 22.3399 KOps/s
test_step_mdp_speed[True-True-False-True-False] 0.4484ms 26.1716μs 38.2093 KOps/s
test_step_mdp_speed[True-True-False-False-True] 60.2610μs 26.1796μs 38.1977 KOps/s
test_step_mdp_speed[True-True-False-False-False] 0.4410ms 15.7871μs 63.3430 KOps/s
test_step_mdp_speed[True-False-True-True-True] 0.4666ms 47.0067μs 21.2736 KOps/s
test_step_mdp_speed[True-False-True-True-False] 62.9610μs 28.9417μs 34.5522 KOps/s
test_step_mdp_speed[True-False-True-False-True] 64.5910μs 25.9497μs 38.5361 KOps/s
test_step_mdp_speed[True-False-True-False-False] 0.4330ms 15.8664μs 63.0263 KOps/s
test_step_mdp_speed[True-False-False-True-True] 0.4657ms 49.4999μs 20.2021 KOps/s
test_step_mdp_speed[True-False-False-True-False] 64.2110μs 31.5034μs 31.7426 KOps/s
test_step_mdp_speed[True-False-False-False-True] 0.4525ms 29.1838μs 34.2656 KOps/s
test_step_mdp_speed[True-False-False-False-False] 0.4437ms 18.3913μs 54.3734 KOps/s
test_step_mdp_speed[False-True-True-True-True] 0.4681ms 47.5304μs 21.0391 KOps/s
test_step_mdp_speed[False-True-True-True-False] 63.6710μs 29.0510μs 34.4223 KOps/s
test_step_mdp_speed[False-True-True-False-True] 2.4085ms 30.6964μs 32.5771 KOps/s
test_step_mdp_speed[False-True-True-False-False] 51.4510μs 17.4584μs 57.2789 KOps/s
test_step_mdp_speed[False-True-False-True-True] 0.4680ms 50.0099μs 19.9961 KOps/s
test_step_mdp_speed[False-True-False-True-False] 0.4467ms 31.6361μs 31.6094 KOps/s
test_step_mdp_speed[False-True-False-False-True] 0.4480ms 32.1835μs 31.0719 KOps/s
test_step_mdp_speed[False-True-False-False-False] 46.6910μs 20.0630μs 49.8431 KOps/s
test_step_mdp_speed[False-False-True-True-True] 0.4704ms 52.6648μs 18.9880 KOps/s
test_step_mdp_speed[False-False-True-True-False] 0.4458ms 33.9682μs 29.4393 KOps/s
test_step_mdp_speed[False-False-True-False-True] 63.5310μs 32.1977μs 31.0582 KOps/s
test_step_mdp_speed[False-False-True-False-False] 50.2310μs 19.9584μs 50.1043 KOps/s
test_step_mdp_speed[False-False-False-True-True] 0.4715ms 53.9240μs 18.5446 KOps/s
test_step_mdp_speed[False-False-False-True-False] 0.4466ms 36.2060μs 27.6197 KOps/s
test_step_mdp_speed[False-False-False-False-True] 0.4577ms 34.2263μs 29.2173 KOps/s
test_step_mdp_speed[False-False-False-False-False] 61.9920μs 22.2422μs 44.9596 KOps/s
test_non_tensor_env_rollout_speed[1000-single-True] 0.8458s 0.7409s 1.3498 Ops/s
test_non_tensor_env_rollout_speed[1000-single-False] 0.7050s 0.6070s 1.6475 Ops/s
test_non_tensor_env_rollout_speed[1000-serial-no-buffers-True] 1.7169s 1.6363s 0.6111 Ops/s
test_non_tensor_env_rollout_speed[1000-serial-no-buffers-False] 1.4902s 1.4128s 0.7078 Ops/s
test_non_tensor_env_rollout_speed[1000-serial-buffers-True] 1.9628s 1.8824s 0.5312 Ops/s
test_non_tensor_env_rollout_speed[1000-serial-buffers-False] 1.7422s 1.6624s 0.6015 Ops/s
test_non_tensor_env_rollout_speed[1000-parallel-no-buffers-True] 4.7562s 4.6613s 0.2145 Ops/s
test_non_tensor_env_rollout_speed[1000-parallel-no-buffers-False] 4.4976s 4.4267s 0.2259 Ops/s
test_non_tensor_env_rollout_speed[1000-parallel-buffers-True] 2.0455s 1.8909s 0.5288 Ops/s
test_non_tensor_env_rollout_speed[1000-parallel-buffers-False] 1.6601s 1.5749s 0.6350 Ops/s
test_values[generalized_advantage_estimate-True-True] 20.9778ms 20.4722ms 48.8466 Ops/s
test_values[vec_generalized_advantage_estimate-True-True] 0.1324s 3.5749ms 279.7303 Ops/s
test_values[td0_return_estimate-False-False] 0.1088ms 83.5936μs 11.9626 KOps/s
test_values[td1_return_estimate-False-False] 49.2191ms 48.5418ms 20.6008 Ops/s
test_values[vec_td1_return_estimate-False-False] 1.3408ms 1.0943ms 913.7904 Ops/s
test_values[td_lambda_return_estimate-True-False] 80.2045ms 79.4248ms 12.5905 Ops/s
test_values[vec_td_lambda_return_estimate-True-False] 1.3180ms 1.0868ms 920.1636 Ops/s
test_gae_speed[generalized_advantage_estimate-False-1-512] 20.9356ms 20.5361ms 48.6947 Ops/s
test_gae_speed[vec_generalized_advantage_estimate-True-1-512] 1.0232ms 0.7499ms 1.3336 KOps/s
test_gae_speed[vec_generalized_advantage_estimate-False-1-512] 1.1000ms 0.6797ms 1.4712 KOps/s
test_gae_speed[vec_generalized_advantage_estimate-True-32-512] 1.5336ms 1.4949ms 668.9619 Ops/s
test_gae_speed[vec_generalized_advantage_estimate-False-32-512] 0.8398ms 0.6961ms 1.4366 KOps/s
test_dqn_speed[False-None] 1.7086ms 1.5513ms 644.6280 Ops/s
test_dqn_speed[False-backward] 2.2704ms 2.1733ms 460.1283 Ops/s
test_dqn_speed[True-None] 0.6362ms 0.5646ms 1.7712 KOps/s
test_dqn_speed[True-backward] 1.2490ms 1.2069ms 828.5598 Ops/s
test_dqn_speed[reduce-overhead-None] 0.6861ms 0.5867ms 1.7045 KOps/s
test_ddpg_speed[False-None] 3.3278ms 2.9100ms 343.6428 Ops/s
test_ddpg_speed[False-backward] 4.7190ms 4.3125ms 231.8836 Ops/s
test_ddpg_speed[True-None] 1.6280ms 1.3122ms 762.0532 Ops/s
test_ddpg_speed[True-backward] 2.5485ms 2.5087ms 398.6147 Ops/s
test_ddpg_speed[reduce-overhead-None] 1.4272ms 1.3352ms 748.9489 Ops/s
test_sac_speed[False-None] 8.6696ms 8.2568ms 121.1122 Ops/s
test_sac_speed[False-backward] 12.0267ms 11.5123ms 86.8639 Ops/s
test_sac_speed[True-None] 1.9252ms 1.8064ms 553.5851 Ops/s
test_sac_speed[True-backward] 3.7055ms 3.5727ms 279.8996 Ops/s
test_sac_speed[reduce-overhead-None] 19.6880ms 11.1851ms 89.4048 Ops/s
test_redq_deprec_speed[False-None] 10.3500ms 9.4376ms 105.9597 Ops/s
test_redq_deprec_speed[False-backward] 13.0766ms 12.6965ms 78.7617 Ops/s
test_redq_deprec_speed[True-None] 2.6864ms 2.5372ms 394.1286 Ops/s
test_redq_deprec_speed[True-backward] 4.5300ms 4.3171ms 231.6383 Ops/s
test_redq_deprec_speed[reduce-overhead-None] 16.2904ms 9.9656ms 100.3448 Ops/s
test_td3_speed[False-None] 8.5879ms 8.2022ms 121.9182 Ops/s
test_td3_speed[False-backward] 11.3093ms 10.8223ms 92.4016 Ops/s
test_td3_speed[True-None] 1.6613ms 1.6322ms 612.6703 Ops/s
test_td3_speed[True-backward] 3.2940ms 3.2327ms 309.3342 Ops/s
test_td3_speed[reduce-overhead-None] 88.5670ms 25.1235ms 39.8033 Ops/s
test_cql_speed[False-None] 17.5286ms 17.2692ms 57.9066 Ops/s
test_cql_speed[False-backward] 23.8407ms 22.8886ms 43.6898 Ops/s
test_cql_speed[True-None] 3.3187ms 3.2232ms 310.2515 Ops/s
test_cql_speed[True-backward] 5.7927ms 5.4645ms 183.0003 Ops/s
test_cql_speed[reduce-overhead-None] 19.4139ms 12.1051ms 82.6097 Ops/s
test_a2c_speed[False-None] 4.1353ms 3.2516ms 307.5404 Ops/s
test_a2c_speed[False-backward] 6.5136ms 6.4079ms 156.0564 Ops/s
test_a2c_speed[True-None] 1.3880ms 1.3336ms 749.8354 Ops/s
test_a2c_speed[True-backward] 3.1044ms 3.0638ms 326.3971 Ops/s
test_a2c_speed[reduce-overhead-None] 1.0737ms 0.9811ms 1.0193 KOps/s
test_ppo_speed[False-None] 4.0687ms 3.8840ms 257.4689 Ops/s
test_ppo_speed[False-backward] 7.6248ms 7.1914ms 139.0552 Ops/s
test_ppo_speed[True-None] 1.4966ms 1.4193ms 704.5771 Ops/s
test_ppo_speed[True-backward] 3.3428ms 3.2474ms 307.9410 Ops/s
test_ppo_speed[reduce-overhead-None] 1.1519ms 1.0545ms 948.2793 Ops/s
test_reinforce_speed[False-None] 2.3768ms 2.2891ms 436.8506 Ops/s
test_reinforce_speed[False-backward] 3.9125ms 3.4692ms 288.2543 Ops/s
test_reinforce_speed[True-None] 1.3843ms 1.2873ms 776.7976 Ops/s
test_reinforce_speed[True-backward] 3.1422ms 3.0202ms 331.0988 Ops/s
test_reinforce_speed[reduce-overhead-None] 0.4433s 10.6577ms 93.8287 Ops/s
test_iql_speed[False-None] 10.0643ms 9.4480ms 105.8424 Ops/s
test_iql_speed[False-backward] 13.8973ms 13.4614ms 74.2865 Ops/s
test_iql_speed[True-None] 2.4128ms 2.1702ms 460.7824 Ops/s
test_iql_speed[True-backward] 4.9784ms 4.8395ms 206.6310 Ops/s
test_iql_speed[reduce-overhead-None] 18.3073ms 10.7368ms 93.1378 Ops/s
test_rb_sample[TensorDictReplayBuffer-ListStorage-RandomSampler-4000] 6.3601ms 5.9721ms 167.4458 Ops/s
test_rb_sample[TensorDictReplayBuffer-LazyMemmapStorage-RandomSampler-10000] 1.0966ms 0.3529ms 2.8334 KOps/s
test_rb_sample[TensorDictReplayBuffer-LazyTensorStorage-RandomSampler-10000] 0.5244ms 0.2692ms 3.7141 KOps/s
test_rb_sample[TensorDictReplayBuffer-ListStorage-SamplerWithoutReplacement-4000] 6.0441ms 5.7072ms 175.2174 Ops/s
test_rb_sample[TensorDictReplayBuffer-LazyMemmapStorage-SamplerWithoutReplacement-10000] 0.7938ms 0.3630ms 2.7548 KOps/s
test_rb_sample[TensorDictReplayBuffer-LazyTensorStorage-SamplerWithoutReplacement-10000] 0.5838ms 0.3488ms 2.8667 KOps/s
test_rb_sample[TensorDictReplayBuffer-LazyMemmapStorage-sampler6-10000] 1.6358ms 1.4215ms 703.5069 Ops/s
test_rb_sample[TensorDictReplayBuffer-LazyTensorStorage-sampler7-10000] 1.6463ms 1.3408ms 745.7967 Ops/s
test_rb_sample[TensorDictPrioritizedReplayBuffer-ListStorage-None-4000] 9.8676ms 6.1273ms 163.2039 Ops/s
test_rb_sample[TensorDictPrioritizedReplayBuffer-LazyMemmapStorage-None-10000] 1.6085ms 0.4832ms 2.0697 KOps/s
test_rb_sample[TensorDictPrioritizedReplayBuffer-LazyTensorStorage-None-10000] 0.7943ms 0.4617ms 2.1657 KOps/s
test_rb_iterate[TensorDictReplayBuffer-ListStorage-RandomSampler-4000] 5.9205ms 5.8033ms 172.3146 Ops/s
test_rb_iterate[TensorDictReplayBuffer-LazyMemmapStorage-RandomSampler-10000] 0.8920ms 0.3774ms 2.6495 KOps/s
test_rb_iterate[TensorDictReplayBuffer-LazyTensorStorage-RandomSampler-10000] 0.5750ms 0.3294ms 3.0359 KOps/s
test_rb_iterate[TensorDictReplayBuffer-ListStorage-SamplerWithoutReplacement-4000] 6.2861ms 5.8107ms 172.0950 Ops/s
test_rb_iterate[TensorDictReplayBuffer-LazyMemmapStorage-SamplerWithoutReplacement-10000] 1.0503ms 0.3594ms 2.7820 KOps/s
test_rb_iterate[TensorDictReplayBuffer-LazyTensorStorage-SamplerWithoutReplacement-10000] 0.6045ms 0.3479ms 2.8748 KOps/s
test_rb_iterate[TensorDictPrioritizedReplayBuffer-ListStorage-None-4000] 6.2421ms 6.0641ms 164.9044 Ops/s
test_rb_iterate[TensorDictPrioritizedReplayBuffer-LazyMemmapStorage-None-10000] 1.1504ms 0.4482ms 2.2314 KOps/s
test_rb_iterate[TensorDictPrioritizedReplayBuffer-LazyTensorStorage-None-10000] 0.7529ms 0.4804ms 2.0814 KOps/s
test_rb_populate[TensorDictReplayBuffer-ListStorage-RandomSampler-400] 6.4582ms 4.9883ms 200.4703 Ops/s
test_rb_populate[TensorDictReplayBuffer-LazyMemmapStorage-RandomSampler-400] 9.3006ms 2.3202ms 430.9890 Ops/s
test_rb_populate[TensorDictReplayBuffer-LazyTensorStorage-RandomSampler-400] 1.0940ms 0.9608ms 1.0408 KOps/s
test_rb_populate[TensorDictReplayBuffer-ListStorage-SamplerWithoutReplacement-400] 0.5824s 16.6990ms 59.8838 Ops/s
test_rb_populate[TensorDictReplayBuffer-LazyMemmapStorage-SamplerWithoutReplacement-400] 10.8254ms 2.0052ms 498.7150 Ops/s
test_rb_populate[TensorDictReplayBuffer-LazyTensorStorage-SamplerWithoutReplacement-400] 7.6982ms 1.3006ms 768.8834 Ops/s
test_rb_populate[TensorDictPrioritizedReplayBuffer-ListStorage-None-400] 7.0809ms 5.3333ms 187.5010 Ops/s
test_rb_populate[TensorDictPrioritizedReplayBuffer-LazyMemmapStorage-None-400] 4.2823ms 1.9875ms 503.1343 Ops/s
test_rb_populate[TensorDictPrioritizedReplayBuffer-LazyTensorStorage-None-400] 1.4354ms 1.1251ms 888.7927 Ops/s
test_rb_extend_sample[ReplayBuffer-LazyTensorStorage-RandomSampler-10000-10000-100-True] 38.5477ms 36.0721ms 27.7222 Ops/s
test_rb_extend_sample[ReplayBuffer-LazyTensorStorage-RandomSampler-10000-10000-100-False] 20.5013ms 18.7562ms 53.3158 Ops/s
test_rb_extend_sample[ReplayBuffer-LazyTensorStorage-RandomSampler-100000-10000-100-True] 41.6152ms 37.7767ms 26.4713 Ops/s
test_rb_extend_sample[ReplayBuffer-LazyTensorStorage-RandomSampler-100000-10000-100-False] 20.7458ms 19.2260ms 52.0128 Ops/s
test_rb_extend_sample[ReplayBuffer-LazyTensorStorage-RandomSampler-1000000-10000-100-True] 42.0768ms 39.4308ms 25.3609 Ops/s
test_rb_extend_sample[ReplayBuffer-LazyTensorStorage-RandomSampler-1000000-10000-100-False] 21.5240ms 20.2435ms 49.3987 Ops/s
test_storage_write_lazystack[50-img_shape0-small] 0.7610ms 0.2182ms 4.5827 KOps/s
test_storage_write_lazystack[100-img_shape1-atari] 1.6645ms 1.3759ms 726.7832 Ops/s
test_storage_write_lazystack[100-img_shape2-large_img] 2.8643ms 2.3688ms 422.1472 Ops/s
test_storage_write_lazystack[200-img_shape3-large_batch] 3.3445ms 2.9426ms 339.8355 Ops/s
test_storage_write_contiguous[50-img_shape0-small] 0.2579ms 0.1639ms 6.1026 KOps/s
test_storage_write_contiguous[100-img_shape1-atari] 0.3610ms 0.2374ms 4.2130 KOps/s
test_storage_write_contiguous[100-img_shape2-large_img] 2.1056ms 1.8424ms 542.7796 Ops/s
test_storage_write_contiguous[200-img_shape3-large_batch] 1.7305ms 1.3271ms 753.5342 Ops/s
test_collector_stack_then_write[50-img_shape0-small] 1.3157ms 1.1628ms 860.0244 Ops/s
test_collector_stack_then_write[100-img_shape1-atari] 3.7805ms 3.6056ms 277.3499 Ops/s
test_collector_stack_then_write[100-img_shape2-large_img] 6.2311ms 5.9252ms 168.7694 Ops/s
test_collector_stack_then_write[200-img_shape3-large_batch] 7.7113ms 7.3865ms 135.3826 Ops/s
test_collector_lazystack_then_write[50-img_shape0-small] 0.5120s 0.5187ms 1.9280 KOps/s
test_collector_lazystack_then_write[100-img_shape1-atari] 1.8524ms 1.5410ms 648.9474 Ops/s
test_collector_lazystack_then_write[100-img_shape2-large_img] 2.9943ms 2.4806ms 403.1227 Ops/s
test_collector_lazystack_then_write[200-img_shape3-large_batch] 3.6211ms 3.2526ms 307.4426 Ops/s
test_collector_without_rb[100-img_shape0-atari] 35.1498ms 33.8025ms 29.5836 Ops/s
test_collector_without_rb[200-img_shape1-large_batch] 68.0926ms 66.3700ms 15.0671 Ops/s
test_collector_with_rb[100-img_shape0-atari] 38.5722ms 37.9796ms 26.3300 Ops/s
test_collector_with_rb[200-img_shape1-large_batch] 75.8679ms 74.2791ms 13.4627 Ops/s
test_collector_without_rb_cuda[100-img_shape0-atari] 56.7247ms 55.0668ms 18.1598 Ops/s
test_collector_without_rb_cuda[200-img_shape1-large_batch] 0.1136s 0.1107s 9.0344 Ops/s
test_collector_with_rb_cuda[100-img_shape0-atari] 59.4896ms 57.3597ms 17.4338 Ops/s
test_collector_with_rb_cuda[200-img_shape1-large_batch] 0.1171s 0.1147s 8.7206 Ops/s

This was referenced Feb 12, 2026
Open
Open
@vmoens
Update
50fb5ba 
[ghstack-poisoned]
@vmoens vmoens mentioned this pull request Feb 16, 2026
Open
@vmoens
Update
19ac971 
[ghstack-poisoned]
@vmoens
Update
afcd83c 
[ghstack-poisoned]
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Reviewers

No reviews

Assignees

No one assigned

Labels

CLA Signed This label is managed by the Facebook bot. Authors need to sign the CLA before a PR can be reviewed. Feature New feature Modules

Projects

None yet

Milestone

No milestone

Development

Successfully merging this pull request may close these issues.

1 participant

@vmoens