Skip to content

[Minor] Support init lazy storages with add #2028

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom
Merged
merged 1 commit into from
Mar 20, 2024
Merged

[Minor] Support init lazy storages with add #2028

merged 1 commit into from
Mar 20, 2024

Conversation

@vmoens
Copy link
Collaborator

@vmoens vmoens commented Mar 20, 2024

No description provided.

@pytorch-bot
Copy link

pytorch-bot bot commented Mar 20, 2024 •
edited
Loading

🔗 Helpful Links

🧪 See artifacts and rendered test results at hud.pytorch.org/pr/pytorch/rl/2028

Note: Links to docs will display an error until the docs builds have been completed.

❌ 2 New Failures

As of commit 1834af5 with merge base d937478 (image):

NEW FAILURES - The following jobs have failed:

This comment was automatically generated by Dr. CI and updates every 15 minutes.

@facebook-github-bot facebook-github-bot added the CLA Signed This label is managed by the Facebook bot. Authors need to sign the CLA before a PR can be reviewed. label Mar 20, 2024
@vmoens vmoens added the bug Something isn't working label Mar 20, 2024
@vmoens vmoens merged commit 660d827 into main Mar 20, 2024
@github-actions
Copy link

github-actions bot commented Mar 20, 2024

$\color{#D29922}\textsf{\Large⚠\kern{0.2cm}\normalsize Warning}$ Result of GPU Benchmark Tests

Total Benchmarks: 94. Improved: $\large\color{#35bf28}6$. Worsened: $\large\color{#d91a1a}3$.

Expand to view detailed results
Name Max Mean Ops Ops on Repo HEAD Change
test_single 0.1053s 0.1021s 9.7915 Ops/s 9.2964 Ops/s $\textbf{\color{#35bf28}+5.33\%}$
test_sync 93.5976ms 90.9978ms 10.9893 Ops/s 11.0253 Ops/s $\color{#d91a1a}-0.33\%$
test_async 0.1729s 88.3040ms 11.3245 Ops/s 11.9900 Ops/s $\textbf{\color{#d91a1a}-5.55\%}$
test_single_pixels 0.1122s 0.1120s 8.9248 Ops/s 8.9446 Ops/s $\color{#d91a1a}-0.22\%$
test_sync_pixels 67.6900ms 66.3041ms 15.0820 Ops/s 15.0134 Ops/s $\color{#35bf28}+0.46\%$
test_async_pixels 0.1207s 56.3986ms 17.7310 Ops/s 18.1328 Ops/s $\color{#d91a1a}-2.22\%$
test_simple 0.6651s 0.6609s 1.5131 Ops/s 1.4860 Ops/s $\color{#35bf28}+1.83\%$
test_transformed 0.8903s 0.8819s 1.1339 Ops/s 1.1377 Ops/s $\color{#d91a1a}-0.33\%$
test_serial 2.1492s 2.0902s 0.4784 Ops/s 0.4701 Ops/s $\color{#35bf28}+1.78\%$
test_parallel 1.8523s 1.7914s 0.5582 Ops/s 0.5510 Ops/s $\color{#35bf28}+1.32\%$
test_step_mdp_speed[True-True-True-True-True] 97.5220μs 32.7076μs 30.5740 KOps/s 30.0362 KOps/s $\color{#35bf28}+1.79\%$
test_step_mdp_speed[True-True-True-True-False] 35.1110μs 19.2559μs 51.9321 KOps/s 50.9377 KOps/s $\color{#35bf28}+1.95\%$
test_step_mdp_speed[True-True-True-False-True] 46.7210μs 18.6393μs 53.6502 KOps/s 53.5082 KOps/s $\color{#35bf28}+0.27\%$
test_step_mdp_speed[True-True-True-False-False] 25.6310μs 10.9653μs 91.1970 KOps/s 90.4395 KOps/s $\color{#35bf28}+0.84\%$
test_step_mdp_speed[True-True-False-True-True] 61.2720μs 34.2444μs 29.2018 KOps/s 28.4868 KOps/s $\color{#35bf28}+2.51\%$
test_step_mdp_speed[True-True-False-True-False] 38.1800μs 20.9428μs 47.7492 KOps/s 47.4651 KOps/s $\color{#35bf28}+0.60\%$
test_step_mdp_speed[True-True-False-False-True] 38.0910μs 20.3085μs 49.2405 KOps/s 49.6583 KOps/s $\color{#d91a1a}-0.84\%$
test_step_mdp_speed[True-True-False-False-False] 29.0310μs 12.7989μs 78.1316 KOps/s 77.0198 KOps/s $\color{#35bf28}+1.44\%$
test_step_mdp_speed[True-False-True-True-True] 66.9810μs 35.6477μs 28.0523 KOps/s 27.1821 KOps/s $\color{#35bf28}+3.20\%$
test_step_mdp_speed[True-False-True-True-False] 43.0610μs 23.3413μs 42.8425 KOps/s 43.3616 KOps/s $\color{#d91a1a}-1.20\%$
test_step_mdp_speed[True-False-True-False-True] 46.4110μs 20.0938μs 49.7667 KOps/s 49.4829 KOps/s $\color{#35bf28}+0.57\%$
test_step_mdp_speed[True-False-True-False-False] 40.7910μs 12.8581μs 77.7719 KOps/s 76.2965 KOps/s $\color{#35bf28}+1.93\%$
test_step_mdp_speed[True-False-False-True-True] 55.6110μs 37.6118μs 26.5874 KOps/s 26.3358 KOps/s $\color{#35bf28}+0.96\%$
test_step_mdp_speed[True-False-False-True-False] 46.1010μs 24.5312μs 40.7643 KOps/s 40.2231 KOps/s $\color{#35bf28}+1.35\%$
test_step_mdp_speed[True-False-False-False-True] 45.2410μs 21.9876μs 45.4802 KOps/s 44.9783 KOps/s $\color{#35bf28}+1.12\%$
test_step_mdp_speed[True-False-False-False-False] 38.0810μs 14.4978μs 68.9761 KOps/s 67.0961 KOps/s $\color{#35bf28}+2.80\%$
test_step_mdp_speed[False-True-True-True-True] 68.3710μs 36.1347μs 27.6742 KOps/s 27.2559 KOps/s $\color{#35bf28}+1.53\%$
test_step_mdp_speed[False-True-True-True-False] 38.8510μs 22.9433μs 43.5857 KOps/s 43.0000 KOps/s $\color{#35bf28}+1.36\%$
test_step_mdp_speed[False-True-True-False-True] 39.8810μs 23.9423μs 41.7672 KOps/s 41.4558 KOps/s $\color{#35bf28}+0.75\%$
test_step_mdp_speed[False-True-True-False-False] 35.6210μs 14.5396μs 68.7779 KOps/s 67.1659 KOps/s $\color{#35bf28}+2.40\%$
test_step_mdp_speed[False-True-False-True-True] 71.6820μs 38.2133μs 26.1689 KOps/s 25.6260 KOps/s $\color{#35bf28}+2.12\%$
test_step_mdp_speed[False-True-False-True-False] 51.9410μs 24.9085μs 40.1469 KOps/s 39.1906 KOps/s $\color{#35bf28}+2.44\%$
test_step_mdp_speed[False-True-False-False-True] 62.7010μs 25.8395μs 38.7005 KOps/s 38.7882 KOps/s $\color{#d91a1a}-0.23\%$
test_step_mdp_speed[False-True-False-False-False] 41.7210μs 16.4943μs 60.6271 KOps/s 60.0153 KOps/s $\color{#35bf28}+1.02\%$
test_step_mdp_speed[False-False-True-True-True] 61.0720μs 40.1686μs 24.8950 KOps/s 24.5517 KOps/s $\color{#35bf28}+1.40\%$
test_step_mdp_speed[False-False-True-True-False] 45.9310μs 26.8932μs 37.1842 KOps/s 36.8747 KOps/s $\color{#35bf28}+0.84\%$
test_step_mdp_speed[False-False-True-False-True] 63.0210μs 25.6925μs 38.9219 KOps/s 38.5148 KOps/s $\color{#35bf28}+1.06\%$
test_step_mdp_speed[False-False-True-False-False] 41.3910μs 16.2077μs 61.6991 KOps/s 60.4188 KOps/s $\color{#35bf28}+2.12\%$
test_step_mdp_speed[False-False-False-True-True] 67.2220μs 40.9712μs 24.4074 KOps/s 23.9938 KOps/s $\color{#35bf28}+1.72\%$
test_step_mdp_speed[False-False-False-True-False] 47.6620μs 28.3304μs 35.2978 KOps/s 34.6776 KOps/s $\color{#35bf28}+1.79\%$
test_step_mdp_speed[False-False-False-False-True] 48.9710μs 27.1949μs 36.7716 KOps/s 36.6049 KOps/s $\color{#35bf28}+0.46\%$
test_step_mdp_speed[False-False-False-False-False] 36.0620μs 18.0546μs 55.3875 KOps/s 55.3115 KOps/s $\color{#35bf28}+0.14\%$
test_values[generalized_advantage_estimate-True-True] 26.0550ms 24.3820ms 41.0139 Ops/s 40.8530 Ops/s $\color{#35bf28}+0.39\%$
test_values[vec_generalized_advantage_estimate-True-True] 87.9208ms 3.3136ms 301.7859 Ops/s 311.5456 Ops/s $\color{#d91a1a}-3.13\%$
test_values[td0_return_estimate-False-False] 89.4420μs 62.5741μs 15.9810 KOps/s 15.8097 KOps/s $\color{#35bf28}+1.08\%$
test_values[td1_return_estimate-False-False] 55.6089ms 52.1177ms 19.1873 Ops/s 19.0567 Ops/s $\color{#35bf28}+0.69\%$
test_values[vec_td1_return_estimate-False-False] 2.0242ms 1.7458ms 572.8084 Ops/s 570.4174 Ops/s $\color{#35bf28}+0.42\%$
test_values[td_lambda_return_estimate-True-False] 88.1434ms 84.3838ms 11.8506 Ops/s 11.9780 Ops/s $\color{#d91a1a}-1.06\%$
test_values[vec_td_lambda_return_estimate-True-False] 2.0793ms 1.7492ms 571.6797 Ops/s 570.5807 Ops/s $\color{#35bf28}+0.19\%$
test_gae_speed[generalized_advantage_estimate-False-1-512] 24.5920ms 23.5481ms 42.4663 Ops/s 43.2298 Ops/s $\color{#d91a1a}-1.77\%$
test_gae_speed[vec_generalized_advantage_estimate-True-1-512] 0.9092ms 0.6899ms 1.4495 KOps/s 1.4325 KOps/s $\color{#35bf28}+1.18\%$
test_gae_speed[vec_generalized_advantage_estimate-False-1-512] 0.7063ms 0.6388ms 1.5654 KOps/s 1.5611 KOps/s $\color{#35bf28}+0.27\%$
test_gae_speed[vec_generalized_advantage_estimate-True-32-512] 1.5398ms 1.4444ms 692.3494 Ops/s 690.4731 Ops/s $\color{#35bf28}+0.27\%$
test_gae_speed[vec_generalized_advantage_estimate-False-32-512] 0.9125ms 0.6793ms 1.4720 KOps/s 1.5046 KOps/s $\color{#d91a1a}-2.17\%$
test_dqn_speed 8.1051ms 1.4253ms 701.6259 Ops/s 688.4784 Ops/s $\color{#35bf28}+1.91\%$
test_ddpg_speed 2.9685ms 2.6989ms 370.5208 Ops/s 369.7314 Ops/s $\color{#35bf28}+0.21\%$
test_sac_speed 8.5362ms 8.0569ms 124.1172 Ops/s 125.2358 Ops/s $\color{#d91a1a}-0.89\%$
test_redq_speed 11.2146ms 10.1633ms 98.3931 Ops/s 98.7493 Ops/s $\color{#d91a1a}-0.36\%$
test_redq_deprec_speed 11.7068ms 11.0003ms 90.9065 Ops/s 89.0745 Ops/s $\color{#35bf28}+2.06\%$
test_td3_speed 8.2603ms 8.1665ms 122.4516 Ops/s 126.7982 Ops/s $\color{#d91a1a}-3.43\%$
test_cql_speed 26.2977ms 25.1414ms 39.7750 Ops/s 40.1239 Ops/s $\color{#d91a1a}-0.87\%$
test_a2c_speed 6.2426ms 5.5145ms 181.3395 Ops/s 182.3900 Ops/s $\color{#d91a1a}-0.58\%$
test_ppo_speed 6.1855ms 5.9345ms 168.5048 Ops/s 170.6982 Ops/s $\color{#d91a1a}-1.28\%$
test_reinforce_speed 4.6334ms 4.4149ms 226.5068 Ops/s 222.5017 Ops/s $\color{#35bf28}+1.80\%$
test_iql_speed 19.6036ms 18.7857ms 53.2320 Ops/s 52.6956 Ops/s $\color{#35bf28}+1.02\%$
test_rb_sample[TensorDictReplayBuffer-ListStorage-RandomSampler-4000] 2.9457ms 2.8693ms 348.5217 Ops/s 343.8481 Ops/s $\color{#35bf28}+1.36\%$
test_rb_sample[TensorDictReplayBuffer-LazyMemmapStorage-RandomSampler-10000] 1.2998ms 0.5393ms 1.8544 KOps/s 1.8590 KOps/s $\color{#d91a1a}-0.25\%$
test_rb_sample[TensorDictReplayBuffer-LazyTensorStorage-RandomSampler-10000] 0.6743ms 0.5116ms 1.9548 KOps/s 1.9180 KOps/s $\color{#35bf28}+1.92\%$
test_rb_sample[TensorDictReplayBuffer-ListStorage-SamplerWithoutReplacement-4000] 3.0697ms 2.8780ms 347.4680 Ops/s 341.9652 Ops/s $\color{#35bf28}+1.61\%$
test_rb_sample[TensorDictReplayBuffer-LazyMemmapStorage-SamplerWithoutReplacement-10000] 1.5222ms 0.5312ms 1.8825 KOps/s 1.8678 KOps/s $\color{#35bf28}+0.78\%$
test_rb_sample[TensorDictReplayBuffer-LazyTensorStorage-SamplerWithoutReplacement-10000] 0.6640ms 0.5068ms 1.9730 KOps/s 1.9538 KOps/s $\color{#35bf28}+0.98\%$
test_rb_sample[TensorDictReplayBuffer-LazyMemmapStorage-sampler6-10000] 0.1099s 1.7023ms 587.4425 Ops/s 657.1787 Ops/s $\textbf{\color{#d91a1a}-10.61\%}$
test_rb_sample[TensorDictReplayBuffer-LazyTensorStorage-sampler7-10000] 1.6169ms 1.4340ms 697.3487 Ops/s 682.9013 Ops/s $\color{#35bf28}+2.12\%$
test_rb_sample[TensorDictPrioritizedReplayBuffer-ListStorage-None-4000] 3.1519ms 3.0349ms 329.4952 Ops/s 329.8694 Ops/s $\color{#d91a1a}-0.11\%$
test_rb_sample[TensorDictPrioritizedReplayBuffer-LazyMemmapStorage-None-10000] 0.8100ms 0.6658ms 1.5019 KOps/s 1.5161 KOps/s $\color{#d91a1a}-0.93\%$
test_rb_sample[TensorDictPrioritizedReplayBuffer-LazyTensorStorage-None-10000] 4.4552ms 0.6476ms 1.5443 KOps/s 1.5542 KOps/s $\color{#d91a1a}-0.64\%$
test_rb_iterate[TensorDictReplayBuffer-ListStorage-RandomSampler-4000] 3.0151ms 2.8807ms 347.1381 Ops/s 342.4833 Ops/s $\color{#35bf28}+1.36\%$
test_rb_iterate[TensorDictReplayBuffer-LazyMemmapStorage-RandomSampler-10000] 0.6700ms 0.5357ms 1.8665 KOps/s 1.8502 KOps/s $\color{#35bf28}+0.88\%$
test_rb_iterate[TensorDictReplayBuffer-LazyTensorStorage-RandomSampler-10000] 4.4158ms 0.5228ms 1.9128 KOps/s 1.9302 KOps/s $\color{#d91a1a}-0.90\%$
test_rb_iterate[TensorDictReplayBuffer-ListStorage-SamplerWithoutReplacement-4000] 3.0554ms 2.8952ms 345.4014 Ops/s 341.7876 Ops/s $\color{#35bf28}+1.06\%$
test_rb_iterate[TensorDictReplayBuffer-LazyMemmapStorage-SamplerWithoutReplacement-10000] 0.6476ms 0.5310ms 1.8833 KOps/s 1.8844 KOps/s $\color{#d91a1a}-0.06\%$
test_rb_iterate[TensorDictReplayBuffer-LazyTensorStorage-SamplerWithoutReplacement-10000] 0.6737ms 0.5078ms 1.9692 KOps/s 1.5200 KOps/s $\textbf{\color{#35bf28}+29.56\%}$
test_rb_iterate[TensorDictPrioritizedReplayBuffer-ListStorage-None-4000] 3.1649ms 3.0227ms 330.8327 Ops/s 328.8005 Ops/s $\color{#35bf28}+0.62\%$
test_rb_iterate[TensorDictPrioritizedReplayBuffer-LazyMemmapStorage-None-10000] 0.8344ms 0.6731ms 1.4857 KOps/s 1.5044 KOps/s $\color{#d91a1a}-1.24\%$
test_rb_iterate[TensorDictPrioritizedReplayBuffer-LazyTensorStorage-None-10000] 4.6617ms 0.6487ms 1.5416 KOps/s 1.5685 KOps/s $\color{#d91a1a}-1.72\%$
test_rb_populate[TensorDictReplayBuffer-ListStorage-RandomSampler-400] 0.1195s 7.0491ms 141.8622 Ops/s 115.0500 Ops/s $\textbf{\color{#35bf28}+23.30\%}$
test_rb_populate[TensorDictReplayBuffer-LazyMemmapStorage-RandomSampler-400] 19.9213ms 14.6281ms 68.3614 Ops/s 67.9136 Ops/s $\color{#35bf28}+0.66\%$
test_rb_populate[TensorDictReplayBuffer-LazyTensorStorage-RandomSampler-400] 2.2177ms 1.0921ms 915.6945 Ops/s 855.0910 Ops/s $\textbf{\color{#35bf28}+7.09\%}$
test_rb_populate[TensorDictReplayBuffer-ListStorage-SamplerWithoutReplacement-400] 0.1004s 8.5755ms 116.6114 Ops/s 148.7293 Ops/s $\textbf{\color{#d91a1a}-21.59\%}$
test_rb_populate[TensorDictReplayBuffer-LazyMemmapStorage-SamplerWithoutReplacement-400] 16.7968ms 14.4288ms 69.3059 Ops/s 68.5650 Ops/s $\color{#35bf28}+1.08\%$
test_rb_populate[TensorDictReplayBuffer-LazyTensorStorage-SamplerWithoutReplacement-400] 2.1051ms 1.0905ms 917.0429 Ops/s 886.3880 Ops/s $\color{#35bf28}+3.46\%$
test_rb_populate[TensorDictPrioritizedReplayBuffer-ListStorage-None-400] 0.1021s 7.0697ms 141.4482 Ops/s 110.5258 Ops/s $\textbf{\color{#35bf28}+27.98\%}$
test_rb_populate[TensorDictPrioritizedReplayBuffer-LazyMemmapStorage-None-400] 17.5695ms 15.0627ms 66.3892 Ops/s 66.6765 Ops/s $\color{#d91a1a}-0.43\%$
test_rb_populate[TensorDictPrioritizedReplayBuffer-LazyTensorStorage-None-400] 2.6811ms 1.4580ms 685.8876 Ops/s 624.3746 Ops/s $\textbf{\color{#35bf28}+9.85\%}$

@github-actions
Copy link

github-actions bot commented Mar 20, 2024

$\color{#D29922}\textsf{\Large⚠\kern{0.2cm}\normalsize Warning}$ Result of CPU Benchmark Tests

Total Benchmarks: 91. Improved: $\large\color{#35bf28}5$. Worsened: $\large\color{#d91a1a}6$.

Expand to view detailed results
Name Max Mean Ops Ops on Repo HEAD Change
test_single 53.5416ms 51.4568ms 19.4338 Ops/s 18.0843 Ops/s $\textbf{\color{#35bf28}+7.46\%}$
test_sync 42.8238ms 29.1070ms 34.3560 Ops/s 35.1815 Ops/s $\color{#d91a1a}-2.35\%$
test_async 52.1107ms 26.8242ms 37.2797 Ops/s 39.3935 Ops/s $\textbf{\color{#d91a1a}-5.37\%}$
test_simple 0.3652s 0.3153s 3.1719 Ops/s 3.2618 Ops/s $\color{#d91a1a}-2.76\%$
test_transformed 0.4540s 0.4448s 2.2480 Ops/s 2.2154 Ops/s $\color{#35bf28}+1.47\%$
test_serial 1.1677s 1.1284s 0.8862 Ops/s 0.8837 Ops/s $\color{#35bf28}+0.29\%$
test_parallel 1.0339s 0.9941s 1.0059 Ops/s 1.0250 Ops/s $\color{#d91a1a}-1.86\%$
test_step_mdp_speed[True-True-True-True-True] 0.1044ms 20.3899μs 49.0439 KOps/s 49.5854 KOps/s $\color{#d91a1a}-1.09\%$
test_step_mdp_speed[True-True-True-True-False] 31.3790μs 12.6480μs 79.0638 KOps/s 82.8263 KOps/s $\color{#d91a1a}-4.54\%$
test_step_mdp_speed[True-True-True-False-True] 31.8090μs 12.0132μs 83.2421 KOps/s 84.1913 KOps/s $\color{#d91a1a}-1.13\%$
test_step_mdp_speed[True-True-True-False-False] 0.3322ms 7.2226μs 138.4540 KOps/s 139.7320 KOps/s $\color{#d91a1a}-0.91\%$
test_step_mdp_speed[True-True-False-True-True] 52.4180μs 21.4641μs 46.5894 KOps/s 46.2887 KOps/s $\color{#35bf28}+0.65\%$
test_step_mdp_speed[True-True-False-True-False] 33.8530μs 13.5343μs 73.8863 KOps/s 74.2818 KOps/s $\color{#d91a1a}-0.53\%$
test_step_mdp_speed[True-True-False-False-True] 34.8660μs 13.1330μs 76.1443 KOps/s 78.2166 KOps/s $\color{#d91a1a}-2.65\%$
test_step_mdp_speed[True-True-False-False-False] 30.1970μs 8.5007μs 117.6380 KOps/s 121.2268 KOps/s $\color{#d91a1a}-2.96\%$
test_step_mdp_speed[True-False-True-True-True] 51.9470μs 22.9836μs 43.5092 KOps/s 44.9036 KOps/s $\color{#d91a1a}-3.11\%$
test_step_mdp_speed[True-False-True-True-False] 51.7670μs 14.6228μs 68.3864 KOps/s 67.3863 KOps/s $\color{#35bf28}+1.48\%$
test_step_mdp_speed[True-False-True-False-True] 42.7300μs 13.1275μs 76.1760 KOps/s 78.0963 KOps/s $\color{#d91a1a}-2.46\%$
test_step_mdp_speed[True-False-True-False-False] 26.8500μs 8.5652μs 116.7517 KOps/s 119.9464 KOps/s $\color{#d91a1a}-2.66\%$
test_step_mdp_speed[True-False-False-True-True] 50.9550μs 23.7894μs 42.0354 KOps/s 42.1861 KOps/s $\color{#d91a1a}-0.36\%$
test_step_mdp_speed[True-False-False-True-False] 34.7040μs 16.0079μs 62.4692 KOps/s 64.1639 KOps/s $\color{#d91a1a}-2.64\%$
test_step_mdp_speed[True-False-False-False-True] 43.4510μs 14.1181μs 70.8309 KOps/s 71.5808 KOps/s $\color{#d91a1a}-1.05\%$
test_step_mdp_speed[True-False-False-False-False] 28.1330μs 9.5354μs 104.8719 KOps/s 106.0339 KOps/s $\color{#d91a1a}-1.10\%$
test_step_mdp_speed[False-True-True-True-True] 52.6790μs 22.7988μs 43.8620 KOps/s 43.9053 KOps/s $\color{#d91a1a}-0.10\%$
test_step_mdp_speed[False-True-True-True-False] 40.1250μs 14.9677μs 66.8107 KOps/s 67.9208 KOps/s $\color{#d91a1a}-1.63\%$
test_step_mdp_speed[False-True-True-False-True] 36.4570μs 15.3729μs 65.0496 KOps/s 66.7165 KOps/s $\color{#d91a1a}-2.50\%$
test_step_mdp_speed[False-True-True-False-False] 35.4570μs 9.5925μs 104.2483 KOps/s 104.8686 KOps/s $\color{#d91a1a}-0.59\%$
test_step_mdp_speed[False-True-False-True-True] 63.6390μs 25.3697μs 39.4172 KOps/s 39.4831 KOps/s $\color{#d91a1a}-0.17\%$
test_step_mdp_speed[False-True-False-True-False] 46.9980μs 16.4036μs 60.9622 KOps/s 62.7630 KOps/s $\color{#d91a1a}-2.87\%$
test_step_mdp_speed[False-True-False-False-True] 0.3872ms 16.3062μs 61.3263 KOps/s 62.5071 KOps/s $\color{#d91a1a}-1.89\%$
test_step_mdp_speed[False-True-False-False-False] 32.8910μs 10.6617μs 93.7934 KOps/s 94.7591 KOps/s $\color{#d91a1a}-1.02\%$
test_step_mdp_speed[False-False-True-True-True] 54.0810μs 24.9154μs 40.1359 KOps/s 40.0174 KOps/s $\color{#35bf28}+0.30\%$
test_step_mdp_speed[False-False-True-True-False] 44.0920μs 17.2324μs 58.0302 KOps/s 58.3292 KOps/s $\color{#d91a1a}-0.51\%$
test_step_mdp_speed[False-False-True-False-True] 35.8470μs 16.4956μs 60.6222 KOps/s 62.0779 KOps/s $\color{#d91a1a}-2.35\%$
test_step_mdp_speed[False-False-True-False-False] 29.4650μs 10.7355μs 93.1485 KOps/s 95.0655 KOps/s $\color{#d91a1a}-2.02\%$
test_step_mdp_speed[False-False-False-True-True] 63.9500μs 26.5026μs 37.7321 KOps/s 39.2884 KOps/s $\color{#d91a1a}-3.96\%$
test_step_mdp_speed[False-False-False-True-False] 48.0000μs 18.1087μs 55.2221 KOps/s 56.7767 KOps/s $\color{#d91a1a}-2.74\%$
test_step_mdp_speed[False-False-False-False-True] 40.4760μs 16.9136μs 59.1241 KOps/s 58.2184 KOps/s $\color{#35bf28}+1.56\%$
test_step_mdp_speed[False-False-False-False-False] 32.1400μs 11.5481μs 86.5941 KOps/s 85.8501 KOps/s $\color{#35bf28}+0.87\%$
test_values[generalized_advantage_estimate-True-True] 9.5606ms 8.5805ms 116.5439 Ops/s 109.7579 Ops/s $\textbf{\color{#35bf28}+6.18\%}$
test_values[vec_generalized_advantage_estimate-True-True] 35.0969ms 33.7353ms 29.6425 Ops/s 29.6287 Ops/s $\color{#35bf28}+0.05\%$
test_values[td0_return_estimate-False-False] 0.2309ms 0.1535ms 6.5136 KOps/s 6.4510 KOps/s $\color{#35bf28}+0.97\%$
test_values[td1_return_estimate-False-False] 22.1335ms 21.1882ms 47.1961 Ops/s 45.4988 Ops/s $\color{#35bf28}+3.73\%$
test_values[vec_td1_return_estimate-False-False] 36.4404ms 34.2358ms 29.2092 Ops/s 29.1729 Ops/s $\color{#35bf28}+0.12\%$
test_values[td_lambda_return_estimate-True-False] 31.9377ms 30.3826ms 32.9135 Ops/s 31.3888 Ops/s $\color{#35bf28}+4.86\%$
test_values[vec_td_lambda_return_estimate-True-False] 36.2597ms 34.1497ms 29.2829 Ops/s 29.4850 Ops/s $\color{#d91a1a}-0.69\%$
test_gae_speed[generalized_advantage_estimate-False-1-512] 9.8065ms 7.5657ms 132.1748 Ops/s 127.6863 Ops/s $\color{#35bf28}+3.52\%$
test_gae_speed[vec_generalized_advantage_estimate-True-1-512] 2.2486ms 1.9054ms 524.8224 Ops/s 521.7178 Ops/s $\color{#35bf28}+0.60\%$
test_gae_speed[vec_generalized_advantage_estimate-False-1-512] 0.5653ms 0.3246ms 3.0803 KOps/s 2.8332 KOps/s $\textbf{\color{#35bf28}+8.72\%}$
test_gae_speed[vec_generalized_advantage_estimate-True-32-512] 45.6868ms 42.8774ms 23.3223 Ops/s 22.7898 Ops/s $\color{#35bf28}+2.34\%$
test_gae_speed[vec_generalized_advantage_estimate-False-32-512] 3.8427ms 2.9475ms 339.2684 Ops/s 349.6554 Ops/s $\color{#d91a1a}-2.97\%$
test_dqn_speed 6.6078ms 1.2918ms 774.0905 Ops/s 799.8474 Ops/s $\color{#d91a1a}-3.22\%$
test_ddpg_speed 2.9353ms 2.5010ms 399.8331 Ops/s 393.8334 Ops/s $\color{#35bf28}+1.52\%$
test_sac_speed 8.8206ms 7.6776ms 130.2491 Ops/s 128.9474 Ops/s $\color{#35bf28}+1.01\%$
test_redq_speed 13.6120ms 12.4541ms 80.2949 Ops/s 81.0350 Ops/s $\color{#d91a1a}-0.91\%$
test_redq_deprec_speed 13.0226ms 12.0849ms 82.7481 Ops/s 81.3685 Ops/s $\color{#35bf28}+1.70\%$
test_td3_speed 10.1240ms 7.7002ms 129.8670 Ops/s 131.3192 Ops/s $\color{#d91a1a}-1.11\%$
test_cql_speed 37.0875ms 34.6662ms 28.8465 Ops/s 29.2381 Ops/s $\color{#d91a1a}-1.34\%$
test_a2c_speed 7.6838ms 6.9712ms 143.4480 Ops/s 142.6292 Ops/s $\color{#35bf28}+0.57\%$
test_ppo_speed 8.3833ms 7.3298ms 136.4295 Ops/s 141.0424 Ops/s $\color{#d91a1a}-3.27\%$
test_reinforce_speed 7.4048ms 6.2638ms 159.6463 Ops/s 161.2173 Ops/s $\color{#d91a1a}-0.97\%$
test_iql_speed 31.8441ms 30.1174ms 33.2034 Ops/s 32.2744 Ops/s $\color{#35bf28}+2.88\%$
test_rb_sample[TensorDictReplayBuffer-ListStorage-RandomSampler-4000] 2.1656ms 1.9823ms 504.4535 Ops/s 503.4376 Ops/s $\color{#35bf28}+0.20\%$
test_rb_sample[TensorDictReplayBuffer-LazyMemmapStorage-RandomSampler-10000] 0.7589ms 0.4648ms 2.1513 KOps/s 2.1402 KOps/s $\color{#35bf28}+0.52\%$
test_rb_sample[TensorDictReplayBuffer-LazyTensorStorage-RandomSampler-10000] 0.8228ms 0.4495ms 2.2249 KOps/s 2.2586 KOps/s $\color{#d91a1a}-1.49\%$
test_rb_sample[TensorDictReplayBuffer-ListStorage-SamplerWithoutReplacement-4000] 2.2124ms 1.9439ms 514.4376 Ops/s 513.3956 Ops/s $\color{#35bf28}+0.20\%$
test_rb_sample[TensorDictReplayBuffer-LazyMemmapStorage-SamplerWithoutReplacement-10000] 0.9439ms 0.4690ms 2.1323 KOps/s 2.2031 KOps/s $\color{#d91a1a}-3.22\%$
test_rb_sample[TensorDictReplayBuffer-LazyTensorStorage-SamplerWithoutReplacement-10000] 0.5934ms 0.4401ms 2.2722 KOps/s 2.2571 KOps/s $\color{#35bf28}+0.67\%$
test_rb_sample[TensorDictReplayBuffer-LazyMemmapStorage-sampler6-10000] 1.7563ms 1.2040ms 830.5895 Ops/s 832.1165 Ops/s $\color{#d91a1a}-0.18\%$
test_rb_sample[TensorDictReplayBuffer-LazyTensorStorage-sampler7-10000] 1.4008ms 1.1215ms 891.6546 Ops/s 882.0257 Ops/s $\color{#35bf28}+1.09\%$
test_rb_sample[TensorDictPrioritizedReplayBuffer-ListStorage-None-4000] 3.0487ms 2.0684ms 483.4694 Ops/s 482.4064 Ops/s $\color{#35bf28}+0.22\%$
test_rb_sample[TensorDictPrioritizedReplayBuffer-LazyMemmapStorage-None-10000] 0.8804ms 0.5768ms 1.7336 KOps/s 1.7509 KOps/s $\color{#d91a1a}-0.98\%$
test_rb_sample[TensorDictPrioritizedReplayBuffer-LazyTensorStorage-None-10000] 0.8937ms 0.5508ms 1.8156 KOps/s 1.8274 KOps/s $\color{#d91a1a}-0.65\%$
test_rb_iterate[TensorDictReplayBuffer-ListStorage-RandomSampler-4000] 3.1104ms 1.9448ms 514.1944 Ops/s 501.2964 Ops/s $\color{#35bf28}+2.57\%$
test_rb_iterate[TensorDictReplayBuffer-LazyMemmapStorage-RandomSampler-10000] 0.7450ms 0.4712ms 2.1223 KOps/s 2.1549 KOps/s $\color{#d91a1a}-1.51\%$
test_rb_iterate[TensorDictReplayBuffer-LazyTensorStorage-RandomSampler-10000] 0.5510ms 0.4356ms 2.2958 KOps/s 2.2471 KOps/s $\color{#35bf28}+2.17\%$
test_rb_iterate[TensorDictReplayBuffer-ListStorage-SamplerWithoutReplacement-4000] 3.1264ms 1.9241ms 519.7118 Ops/s 516.7309 Ops/s $\color{#35bf28}+0.58\%$
test_rb_iterate[TensorDictReplayBuffer-LazyMemmapStorage-SamplerWithoutReplacement-10000] 0.5809ms 0.4630ms 2.1598 KOps/s 2.2223 KOps/s $\color{#d91a1a}-2.82\%$
test_rb_iterate[TensorDictReplayBuffer-LazyTensorStorage-SamplerWithoutReplacement-10000] 83.4875ms 0.5589ms 1.7891 KOps/s 2.3377 KOps/s $\textbf{\color{#d91a1a}-23.47\%}$
test_rb_iterate[TensorDictPrioritizedReplayBuffer-ListStorage-None-4000] 2.3631ms 2.0875ms 479.0493 Ops/s 476.1742 Ops/s $\color{#35bf28}+0.60\%$
test_rb_iterate[TensorDictPrioritizedReplayBuffer-LazyMemmapStorage-None-10000] 1.0994ms 0.5734ms 1.7440 KOps/s 1.7354 KOps/s $\color{#35bf28}+0.50\%$
test_rb_iterate[TensorDictPrioritizedReplayBuffer-LazyTensorStorage-None-10000] 0.9468ms 0.5563ms 1.7974 KOps/s 1.8099 KOps/s $\color{#d91a1a}-0.69\%$
test_rb_populate[TensorDictReplayBuffer-ListStorage-RandomSampler-400] 88.6181ms 6.6971ms 149.3173 Ops/s 149.0047 Ops/s $\color{#35bf28}+0.21\%$
test_rb_populate[TensorDictReplayBuffer-LazyMemmapStorage-RandomSampler-400] 13.8845ms 11.6240ms 86.0291 Ops/s 86.4675 Ops/s $\color{#d91a1a}-0.51\%$
test_rb_populate[TensorDictReplayBuffer-LazyTensorStorage-RandomSampler-400] 1.5678ms 1.0613ms 942.2613 Ops/s 1.0076 KOps/s $\textbf{\color{#d91a1a}-6.48\%}$
test_rb_populate[TensorDictReplayBuffer-ListStorage-SamplerWithoutReplacement-400] 85.0725ms 5.0808ms 196.8181 Ops/s 152.9659 Ops/s $\textbf{\color{#35bf28}+28.67\%}$
test_rb_populate[TensorDictReplayBuffer-LazyMemmapStorage-SamplerWithoutReplacement-400] 13.7963ms 11.4694ms 87.1883 Ops/s 87.7838 Ops/s $\color{#d91a1a}-0.68\%$
test_rb_populate[TensorDictReplayBuffer-LazyTensorStorage-SamplerWithoutReplacement-400] 3.5312ms 1.1115ms 899.6952 Ops/s 1.0187 KOps/s $\textbf{\color{#d91a1a}-11.68\%}$
test_rb_populate[TensorDictPrioritizedReplayBuffer-ListStorage-None-400] 86.0626ms 6.9322ms 144.2536 Ops/s 187.6161 Ops/s $\textbf{\color{#d91a1a}-23.11\%}$
test_rb_populate[TensorDictPrioritizedReplayBuffer-LazyMemmapStorage-None-400] 14.4424ms 11.7227ms 85.3043 Ops/s 74.3504 Ops/s $\textbf{\color{#35bf28}+14.73\%}$
test_rb_populate[TensorDictPrioritizedReplayBuffer-LazyTensorStorage-None-400] 4.0174ms 1.3989ms 714.8463 Ops/s 777.3869 Ops/s $\textbf{\color{#d91a1a}-8.04\%}$

@vmoens vmoens deleted the hetero-storage branch March 21, 2024 09:14
vmoens pushed a commit that referenced this pull request Mar 25, 2024
[Minor] Support init lazy storages with add (#2028)
8d92327 
(cherry picked from commit 660d827)
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Reviewers

No reviews

Assignees

No one assigned

Labels

bug Something isn't working CLA Signed This label is managed by the Facebook bot. Authors need to sign the CLA before a PR can be reviewed.

Projects

None yet

Milestone

No milestone

Development

Successfully merging this pull request may close these issues.

3 participants

@vmoens @facebook-github-bot