Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

[CI] Less warnings in CI (costs) #1349

Merged
merged 4 commits into from
Jul 3, 2023
Merged

[CI] Less warnings in CI (costs) #1349

merged 4 commits into from
Jul 3, 2023

Conversation

vmoens
Copy link
Contributor

@vmoens vmoens commented Jul 3, 2023 •
edited
Loading

No description provided.

@facebook-github-bot facebook-github-bot added the CLA Signed This label is managed by the Facebook bot. Authors need to sign the CLA before a PR can be reviewed. label Jul 3, 2023
@vmoens vmoens changed the title [CI] Less warnings in CI [CI] Less warnings in CI (costs) Jul 3, 2023
@vmoens vmoens added the CI Has to do with CI setup (e.g. wheels & builds, tests...) label Jul 3, 2023
@github-actions
Copy link

github-actions bot commented Jul 3, 2023 •
edited
Loading

$\color{#D29922}\textsf{\Large⚠\kern{0.2cm}\normalsize Warning}$ Result of CPU Benchmark Tests

Total Benchmarks: 89. Improved: $\large\color{#35bf28}6$. Worsened: $\large\color{#d91a1a}6$.

Expand to view detailed results
Name Max Mean Ops Ops on Repo HEAD Change
test_single 0.1632s 0.1625s 6.1543 Ops/s 6.1466 Ops/s $\color{#35bf28}+0.13\%$
test_sync 89.5398ms 86.3768ms 11.5772 Ops/s 11.4071 Ops/s $\color{#35bf28}+1.49\%$
test_async 0.1960s 83.2374ms 12.0138 Ops/s 11.9289 Ops/s $\color{#35bf28}+0.71\%$
test_simple 0.7077s 0.6270s 1.5948 Ops/s 1.5848 Ops/s $\color{#35bf28}+0.63\%$
test_transformed 1.7378s 1.6788s 0.5957 Ops/s 0.5877 Ops/s $\color{#35bf28}+1.36\%$
test_serial 2.0359s 1.9580s 0.5107 Ops/s 0.5063 Ops/s $\color{#35bf28}+0.88\%$
test_parallel 1.6898s 1.5889s 0.6294 Ops/s 0.6283 Ops/s $\color{#35bf28}+0.17\%$
test_step_mdp_speed[True-True-True-True-True] 75.7020μs 39.2542μs 25.4750 KOps/s 25.3390 KOps/s $\color{#35bf28}+0.54\%$
test_step_mdp_speed[True-True-True-True-False] 49.2010μs 21.9300μs 45.5996 KOps/s 45.9750 KOps/s $\color{#d91a1a}-0.82\%$
test_step_mdp_speed[True-True-True-False-True] 71.3010μs 30.1253μs 33.1947 KOps/s 33.3868 KOps/s $\color{#d91a1a}-0.58\%$
test_step_mdp_speed[True-True-True-False-False] 42.3010μs 16.6688μs 59.9924 KOps/s 60.4343 KOps/s $\color{#d91a1a}-0.73\%$
test_step_mdp_speed[True-True-False-True-True] 66.7020μs 40.3998μs 24.7526 KOps/s 24.9104 KOps/s $\color{#d91a1a}-0.63\%$
test_step_mdp_speed[True-True-False-True-False] 0.1821ms 23.3845μs 42.7634 KOps/s 43.2876 KOps/s $\color{#d91a1a}-1.21\%$
test_step_mdp_speed[True-True-False-False-True] 59.1010μs 31.2846μs 31.9646 KOps/s 32.0828 KOps/s $\color{#d91a1a}-0.37\%$
test_step_mdp_speed[True-True-False-False-False] 55.6010μs 18.0232μs 55.4839 KOps/s 55.8156 KOps/s $\color{#d91a1a}-0.59\%$
test_step_mdp_speed[True-False-True-True-True] 0.1158ms 41.6709μs 23.9976 KOps/s 24.1974 KOps/s $\color{#d91a1a}-0.83\%$
test_step_mdp_speed[True-False-True-True-False] 0.1748ms 24.4431μs 40.9114 KOps/s 41.2310 KOps/s $\color{#d91a1a}-0.78\%$
test_step_mdp_speed[True-False-True-False-True] 55.0010μs 31.3001μs 31.9488 KOps/s 32.3063 KOps/s $\color{#d91a1a}-1.11\%$
test_step_mdp_speed[True-False-True-False-False] 39.6010μs 17.7857μs 56.2250 KOps/s 54.8846 KOps/s $\color{#35bf28}+2.44\%$
test_step_mdp_speed[True-False-False-True-True] 85.7000μs 42.6441μs 23.4499 KOps/s 23.3775 KOps/s $\color{#35bf28}+0.31\%$
test_step_mdp_speed[True-False-False-True-False] 53.9010μs 25.7916μs 38.7723 KOps/s 39.7362 KOps/s $\color{#d91a1a}-2.43\%$
test_step_mdp_speed[True-False-False-False-True] 81.3010μs 32.7548μs 30.5298 KOps/s 31.3467 KOps/s $\color{#d91a1a}-2.61\%$
test_step_mdp_speed[True-False-False-False-False] 55.0000μs 19.2787μs 51.8707 KOps/s 53.5867 KOps/s $\color{#d91a1a}-3.20\%$
test_step_mdp_speed[False-True-True-True-True] 70.5000μs 41.6486μs 24.0104 KOps/s 23.7888 KOps/s $\color{#35bf28}+0.93\%$
test_step_mdp_speed[False-True-True-True-False] 90.1000μs 24.5858μs 40.6738 KOps/s 41.4900 KOps/s $\color{#d91a1a}-1.97\%$
test_step_mdp_speed[False-True-True-False-True] 62.6000μs 35.3376μs 28.2985 KOps/s 28.3043 KOps/s $\color{#d91a1a}-0.02\%$
test_step_mdp_speed[False-True-True-False-False] 99.0010μs 19.9373μs 50.1572 KOps/s 51.7137 KOps/s $\color{#d91a1a}-3.01\%$
test_step_mdp_speed[False-True-False-True-True] 67.3000μs 42.8314μs 23.3474 KOps/s 23.2797 KOps/s $\color{#35bf28}+0.29\%$
test_step_mdp_speed[False-True-False-True-False] 47.0000μs 25.7444μs 38.8434 KOps/s 39.3775 KOps/s $\color{#d91a1a}-1.36\%$
test_step_mdp_speed[False-True-False-False-True] 85.1010μs 36.6436μs 27.2899 KOps/s 27.3262 KOps/s $\color{#d91a1a}-0.13\%$
test_step_mdp_speed[False-True-False-False-False] 81.9010μs 21.1195μs 47.3496 KOps/s 48.5470 KOps/s $\color{#d91a1a}-2.47\%$
test_step_mdp_speed[False-False-True-True-True] 71.4000μs 43.8909μs 22.7838 KOps/s 22.5895 KOps/s $\color{#35bf28}+0.86\%$
test_step_mdp_speed[False-False-True-True-False] 86.2000μs 26.7009μs 37.4519 KOps/s 37.6386 KOps/s $\color{#d91a1a}-0.50\%$
test_step_mdp_speed[False-False-True-False-True] 52.5000μs 36.9759μs 27.0446 KOps/s 27.0697 KOps/s $\color{#d91a1a}-0.09\%$
test_step_mdp_speed[False-False-True-False-False] 95.9010μs 20.7395μs 48.2171 KOps/s 49.2821 KOps/s $\color{#d91a1a}-2.16\%$
test_step_mdp_speed[False-False-False-True-True] 92.6000μs 45.4952μs 21.9803 KOps/s 22.5737 KOps/s $\color{#d91a1a}-2.63\%$
test_step_mdp_speed[False-False-False-True-False] 56.0000μs 27.7374μs 36.0524 KOps/s 36.2512 KOps/s $\color{#d91a1a}-0.55\%$
test_step_mdp_speed[False-False-False-False-True] 59.2010μs 37.4781μs 26.6823 KOps/s 27.0444 KOps/s $\color{#d91a1a}-1.34\%$
test_step_mdp_speed[False-False-False-False-False] 46.8000μs 21.6413μs 46.2079 KOps/s 46.8339 KOps/s $\color{#d91a1a}-1.34\%$
test_values[generalized_advantage_estimate-True-True] 14.8541ms 13.7796ms 72.5712 Ops/s 71.8762 Ops/s $\color{#35bf28}+0.97\%$
test_values[vec_generalized_advantage_estimate-True-True] 48.2766ms 42.4907ms 23.5346 Ops/s 23.0858 Ops/s $\color{#35bf28}+1.94\%$
test_values[td0_return_estimate-False-False] 0.3904ms 0.1990ms 5.0251 KOps/s 4.3278 KOps/s $\textbf{\color{#35bf28}+16.11\%}$
test_values[td1_return_estimate-False-False] 13.7441ms 13.3995ms 74.6295 Ops/s 75.1544 Ops/s $\color{#d91a1a}-0.70\%$
test_values[vec_td1_return_estimate-False-False] 43.1176ms 42.3890ms 23.5910 Ops/s 23.4903 Ops/s $\color{#35bf28}+0.43\%$
test_values[td_lambda_return_estimate-True-False] 33.1331ms 32.5508ms 30.7212 Ops/s 31.1538 Ops/s $\color{#d91a1a}-1.39\%$
test_values[vec_td_lambda_return_estimate-True-False] 52.3054ms 42.7559ms 23.3886 Ops/s 23.6034 Ops/s $\color{#d91a1a}-0.91\%$
test_gae_speed[generalized_advantage_estimate-False-1-512] 12.5188ms 12.0379ms 83.0712 Ops/s 83.1851 Ops/s $\color{#d91a1a}-0.14\%$
test_gae_speed[vec_generalized_advantage_estimate-True-1-512] 3.7620ms 3.3317ms 300.1475 Ops/s 297.3520 Ops/s $\color{#35bf28}+0.94\%$
test_gae_speed[vec_generalized_advantage_estimate-False-1-512] 0.7630ms 0.4442ms 2.2512 KOps/s 2.1873 KOps/s $\color{#35bf28}+2.92\%$
test_gae_speed[vec_generalized_advantage_estimate-True-32-512] 60.2024ms 56.1640ms 17.8050 Ops/s 18.5137 Ops/s $\color{#d91a1a}-3.83\%$
test_gae_speed[vec_generalized_advantage_estimate-False-32-512] 3.4137ms 2.7868ms 358.8368 Ops/s 356.2603 Ops/s $\color{#35bf28}+0.72\%$
test_dqn_speed 2.3939ms 1.6409ms 609.4256 Ops/s 622.9012 Ops/s $\color{#d91a1a}-2.16\%$
test_ddpg_speed 12.2278ms 2.7386ms 365.1548 Ops/s 378.6492 Ops/s $\color{#d91a1a}-3.56\%$
test_sac_speed 9.0362ms 8.0291ms 124.5465 Ops/s 125.9420 Ops/s $\color{#d91a1a}-1.11\%$
test_redq_speed 16.2162ms 15.2265ms 65.6749 Ops/s 67.0039 Ops/s $\color{#d91a1a}-1.98\%$
test_redq_deprec_speed 15.8522ms 12.0667ms 82.8727 Ops/s 83.1651 Ops/s $\color{#d91a1a}-0.35\%$
test_td3_speed 11.9548ms 10.6312ms 94.0630 Ops/s 93.7104 Ops/s $\color{#35bf28}+0.38\%$
test_cql_speed 28.6118ms 26.3706ms 37.9210 Ops/s 36.2887 Ops/s $\color{#35bf28}+4.50\%$
test_a2c_speed 6.1104ms 5.1578ms 193.8811 Ops/s 192.8716 Ops/s $\color{#35bf28}+0.52\%$
test_ppo_speed 6.4714ms 5.5616ms 179.8036 Ops/s 177.7888 Ops/s $\color{#35bf28}+1.13\%$
test_reinforce_speed 4.5531ms 3.9769ms 251.4523 Ops/s 239.3308 Ops/s $\textbf{\color{#35bf28}+5.06\%}$
test_iql_speed 24.5004ms 21.2015ms 47.1665 Ops/s 47.7134 Ops/s $\color{#d91a1a}-1.15\%$
test_sample_rb[TensorDictReplayBuffer-ListStorage-RandomSampler-4000] 5.3329ms 4.2383ms 235.9456 Ops/s 229.1646 Ops/s $\color{#35bf28}+2.96\%$
test_sample_rb[TensorDictReplayBuffer-LazyMemmapStorage-RandomSampler-10000] 8.1640ms 4.4007ms 227.2359 Ops/s 220.1646 Ops/s $\color{#35bf28}+3.21\%$
test_sample_rb[TensorDictReplayBuffer-LazyTensorStorage-RandomSampler-10000] 0.1934s 5.2723ms 189.6722 Ops/s 220.0265 Ops/s $\textbf{\color{#d91a1a}-13.80\%}$
test_sample_rb[TensorDictReplayBuffer-ListStorage-SamplerWithoutReplacement-4000] 5.1055ms 4.2586ms 234.8192 Ops/s 233.9359 Ops/s $\color{#35bf28}+0.38\%$
test_sample_rb[TensorDictReplayBuffer-LazyMemmapStorage-SamplerWithoutReplacement-10000] 6.6470ms 4.4685ms 223.7891 Ops/s 223.9446 Ops/s $\color{#d91a1a}-0.07\%$
test_sample_rb[TensorDictReplayBuffer-LazyTensorStorage-SamplerWithoutReplacement-10000] 6.4962ms 4.4152ms 226.4922 Ops/s 223.5011 Ops/s $\color{#35bf28}+1.34\%$
test_sample_rb[TensorDictPrioritizedReplayBuffer-ListStorage-None-4000] 5.5706ms 4.2329ms 236.2468 Ops/s 235.3468 Ops/s $\color{#35bf28}+0.38\%$
test_sample_rb[TensorDictPrioritizedReplayBuffer-LazyMemmapStorage-None-10000] 7.8471ms 4.4757ms 223.4306 Ops/s 188.0318 Ops/s $\textbf{\color{#35bf28}+18.83\%}$
test_sample_rb[TensorDictPrioritizedReplayBuffer-LazyTensorStorage-None-10000] 6.7749ms 4.4394ms 225.2581 Ops/s 227.0305 Ops/s $\color{#d91a1a}-0.78\%$
test_iterate_rb[TensorDictReplayBuffer-ListStorage-RandomSampler-4000] 0.1571s 4.9411ms 202.3821 Ops/s 235.7735 Ops/s $\textbf{\color{#d91a1a}-14.16\%}$
test_iterate_rb[TensorDictReplayBuffer-LazyMemmapStorage-RandomSampler-10000] 7.2980ms 4.4844ms 222.9948 Ops/s 224.7132 Ops/s $\color{#d91a1a}-0.76\%$
test_iterate_rb[TensorDictReplayBuffer-LazyTensorStorage-RandomSampler-10000] 7.2306ms 4.4496ms 224.7393 Ops/s 225.4223 Ops/s $\color{#d91a1a}-0.30\%$
test_iterate_rb[TensorDictReplayBuffer-ListStorage-SamplerWithoutReplacement-4000] 0.3237s 5.6606ms 176.6595 Ops/s 237.3196 Ops/s $\textbf{\color{#d91a1a}-25.56\%}$
test_iterate_rb[TensorDictReplayBuffer-LazyMemmapStorage-SamplerWithoutReplacement-10000] 7.3452ms 4.4225ms 226.1188 Ops/s 226.1141 Ops/s $+0.00\%$
test_iterate_rb[TensorDictReplayBuffer-LazyTensorStorage-SamplerWithoutReplacement-10000] 0.1955s 5.3072ms 188.4247 Ops/s 225.2074 Ops/s $\textbf{\color{#d91a1a}-16.33\%}$
test_iterate_rb[TensorDictPrioritizedReplayBuffer-ListStorage-None-4000] 5.7246ms 4.2885ms 233.1802 Ops/s 233.7453 Ops/s $\color{#d91a1a}-0.24\%$
test_iterate_rb[TensorDictPrioritizedReplayBuffer-LazyMemmapStorage-None-10000] 7.6259ms 4.4448ms 224.9833 Ops/s 223.5740 Ops/s $\color{#35bf28}+0.63\%$
test_iterate_rb[TensorDictPrioritizedReplayBuffer-LazyTensorStorage-None-10000] 8.2235ms 4.4765ms 223.3875 Ops/s 226.5309 Ops/s $\color{#d91a1a}-1.39\%$
test_populate_rb[TensorDictReplayBuffer-ListStorage-RandomSampler-400] 0.3286s 41.8889ms 23.8727 Ops/s 27.9193 Ops/s $\textbf{\color{#d91a1a}-14.49\%}$
test_populate_rb[TensorDictReplayBuffer-LazyMemmapStorage-RandomSampler-400] 0.1854s 35.7197ms 27.9958 Ops/s 28.2486 Ops/s $\color{#d91a1a}-0.89\%$
test_populate_rb[TensorDictReplayBuffer-LazyTensorStorage-RandomSampler-400] 0.1867s 36.2173ms 27.6111 Ops/s 25.8866 Ops/s $\textbf{\color{#35bf28}+6.66\%}$
test_populate_rb[TensorDictReplayBuffer-ListStorage-SamplerWithoutReplacement-400] 0.1888s 32.9454ms 30.3532 Ops/s 27.9124 Ops/s $\textbf{\color{#35bf28}+8.74\%}$
test_populate_rb[TensorDictReplayBuffer-LazyMemmapStorage-SamplerWithoutReplacement-400] 0.1895s 36.6343ms 27.2968 Ops/s 27.8024 Ops/s $\color{#d91a1a}-1.82\%$
test_populate_rb[TensorDictReplayBuffer-LazyTensorStorage-SamplerWithoutReplacement-400] 0.1926s 39.6095ms 25.2465 Ops/s 28.2032 Ops/s $\textbf{\color{#d91a1a}-10.48\%}$
test_populate_rb[TensorDictPrioritizedReplayBuffer-ListStorage-None-400] 0.1845s 35.9955ms 27.7812 Ops/s 25.7832 Ops/s $\textbf{\color{#35bf28}+7.75\%}$
test_populate_rb[TensorDictPrioritizedReplayBuffer-LazyMemmapStorage-None-400] 0.1833s 35.7394ms 27.9803 Ops/s 27.9118 Ops/s $\color{#35bf28}+0.25\%$
test_populate_rb[TensorDictPrioritizedReplayBuffer-LazyTensorStorage-None-400] 0.1902s 35.7369ms 27.9823 Ops/s 27.9622 Ops/s $\color{#35bf28}+0.07\%$

@github-actions
Copy link

github-actions bot commented Jul 3, 2023 •
edited
Loading

$\color{#D29922}\textsf{\Large⚠\kern{0.2cm}\normalsize Warning}$ Result of GPU Benchmark Tests

Total Benchmarks: 89. Improved: $\large\color{#35bf28}2$. Worsened: $\large\color{#d91a1a}10$.

Expand to view detailed results
Name Max Mean Ops Ops on Repo HEAD Change
test_single 0.1608s 0.1606s 6.2277 Ops/s 6.2556 Ops/s $\color{#d91a1a}-0.45\%$
test_sync 85.6371ms 82.7248ms 12.0883 Ops/s 12.1944 Ops/s $\color{#d91a1a}-0.87\%$
test_async 0.1411s 81.4905ms 12.2714 Ops/s 12.3732 Ops/s $\color{#d91a1a}-0.82\%$
test_simple 0.7075s 0.6329s 1.5801 Ops/s 1.5786 Ops/s $\color{#35bf28}+0.09\%$
test_transformed 1.6804s 1.6270s 0.6146 Ops/s 0.6193 Ops/s $\color{#d91a1a}-0.75\%$
test_serial 2.0128s 1.9483s 0.5133 Ops/s 0.5113 Ops/s $\color{#35bf28}+0.39\%$
test_parallel 1.6015s 1.5176s 0.6589 Ops/s 0.6499 Ops/s $\color{#35bf28}+1.38\%$
test_step_mdp_speed[True-True-True-True-True] 0.3113ms 31.1985μs 32.0529 KOps/s 32.3709 KOps/s $\color{#d91a1a}-0.98\%$
test_step_mdp_speed[True-True-True-True-False] 79.1010μs 17.6076μs 56.7937 KOps/s 57.6276 KOps/s $\color{#d91a1a}-1.45\%$
test_step_mdp_speed[True-True-True-False-True] 81.9010μs 23.2670μs 42.9793 KOps/s 43.3206 KOps/s $\color{#d91a1a}-0.79\%$
test_step_mdp_speed[True-True-True-False-False] 32.9000μs 12.9981μs 76.9343 KOps/s 77.7979 KOps/s $\color{#d91a1a}-1.11\%$
test_step_mdp_speed[True-True-False-True-True] 51.8010μs 32.4034μs 30.8609 KOps/s 31.3071 KOps/s $\color{#d91a1a}-1.42\%$
test_step_mdp_speed[True-True-False-True-False] 0.2170ms 18.6558μs 53.6025 KOps/s 54.3789 KOps/s $\color{#d91a1a}-1.43\%$
test_step_mdp_speed[True-True-False-False-True] 69.9010μs 24.5727μs 40.6956 KOps/s 41.3353 KOps/s $\color{#d91a1a}-1.55\%$
test_step_mdp_speed[True-True-False-False-False] 0.6501ms 14.3063μs 69.8994 KOps/s 71.9250 KOps/s $\color{#d91a1a}-2.82\%$
test_step_mdp_speed[True-False-True-True-True] 0.2557ms 33.2961μs 30.0335 KOps/s 30.2442 KOps/s $\color{#d91a1a}-0.70\%$
test_step_mdp_speed[True-False-True-True-False] 0.2315ms 19.5457μs 51.1621 KOps/s 51.8237 KOps/s $\color{#d91a1a}-1.28\%$
test_step_mdp_speed[True-False-True-False-True] 63.5010μs 24.2878μs 41.1729 KOps/s 41.3723 KOps/s $\color{#d91a1a}-0.48\%$
test_step_mdp_speed[True-False-True-False-False] 0.1233ms 14.0480μs 71.1843 KOps/s 72.6488 KOps/s $\color{#d91a1a}-2.02\%$
test_step_mdp_speed[True-False-False-True-True] 0.1215ms 34.6636μs 28.8487 KOps/s 29.0947 KOps/s $\color{#d91a1a}-0.85\%$
test_step_mdp_speed[True-False-False-True-False] 0.1258ms 20.9145μs 47.8138 KOps/s 48.8898 KOps/s $\color{#d91a1a}-2.20\%$
test_step_mdp_speed[True-False-False-False-True] 52.6010μs 25.5203μs 39.1845 KOps/s 38.8664 KOps/s $\color{#35bf28}+0.82\%$
test_step_mdp_speed[True-False-False-False-False] 34.8000μs 15.0171μs 66.5906 KOps/s 67.5271 KOps/s $\color{#d91a1a}-1.39\%$
test_step_mdp_speed[False-True-True-True-True] 55.0010μs 33.2963μs 30.0334 KOps/s 30.3098 KOps/s $\color{#d91a1a}-0.91\%$
test_step_mdp_speed[False-True-True-True-False] 96.0010μs 20.0758μs 49.8112 KOps/s 51.4092 KOps/s $\color{#d91a1a}-3.11\%$
test_step_mdp_speed[False-True-True-False-True] 53.7000μs 28.1309μs 35.5480 KOps/s 35.9122 KOps/s $\color{#d91a1a}-1.01\%$
test_step_mdp_speed[False-True-True-False-False] 40.5000μs 15.6503μs 63.8966 KOps/s 65.3898 KOps/s $\color{#d91a1a}-2.28\%$
test_step_mdp_speed[False-True-False-True-True] 57.4010μs 34.1771μs 29.2594 KOps/s 29.2517 KOps/s $\color{#35bf28}+0.03\%$
test_step_mdp_speed[False-True-False-True-False] 72.6010μs 20.8657μs 47.9256 KOps/s 49.2465 KOps/s $\color{#d91a1a}-2.68\%$
test_step_mdp_speed[False-True-False-False-True] 69.0010μs 28.7961μs 34.7269 KOps/s 34.4131 KOps/s $\color{#35bf28}+0.91\%$
test_step_mdp_speed[False-True-False-False-False] 46.3010μs 16.3897μs 61.0139 KOps/s 60.8621 KOps/s $\color{#35bf28}+0.25\%$
test_step_mdp_speed[False-False-True-True-True] 79.0010μs 35.1822μs 28.4234 KOps/s 28.4009 KOps/s $\color{#35bf28}+0.08\%$
test_step_mdp_speed[False-False-True-True-False] 80.7010μs 21.7438μs 45.9902 KOps/s 47.0009 KOps/s $\color{#d91a1a}-2.15\%$
test_step_mdp_speed[False-False-True-False-True] 46.4000μs 29.6405μs 33.7376 KOps/s 33.8552 KOps/s $\color{#d91a1a}-0.35\%$
test_step_mdp_speed[False-False-True-False-False] 35.5000μs 16.1999μs 61.7287 KOps/s 61.0162 KOps/s $\color{#35bf28}+1.17\%$
test_step_mdp_speed[False-False-False-True-True] 62.5010μs 36.0217μs 27.7610 KOps/s 27.8797 KOps/s $\color{#d91a1a}-0.43\%$
test_step_mdp_speed[False-False-False-True-False] 53.0010μs 22.7490μs 43.9579 KOps/s 44.8496 KOps/s $\color{#d91a1a}-1.99\%$
test_step_mdp_speed[False-False-False-False-True] 0.1750ms 30.1112μs 33.2102 KOps/s 33.7744 KOps/s $\color{#d91a1a}-1.67\%$
test_step_mdp_speed[False-False-False-False-False] 41.4010μs 17.2912μs 57.8329 KOps/s 57.5985 KOps/s $\color{#35bf28}+0.41\%$
test_values[generalized_advantage_estimate-True-True] 14.8893ms 13.8972ms 71.9568 Ops/s 73.5729 Ops/s $\color{#d91a1a}-2.20\%$
test_values[vec_generalized_advantage_estimate-True-True] 47.3017ms 43.8772ms 22.7909 Ops/s 22.4920 Ops/s $\color{#35bf28}+1.33\%$
test_values[td0_return_estimate-False-False] 0.4671ms 0.2078ms 4.8133 KOps/s 4.9374 KOps/s $\color{#d91a1a}-2.51\%$
test_values[td1_return_estimate-False-False] 23.1686ms 13.7301ms 72.8329 Ops/s 76.4096 Ops/s $\color{#d91a1a}-4.68\%$
test_values[vec_td1_return_estimate-False-False] 47.9729ms 43.8261ms 22.8175 Ops/s 22.8818 Ops/s $\color{#d91a1a}-0.28\%$
test_values[td_lambda_return_estimate-True-False] 34.1084ms 33.0021ms 30.3011 Ops/s 30.4540 Ops/s $\color{#d91a1a}-0.50\%$
test_values[vec_td_lambda_return_estimate-True-False] 46.6096ms 43.7245ms 22.8705 Ops/s 23.0021 Ops/s $\color{#d91a1a}-0.57\%$
test_gae_speed[generalized_advantage_estimate-False-1-512] 12.4308ms 12.1626ms 82.2195 Ops/s 84.8715 Ops/s $\color{#d91a1a}-3.12\%$
test_gae_speed[vec_generalized_advantage_estimate-True-1-512] 3.9057ms 3.5367ms 282.7473 Ops/s 297.7549 Ops/s $\textbf{\color{#d91a1a}-5.04\%}$
test_gae_speed[vec_generalized_advantage_estimate-False-1-512] 0.9058ms 0.4671ms 2.1407 KOps/s 2.1449 KOps/s $\color{#d91a1a}-0.19\%$
test_gae_speed[vec_generalized_advantage_estimate-True-32-512] 58.4718ms 56.2957ms 17.7633 Ops/s 18.1470 Ops/s $\color{#d91a1a}-2.11\%$
test_gae_speed[vec_generalized_advantage_estimate-False-32-512] 8.1516ms 2.8378ms 352.3884 Ops/s 356.0795 Ops/s $\color{#d91a1a}-1.04\%$
test_dqn_speed 2.3724ms 1.5778ms 633.7908 Ops/s 636.0871 Ops/s $\color{#d91a1a}-0.36\%$
test_ddpg_speed 3.3347ms 2.6092ms 383.2531 Ops/s 381.3362 Ops/s $\color{#35bf28}+0.50\%$
test_sac_speed 11.5831ms 8.0726ms 123.8766 Ops/s 124.7934 Ops/s $\color{#d91a1a}-0.73\%$
test_redq_speed 17.9954ms 14.2536ms 70.1579 Ops/s 67.9261 Ops/s $\color{#35bf28}+3.29\%$
test_redq_deprec_speed 13.0239ms 11.9541ms 83.6534 Ops/s 85.0505 Ops/s $\color{#d91a1a}-1.64\%$
test_td3_speed 14.3781ms 10.2926ms 97.1569 Ops/s 98.2431 Ops/s $\color{#d91a1a}-1.11\%$
test_cql_speed 28.4073ms 26.9863ms 37.0558 Ops/s 37.3229 Ops/s $\color{#d91a1a}-0.72\%$
test_a2c_speed 5.7690ms 5.2392ms 190.8706 Ops/s 192.7567 Ops/s $\color{#d91a1a}-0.98\%$
test_ppo_speed 7.9823ms 5.7029ms 175.3480 Ops/s 184.9098 Ops/s $\textbf{\color{#d91a1a}-5.17\%}$
test_reinforce_speed 4.9916ms 4.1273ms 242.2916 Ops/s 244.1868 Ops/s $\color{#d91a1a}-0.78\%$
test_iql_speed 22.1594ms 20.9796ms 47.6653 Ops/s 47.5006 Ops/s $\color{#35bf28}+0.35\%$
test_sample_rb[TensorDictReplayBuffer-ListStorage-RandomSampler-4000] 0.1280s 4.0486ms 246.9962 Ops/s 281.5807 Ops/s $\textbf{\color{#d91a1a}-12.28\%}$
test_sample_rb[TensorDictReplayBuffer-LazyMemmapStorage-RandomSampler-10000] 7.0407ms 3.7725ms 265.0727 Ops/s 265.0809 Ops/s $-0.00\%$
test_sample_rb[TensorDictReplayBuffer-LazyTensorStorage-RandomSampler-10000] 5.4900ms 3.7920ms 263.7112 Ops/s 262.4317 Ops/s $\color{#35bf28}+0.49\%$
test_sample_rb[TensorDictReplayBuffer-ListStorage-SamplerWithoutReplacement-4000] 4.9666ms 3.5212ms 283.9932 Ops/s 278.9396 Ops/s $\color{#35bf28}+1.81\%$
test_sample_rb[TensorDictReplayBuffer-LazyMemmapStorage-SamplerWithoutReplacement-10000] 5.7479ms 3.7660ms 265.5363 Ops/s 264.6577 Ops/s $\color{#35bf28}+0.33\%$
test_sample_rb[TensorDictReplayBuffer-LazyTensorStorage-SamplerWithoutReplacement-10000] 11.5742ms 3.9019ms 256.2852 Ops/s 263.2648 Ops/s $\color{#d91a1a}-2.65\%$
test_sample_rb[TensorDictPrioritizedReplayBuffer-ListStorage-None-4000] 0.1232s 3.9885ms 250.7184 Ops/s 274.5446 Ops/s $\textbf{\color{#d91a1a}-8.68\%}$
test_sample_rb[TensorDictPrioritizedReplayBuffer-LazyMemmapStorage-None-10000] 5.6011ms 3.7900ms 263.8512 Ops/s 270.2504 Ops/s $\color{#d91a1a}-2.37\%$
test_sample_rb[TensorDictPrioritizedReplayBuffer-LazyTensorStorage-None-10000] 0.1722s 4.4214ms 226.1746 Ops/s 265.0043 Ops/s $\textbf{\color{#d91a1a}-14.65\%}$
test_iterate_rb[TensorDictReplayBuffer-ListStorage-RandomSampler-4000] 4.6102ms 3.5270ms 283.5261 Ops/s 280.7740 Ops/s $\color{#35bf28}+0.98\%$
test_iterate_rb[TensorDictReplayBuffer-LazyMemmapStorage-RandomSampler-10000] 5.7516ms 3.8481ms 259.8713 Ops/s 265.3321 Ops/s $\color{#d91a1a}-2.06\%$
test_iterate_rb[TensorDictReplayBuffer-LazyTensorStorage-RandomSampler-10000] 5.3513ms 3.7861ms 264.1249 Ops/s 267.6858 Ops/s $\color{#d91a1a}-1.33\%$
test_iterate_rb[TensorDictReplayBuffer-ListStorage-SamplerWithoutReplacement-4000] 3.8935ms 3.5350ms 282.8891 Ops/s 278.8069 Ops/s $\color{#35bf28}+1.46\%$
test_iterate_rb[TensorDictReplayBuffer-LazyMemmapStorage-SamplerWithoutReplacement-10000] 6.9734ms 3.7729ms 265.0501 Ops/s 266.9949 Ops/s $\color{#d91a1a}-0.73\%$
test_iterate_rb[TensorDictReplayBuffer-LazyTensorStorage-SamplerWithoutReplacement-10000] 0.1704s 4.4010ms 227.2197 Ops/s 260.9673 Ops/s $\textbf{\color{#d91a1a}-12.93\%}$
test_iterate_rb[TensorDictPrioritizedReplayBuffer-ListStorage-None-4000] 4.4657ms 3.5642ms 280.5640 Ops/s 281.4430 Ops/s $\color{#d91a1a}-0.31\%$
test_iterate_rb[TensorDictPrioritizedReplayBuffer-LazyMemmapStorage-None-10000] 6.3665ms 3.8272ms 261.2893 Ops/s 265.3682 Ops/s $\color{#d91a1a}-1.54\%$
test_iterate_rb[TensorDictPrioritizedReplayBuffer-LazyTensorStorage-None-10000] 6.0729ms 3.7739ms 264.9776 Ops/s 260.9461 Ops/s $\color{#35bf28}+1.54\%$
test_populate_rb[TensorDictReplayBuffer-ListStorage-RandomSampler-400] 0.2735s 35.6564ms 28.0454 Ops/s 29.7666 Ops/s $\textbf{\color{#d91a1a}-5.78\%}$
test_populate_rb[TensorDictReplayBuffer-LazyMemmapStorage-RandomSampler-400] 0.1576s 30.3958ms 32.8992 Ops/s 30.1846 Ops/s $\textbf{\color{#35bf28}+8.99\%}$
test_populate_rb[TensorDictReplayBuffer-LazyTensorStorage-RandomSampler-400] 0.1659s 31.1152ms 32.1386 Ops/s 35.4983 Ops/s $\textbf{\color{#d91a1a}-9.46\%}$
test_populate_rb[TensorDictReplayBuffer-ListStorage-SamplerWithoutReplacement-400] 0.1597s 30.4448ms 32.8463 Ops/s 31.8636 Ops/s $\color{#35bf28}+3.08\%$
test_populate_rb[TensorDictReplayBuffer-LazyMemmapStorage-SamplerWithoutReplacement-400] 0.1614s 31.1875ms 32.0641 Ops/s 32.3942 Ops/s $\color{#d91a1a}-1.02\%$
test_populate_rb[TensorDictReplayBuffer-LazyTensorStorage-SamplerWithoutReplacement-400] 0.1723s 33.7531ms 29.6269 Ops/s 32.4125 Ops/s $\textbf{\color{#d91a1a}-8.59\%}$
test_populate_rb[TensorDictPrioritizedReplayBuffer-ListStorage-None-400] 0.1701s 30.8963ms 32.3663 Ops/s 29.5771 Ops/s $\textbf{\color{#35bf28}+9.43\%}$
test_populate_rb[TensorDictPrioritizedReplayBuffer-LazyMemmapStorage-None-400] 0.1666s 30.8850ms 32.3782 Ops/s 32.0979 Ops/s $\color{#35bf28}+0.87\%$
test_populate_rb[TensorDictPrioritizedReplayBuffer-LazyTensorStorage-None-400] 0.1625s 30.9769ms 32.2821 Ops/s 36.0496 Ops/s $\textbf{\color{#d91a1a}-10.45\%}$

matteobettini
matteobettini approved these changes Jul 3, 2023
View reviewed changes
Copy link
Contributor

@matteobettini matteobettini left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM

@vmoens vmoens merged commit d12afa0 into main Jul 3, 2023
@vmoens vmoens deleted the less_warnings_cost branch July 3, 2023 16:32
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Reviewers

@matteobettini matteobettini matteobettini approved these changes

Assignees
No one assigned
Labels
CI Has to do with CI setup (e.g. wheels & builds, tests...) CLA Signed This label is managed by the Facebook bot. Authors need to sign the CLA before a PR can be reviewed.
Projects
None yet
Milestone
No milestone
Development

Successfully merging this pull request may close these issues.
3 participants
@vmoens @matteobettini @facebook-github-bot