Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

[CI] Update macos image #1849

Merged
merged 1 commit into from
Jan 30, 2024
Merged

[CI] Update macos image #1849

merged 1 commit into from
Jan 30, 2024

Conversation

vmoens
Copy link
Contributor

@vmoens vmoens commented Jan 30, 2024 •
edited
Loading

No description provided.

@pytorch-bot PyTorch Bot
Copy link

pytorch-bot bot commented Jan 30, 2024 •
edited
Loading

🔗 Helpful Links

🧪 See artifacts and rendered test results at hud.pytorch.org/pr/pytorch/rl/1849

Note: Links to docs will display an error until the docs builds have been completed.

❌ 3 New Failures, 1 Pending, 26 Unrelated Failures

As of commit cc1c5e0 with merge base b1cc796 (image):

NEW FAILURES - The following jobs have failed:

FLAKY - The following jobs failed but were likely due to flakiness present on trunk:

This comment was automatically generated by Dr. CI and updates every 15 minutes.

@facebook-github-bot facebook-github-bot added the CLA Signed This label is managed by the Facebook bot. Authors need to sign the CLA before a PR can be reviewed. label Jan 30, 2024
@vmoens vmoens added the CI Has to do with CI setup (e.g. wheels & builds, tests...) label Jan 30, 2024
@vmoens vmoens merged commit 017bcd0 into main Jan 30, 2024
14 of 33 checks passed
@vmoens vmoens deleted the macos-update branch January 30, 2024 21:39
@github-actions GitHub Actions
Copy link

$\color{#D29922}\textsf{\Large⚠\kern{0.2cm}\normalsize Warning}$ Result of CPU Benchmark Tests

Total Benchmarks: 89. Improved: $\large\color{#35bf28}7$. Worsened: $\large\color{#d91a1a}29$.

Expand to view detailed results
Name Max Mean Ops Ops on Repo HEAD Change
test_single 64.3602ms 63.2780ms 15.8033 Ops/s 15.6971 Ops/s $\color{#35bf28}+0.68\%$
test_sync 41.5966ms 35.9560ms 27.8117 Ops/s 29.0080 Ops/s $\color{#d91a1a}-4.12\%$
test_async 61.7389ms 33.9455ms 29.4590 Ops/s 29.3689 Ops/s $\color{#35bf28}+0.31\%$
test_simple 0.4255s 0.4232s 2.3627 Ops/s 2.2309 Ops/s $\textbf{\color{#35bf28}+5.91\%}$
test_transformed 0.6622s 0.6051s 1.6525 Ops/s 1.6296 Ops/s $\color{#35bf28}+1.41\%$
test_serial 1.4299s 1.3658s 0.7321 Ops/s 0.7306 Ops/s $\color{#35bf28}+0.21\%$
test_parallel 1.2807s 1.2234s 0.8174 Ops/s 0.8205 Ops/s $\color{#d91a1a}-0.37\%$
test_step_mdp_speed[True-True-True-True-True] 0.1988ms 21.0673μs 47.4670 KOps/s 46.7888 KOps/s $\color{#35bf28}+1.45\%$
test_step_mdp_speed[True-True-True-True-False] 37.7000μs 12.9523μs 77.2066 KOps/s 77.4922 KOps/s $\color{#d91a1a}-0.37\%$
test_step_mdp_speed[True-True-True-False-True] 37.8300μs 12.4887μs 80.0727 KOps/s 79.0152 KOps/s $\color{#35bf28}+1.34\%$
test_step_mdp_speed[True-True-True-False-False] 26.1880μs 7.6804μs 130.2016 KOps/s 130.8496 KOps/s $\color{#d91a1a}-0.50\%$
test_step_mdp_speed[True-True-False-True-True] 98.0920μs 22.6406μs 44.1684 KOps/s 43.7042 KOps/s $\color{#35bf28}+1.06\%$
test_step_mdp_speed[True-True-False-True-False] 36.0570μs 14.1440μs 70.7012 KOps/s 69.7914 KOps/s $\color{#35bf28}+1.30\%$
test_step_mdp_speed[True-True-False-False-True] 57.9380μs 13.6495μs 73.2625 KOps/s 72.1008 KOps/s $\color{#35bf28}+1.61\%$
test_step_mdp_speed[True-True-False-False-False] 51.9670μs 8.8141μs 113.4542 KOps/s 112.7903 KOps/s $\color{#35bf28}+0.59\%$
test_step_mdp_speed[True-False-True-True-True] 68.2270μs 23.8567μs 41.9169 KOps/s 41.6311 KOps/s $\color{#35bf28}+0.69\%$
test_step_mdp_speed[True-False-True-True-False] 44.0720μs 15.7417μs 63.5255 KOps/s 64.1924 KOps/s $\color{#d91a1a}-1.04\%$
test_step_mdp_speed[True-False-True-False-True] 38.3820μs 13.6628μs 73.1916 KOps/s 71.9366 KOps/s $\color{#35bf28}+1.74\%$
test_step_mdp_speed[True-False-True-False-False] 27.6920μs 8.9423μs 111.8277 KOps/s 112.3751 KOps/s $\color{#d91a1a}-0.49\%$
test_step_mdp_speed[True-False-False-True-True] 56.8960μs 25.0012μs 39.9981 KOps/s 39.5002 KOps/s $\color{#35bf28}+1.26\%$
test_step_mdp_speed[True-False-False-True-False] 54.3810μs 16.7521μs 59.6940 KOps/s 59.7409 KOps/s $\color{#d91a1a}-0.08\%$
test_step_mdp_speed[True-False-False-False-True] 47.6280μs 14.6030μs 68.4791 KOps/s 67.8152 KOps/s $\color{#35bf28}+0.98\%$
test_step_mdp_speed[True-False-False-False-False] 57.7480μs 9.9337μs 100.6678 KOps/s 100.4511 KOps/s $\color{#35bf28}+0.22\%$
test_step_mdp_speed[False-True-True-True-True] 48.9620μs 23.8167μs 41.9874 KOps/s 40.6729 KOps/s $\color{#35bf28}+3.23\%$
test_step_mdp_speed[False-True-True-True-False] 59.9420μs 15.6396μs 63.9402 KOps/s 64.7089 KOps/s $\color{#d91a1a}-1.19\%$
test_step_mdp_speed[False-True-True-False-True] 35.9470μs 15.8192μs 63.2142 KOps/s 57.4132 KOps/s $\textbf{\color{#35bf28}+10.10\%}$
test_step_mdp_speed[False-True-True-False-False] 39.2230μs 9.9701μs 100.2999 KOps/s 100.0473 KOps/s $\color{#35bf28}+0.25\%$
test_step_mdp_speed[False-True-False-True-True] 48.9410μs 24.7287μs 40.4388 KOps/s 39.7739 KOps/s $\color{#35bf28}+1.67\%$
test_step_mdp_speed[False-True-False-True-False] 51.9970μs 16.9378μs 59.0394 KOps/s 59.5069 KOps/s $\color{#d91a1a}-0.79\%$
test_step_mdp_speed[False-True-False-False-True] 53.8500μs 16.8959μs 59.1860 KOps/s 57.7146 KOps/s $\color{#35bf28}+2.55\%$
test_step_mdp_speed[False-True-False-False-False] 57.2670μs 11.2002μs 89.2839 KOps/s 88.4022 KOps/s $\color{#35bf28}+1.00\%$
test_step_mdp_speed[False-False-True-True-True] 60.3820μs 26.4238μs 37.8446 KOps/s 37.8227 KOps/s $\color{#35bf28}+0.06\%$
test_step_mdp_speed[False-False-True-True-False] 59.8510μs 18.1102μs 55.2176 KOps/s 55.4854 KOps/s $\color{#d91a1a}-0.48\%$
test_step_mdp_speed[False-False-True-False-True] 50.8950μs 17.1914μs 58.1686 KOps/s 57.9312 KOps/s $\color{#35bf28}+0.41\%$
test_step_mdp_speed[False-False-True-False-False] 36.7480μs 11.1984μs 89.2981 KOps/s 88.9755 KOps/s $\color{#35bf28}+0.36\%$
test_step_mdp_speed[False-False-False-True-True] 64.9810μs 27.7452μs 36.0422 KOps/s 36.5200 KOps/s $\color{#d91a1a}-1.31\%$
test_step_mdp_speed[False-False-False-True-False] 55.4130μs 19.4928μs 51.3009 KOps/s 51.9508 KOps/s $\color{#d91a1a}-1.25\%$
test_step_mdp_speed[False-False-False-False-True] 54.1810μs 17.9440μs 55.7288 KOps/s 54.6578 KOps/s $\color{#35bf28}+1.96\%$
test_step_mdp_speed[False-False-False-False-False] 43.4610μs 12.1695μs 82.1726 KOps/s 80.4021 KOps/s $\color{#35bf28}+2.20\%$
test_values[generalized_advantage_estimate-True-True] 21.6188ms 12.7805ms 78.2444 Ops/s 98.8129 Ops/s $\textbf{\color{#d91a1a}-20.82\%}$
test_values[vec_generalized_advantage_estimate-True-True] 34.5065ms 26.8609ms 37.2289 Ops/s 27.9891 Ops/s $\textbf{\color{#35bf28}+33.01\%}$
test_values[td0_return_estimate-False-False] 0.2734ms 0.1948ms 5.1327 KOps/s 5.0338 KOps/s $\color{#35bf28}+1.96\%$
test_values[td1_return_estimate-False-False] 29.5510ms 26.4584ms 37.7952 Ops/s 40.0019 Ops/s $\textbf{\color{#d91a1a}-5.52\%}$
test_values[vec_td1_return_estimate-False-False] 34.7134ms 26.8268ms 37.2761 Ops/s 28.1552 Ops/s $\textbf{\color{#35bf28}+32.40\%}$
test_values[td_lambda_return_estimate-True-False] 39.1473ms 36.6190ms 27.3082 Ops/s 28.4901 Ops/s $\color{#d91a1a}-4.15\%$
test_values[vec_td_lambda_return_estimate-True-False] 34.8143ms 26.8496ms 37.2446 Ops/s 28.0400 Ops/s $\textbf{\color{#35bf28}+32.83\%}$
test_gae_speed[generalized_advantage_estimate-False-1-512] 8.4236ms 8.3071ms 120.3783 Ops/s 116.6504 Ops/s $\color{#35bf28}+3.20\%$
test_gae_speed[vec_generalized_advantage_estimate-True-1-512] 10.2309ms 2.0229ms 494.3477 Ops/s 500.2337 Ops/s $\color{#d91a1a}-1.18\%$
test_gae_speed[vec_generalized_advantage_estimate-False-1-512] 0.6039ms 0.4396ms 2.2748 KOps/s 2.8001 KOps/s $\textbf{\color{#d91a1a}-18.76\%}$
test_gae_speed[vec_generalized_advantage_estimate-True-32-512] 46.3467ms 36.9144ms 27.0897 Ops/s 21.0153 Ops/s $\textbf{\color{#35bf28}+28.90\%}$
test_gae_speed[vec_generalized_advantage_estimate-False-32-512] 10.8726ms 2.7123ms 368.6873 Ops/s 326.2541 Ops/s $\textbf{\color{#35bf28}+13.01\%}$
test_dqn_speed 14.6022ms 7.6374ms 130.9342 Ops/s 171.0220 Ops/s $\textbf{\color{#d91a1a}-23.44\%}$
test_ddpg_speed 16.5001ms 14.7005ms 68.0249 Ops/s 86.7854 Ops/s $\textbf{\color{#d91a1a}-21.62\%}$
test_sac_speed 37.9870ms 29.8756ms 33.4722 Ops/s 41.4216 Ops/s $\textbf{\color{#d91a1a}-19.19\%}$
test_redq_speed 24.0006ms 15.7200ms 63.6132 Ops/s 69.4184 Ops/s $\textbf{\color{#d91a1a}-8.36\%}$
test_redq_deprec_speed 28.0241ms 26.1645ms 38.2197 Ops/s 44.2766 Ops/s $\textbf{\color{#d91a1a}-13.68\%}$
test_td3_speed 30.3894ms 20.8503ms 47.9610 Ops/s 60.2196 Ops/s $\textbf{\color{#d91a1a}-20.36\%}$
test_cql_speed 96.2868ms 89.1291ms 11.2197 Ops/s 13.5705 Ops/s $\textbf{\color{#d91a1a}-17.32\%}$
test_a2c_speed 35.1612ms 27.5627ms 36.2809 Ops/s 44.6828 Ops/s $\textbf{\color{#d91a1a}-18.80\%}$
test_ppo_speed 36.9120ms 27.9849ms 35.7336 Ops/s 44.9826 Ops/s $\textbf{\color{#d91a1a}-20.56\%}$
test_reinforce_speed 27.5620ms 26.4649ms 37.7860 Ops/s 46.6610 Ops/s $\textbf{\color{#d91a1a}-19.02\%}$
test_iql_speed 0.1653s 75.7232ms 13.2060 Ops/s 17.7705 Ops/s $\textbf{\color{#d91a1a}-25.69\%}$
test_rb_sample[TensorDictReplayBuffer-ListStorage-RandomSampler-4000] 3.1697ms 2.9366ms 340.5242 Ops/s 332.4044 Ops/s $\color{#35bf28}+2.44\%$
test_rb_sample[TensorDictReplayBuffer-LazyMemmapStorage-RandomSampler-10000] 0.9820ms 0.5328ms 1.8768 KOps/s 1.9220 KOps/s $\color{#d91a1a}-2.35\%$
test_rb_sample[TensorDictReplayBuffer-LazyTensorStorage-RandomSampler-10000] 96.8788ms 0.5772ms 1.7324 KOps/s 2.0350 KOps/s $\textbf{\color{#d91a1a}-14.87\%}$
test_rb_sample[TensorDictReplayBuffer-ListStorage-SamplerWithoutReplacement-4000] 3.3020ms 2.9851ms 335.0018 Ops/s 335.6222 Ops/s $\color{#d91a1a}-0.18\%$
test_rb_sample[TensorDictReplayBuffer-LazyMemmapStorage-SamplerWithoutReplacement-10000] 9.7770ms 0.5329ms 1.8764 KOps/s 1.9486 KOps/s $\color{#d91a1a}-3.70\%$
test_rb_sample[TensorDictReplayBuffer-LazyTensorStorage-SamplerWithoutReplacement-10000] 0.1066s 0.5867ms 1.7045 KOps/s 2.0323 KOps/s $\textbf{\color{#d91a1a}-16.13\%}$
test_rb_sample[TensorDictPrioritizedReplayBuffer-ListStorage-None-4000] 3.2520ms 2.7198ms 367.6786 Ops/s 372.2838 Ops/s $\color{#d91a1a}-1.24\%$
test_rb_sample[TensorDictPrioritizedReplayBuffer-LazyMemmapStorage-None-10000] 1.3171ms 0.6670ms 1.4993 KOps/s 1.5791 KOps/s $\textbf{\color{#d91a1a}-5.05\%}$
test_rb_sample[TensorDictPrioritizedReplayBuffer-LazyTensorStorage-None-10000] 9.0566ms 0.6488ms 1.5413 KOps/s 1.6469 KOps/s $\textbf{\color{#d91a1a}-6.42\%}$
test_rb_iterate[TensorDictReplayBuffer-ListStorage-RandomSampler-4000] 3.3214ms 2.9505ms 338.9310 Ops/s 341.2919 Ops/s $\color{#d91a1a}-0.69\%$
test_rb_iterate[TensorDictReplayBuffer-LazyMemmapStorage-RandomSampler-10000] 8.8926ms 0.5429ms 1.8418 KOps/s 1.9347 KOps/s $\color{#d91a1a}-4.80\%$
test_rb_iterate[TensorDictReplayBuffer-LazyTensorStorage-RandomSampler-10000] 0.6434ms 0.5068ms 1.9730 KOps/s 2.0245 KOps/s $\color{#d91a1a}-2.55\%$
test_rb_iterate[TensorDictReplayBuffer-ListStorage-SamplerWithoutReplacement-4000] 3.3652ms 2.9820ms 335.3457 Ops/s 326.3104 Ops/s $\color{#35bf28}+2.77\%$
test_rb_iterate[TensorDictReplayBuffer-LazyMemmapStorage-SamplerWithoutReplacement-10000] 8.8666ms 0.5427ms 1.8426 KOps/s 1.9301 KOps/s $\color{#d91a1a}-4.54\%$
test_rb_iterate[TensorDictReplayBuffer-LazyTensorStorage-SamplerWithoutReplacement-10000] 0.6281ms 0.5004ms 1.9984 KOps/s 2.0115 KOps/s $\color{#d91a1a}-0.65\%$
test_rb_iterate[TensorDictPrioritizedReplayBuffer-ListStorage-None-4000] 3.3828ms 2.8063ms 356.3379 Ops/s 358.0358 Ops/s $\color{#d91a1a}-0.47\%$
test_rb_iterate[TensorDictPrioritizedReplayBuffer-LazyMemmapStorage-None-10000] 9.0360ms 0.6905ms 1.4481 KOps/s 1.5707 KOps/s $\textbf{\color{#d91a1a}-7.81\%}$
test_rb_iterate[TensorDictPrioritizedReplayBuffer-LazyTensorStorage-None-10000] 0.8219ms 0.6453ms 1.5498 KOps/s 1.6362 KOps/s $\textbf{\color{#d91a1a}-5.28\%}$
test_rb_populate[TensorDictReplayBuffer-ListStorage-RandomSampler-400] 0.1183s 10.9652ms 91.1980 Ops/s 118.0432 Ops/s $\textbf{\color{#d91a1a}-22.74\%}$
test_rb_populate[TensorDictReplayBuffer-LazyMemmapStorage-RandomSampler-400] 16.3366ms 13.9416ms 71.7280 Ops/s 87.4538 Ops/s $\textbf{\color{#d91a1a}-17.98\%}$
test_rb_populate[TensorDictReplayBuffer-LazyTensorStorage-RandomSampler-400] 6.4362ms 3.4228ms 292.1595 Ops/s 396.8150 Ops/s $\textbf{\color{#d91a1a}-26.37\%}$
test_rb_populate[TensorDictReplayBuffer-ListStorage-SamplerWithoutReplacement-400] 0.1158s 13.1150ms 76.2484 Ops/s 95.7356 Ops/s $\textbf{\color{#d91a1a}-20.36\%}$
test_rb_populate[TensorDictReplayBuffer-LazyMemmapStorage-SamplerWithoutReplacement-400] 16.7008ms 14.0616ms 71.1159 Ops/s 88.0414 Ops/s $\textbf{\color{#d91a1a}-19.22\%}$
test_rb_populate[TensorDictReplayBuffer-LazyTensorStorage-SamplerWithoutReplacement-400] 6.8055ms 3.4065ms 293.5530 Ops/s 399.4907 Ops/s $\textbf{\color{#d91a1a}-26.52\%}$
test_rb_populate[TensorDictPrioritizedReplayBuffer-ListStorage-None-400] 0.1131s 12.9790ms 77.0474 Ops/s 115.2845 Ops/s $\textbf{\color{#d91a1a}-33.17\%}$
test_rb_populate[TensorDictPrioritizedReplayBuffer-LazyMemmapStorage-None-400] 21.0767ms 14.4066ms 69.4127 Ops/s 86.3950 Ops/s $\textbf{\color{#d91a1a}-19.66\%}$
test_rb_populate[TensorDictPrioritizedReplayBuffer-LazyTensorStorage-None-400] 6.8958ms 3.6444ms 274.3948 Ops/s 365.7517 Ops/s $\textbf{\color{#d91a1a}-24.98\%}$

@github-actions GitHub Actions
Copy link

$\color{#D29922}\textsf{\Large⚠\kern{0.2cm}\normalsize Warning}$ Result of GPU Benchmark Tests

Total Benchmarks: 92. Improved: $\large\color{#35bf28}3$. Worsened: $\large\color{#d91a1a}23$.

Expand to view detailed results
Name Max Mean Ops Ops on Repo HEAD Change
test_single 0.1219s 0.1195s 8.3686 Ops/s 8.3738 Ops/s $\color{#d91a1a}-0.06\%$
test_sync 97.0990ms 96.6104ms 10.3509 Ops/s 10.3716 Ops/s $\color{#d91a1a}-0.20\%$
test_async 0.1828s 92.2537ms 10.8397 Ops/s 10.8860 Ops/s $\color{#d91a1a}-0.43\%$
test_single_pixels 0.1277s 0.1276s 7.8373 Ops/s 7.1263 Ops/s $\textbf{\color{#35bf28}+9.98\%}$
test_sync_pixels 84.6139ms 82.4456ms 12.1292 Ops/s 12.7704 Ops/s $\textbf{\color{#d91a1a}-5.02\%}$
test_async_pixels 0.1427s 73.2895ms 13.6445 Ops/s 13.1678 Ops/s $\color{#35bf28}+3.62\%$
test_simple 0.9132s 0.8368s 1.1951 Ops/s 1.1885 Ops/s $\color{#35bf28}+0.55\%$
test_transformed 1.1553s 1.0910s 0.9166 Ops/s 0.9136 Ops/s $\color{#35bf28}+0.32\%$
test_serial 2.4104s 2.3400s 0.4274 Ops/s 0.4295 Ops/s $\color{#d91a1a}-0.51\%$
test_parallel 1.9921s 1.8647s 0.5363 Ops/s 0.5249 Ops/s $\color{#35bf28}+2.16\%$
test_step_mdp_speed[True-True-True-True-True] 0.1013ms 31.8777μs 31.3699 KOps/s 30.8888 KOps/s $\color{#35bf28}+1.56\%$
test_step_mdp_speed[True-True-True-True-False] 43.0700μs 19.0632μs 52.4572 KOps/s 50.8834 KOps/s $\color{#35bf28}+3.09\%$
test_step_mdp_speed[True-True-True-False-True] 96.1120μs 18.2836μs 54.6937 KOps/s 54.3800 KOps/s $\color{#35bf28}+0.58\%$
test_step_mdp_speed[True-True-True-False-False] 95.3410μs 10.8678μs 92.0148 KOps/s 89.1846 KOps/s $\color{#35bf28}+3.17\%$
test_step_mdp_speed[True-True-False-True-True] 65.9610μs 33.8582μs 29.5349 KOps/s 29.3175 KOps/s $\color{#35bf28}+0.74\%$
test_step_mdp_speed[True-True-False-True-False] 55.0410μs 21.2284μs 47.1066 KOps/s 46.9346 KOps/s $\color{#35bf28}+0.37\%$
test_step_mdp_speed[True-True-False-False-True] 0.1246ms 19.7659μs 50.5921 KOps/s 49.4669 KOps/s $\color{#35bf28}+2.27\%$
test_step_mdp_speed[True-True-False-False-False] 49.9910μs 12.8076μs 78.0786 KOps/s 76.6148 KOps/s $\color{#35bf28}+1.91\%$
test_step_mdp_speed[True-False-True-True-True] 61.1610μs 35.9781μs 27.7947 KOps/s 27.8414 KOps/s $\color{#d91a1a}-0.17\%$
test_step_mdp_speed[True-False-True-True-False] 44.9100μs 22.7326μs 43.9897 KOps/s 43.0706 KOps/s $\color{#35bf28}+2.13\%$
test_step_mdp_speed[True-False-True-False-True] 41.4010μs 19.7700μs 50.5817 KOps/s 49.4274 KOps/s $\color{#35bf28}+2.34\%$
test_step_mdp_speed[True-False-True-False-False] 31.8410μs 12.7205μs 78.6130 KOps/s 77.0801 KOps/s $\color{#35bf28}+1.99\%$
test_step_mdp_speed[True-False-False-True-True] 0.1119ms 37.9200μs 26.3713 KOps/s 26.6520 KOps/s $\color{#d91a1a}-1.05\%$
test_step_mdp_speed[True-False-False-True-False] 50.0610μs 24.3977μs 40.9875 KOps/s 40.3143 KOps/s $\color{#35bf28}+1.67\%$
test_step_mdp_speed[True-False-False-False-True] 46.2000μs 21.5617μs 46.3785 KOps/s 45.3170 KOps/s $\color{#35bf28}+2.34\%$
test_step_mdp_speed[True-False-False-False-False] 0.1892ms 14.5163μs 68.8883 KOps/s 67.7844 KOps/s $\color{#35bf28}+1.63\%$
test_step_mdp_speed[False-True-True-True-True] 0.2278ms 35.6441μs 28.0551 KOps/s 27.7465 KOps/s $\color{#35bf28}+1.11\%$
test_step_mdp_speed[False-True-True-True-False] 97.2230μs 22.6385μs 44.1726 KOps/s 43.1889 KOps/s $\color{#35bf28}+2.28\%$
test_step_mdp_speed[False-True-True-False-True] 46.0910μs 23.9858μs 41.6913 KOps/s 41.6633 KOps/s $\color{#35bf28}+0.07\%$
test_step_mdp_speed[False-True-True-False-False] 41.5610μs 14.7501μs 67.7963 KOps/s 67.8508 KOps/s $\color{#d91a1a}-0.08\%$
test_step_mdp_speed[False-True-False-True-True] 61.2020μs 37.5232μs 26.6502 KOps/s 26.2696 KOps/s $\color{#35bf28}+1.45\%$
test_step_mdp_speed[False-True-False-True-False] 51.0410μs 24.9003μs 40.1602 KOps/s 39.8267 KOps/s $\color{#35bf28}+0.84\%$
test_step_mdp_speed[False-True-False-False-True] 58.6610μs 26.0609μs 38.3717 KOps/s 38.7381 KOps/s $\color{#d91a1a}-0.95\%$
test_step_mdp_speed[False-True-False-False-False] 96.9120μs 16.2591μs 61.5041 KOps/s 60.6090 KOps/s $\color{#35bf28}+1.48\%$
test_step_mdp_speed[False-False-True-True-True] 70.1320μs 38.8763μs 25.7226 KOps/s 25.2781 KOps/s $\color{#35bf28}+1.76\%$
test_step_mdp_speed[False-False-True-True-False] 51.6500μs 26.6352μs 37.5444 KOps/s 37.2206 KOps/s $\color{#35bf28}+0.87\%$
test_step_mdp_speed[False-False-True-False-True] 43.6720μs 25.4541μs 39.2864 KOps/s 38.6495 KOps/s $\color{#35bf28}+1.65\%$
test_step_mdp_speed[False-False-True-False-False] 0.1968ms 16.2071μs 61.7014 KOps/s 60.7045 KOps/s $\color{#35bf28}+1.64\%$
test_step_mdp_speed[False-False-False-True-True] 70.6120μs 40.6938μs 24.5738 KOps/s 24.2332 KOps/s $\color{#35bf28}+1.41\%$
test_step_mdp_speed[False-False-False-True-False] 50.6920μs 28.2626μs 35.3824 KOps/s 34.9991 KOps/s $\color{#35bf28}+1.10\%$
test_step_mdp_speed[False-False-False-False-True] 51.6700μs 27.4549μs 36.4234 KOps/s 36.6040 KOps/s $\color{#d91a1a}-0.49\%$
test_step_mdp_speed[False-False-False-False-False] 0.1323ms 17.8679μs 55.9661 KOps/s 55.1010 KOps/s $\color{#35bf28}+1.57\%$
test_values[generalized_advantage_estimate-True-True] 25.8001ms 24.9135ms 40.1389 Ops/s 40.6229 Ops/s $\color{#d91a1a}-1.19\%$
test_values[vec_generalized_advantage_estimate-True-True] 85.7154ms 3.2733ms 305.4996 Ops/s 291.7825 Ops/s $\color{#35bf28}+4.70\%$
test_values[td0_return_estimate-False-False] 94.6430μs 60.8193μs 16.4421 KOps/s 16.6787 KOps/s $\color{#d91a1a}-1.42\%$
test_values[td1_return_estimate-False-False] 53.9676ms 53.1444ms 18.8167 Ops/s 18.8912 Ops/s $\color{#d91a1a}-0.39\%$
test_values[vec_td1_return_estimate-False-False] 1.9960ms 1.7624ms 567.4202 Ops/s 567.5718 Ops/s $\color{#d91a1a}-0.03\%$
test_values[td_lambda_return_estimate-True-False] 85.5310ms 84.3927ms 11.8494 Ops/s 11.8793 Ops/s $\color{#d91a1a}-0.25\%$
test_values[vec_td_lambda_return_estimate-True-False] 4.1561ms 1.7987ms 555.9567 Ops/s 554.7121 Ops/s $\color{#35bf28}+0.22\%$
test_gae_speed[generalized_advantage_estimate-False-1-512] 23.9938ms 23.4204ms 42.6978 Ops/s 42.7669 Ops/s $\color{#d91a1a}-0.16\%$
test_gae_speed[vec_generalized_advantage_estimate-True-1-512] 0.8763ms 0.7070ms 1.4145 KOps/s 1.4347 KOps/s $\color{#d91a1a}-1.41\%$
test_gae_speed[vec_generalized_advantage_estimate-False-1-512] 0.7127ms 0.6506ms 1.5372 KOps/s 1.5423 KOps/s $\color{#d91a1a}-0.33\%$
test_gae_speed[vec_generalized_advantage_estimate-True-32-512] 1.6027ms 1.4461ms 691.5035 Ops/s 690.6487 Ops/s $\color{#35bf28}+0.12\%$
test_gae_speed[vec_generalized_advantage_estimate-False-32-512] 0.9857ms 0.6688ms 1.4951 KOps/s 1.4978 KOps/s $\color{#d91a1a}-0.18\%$
test_dqn_speed 7.8244ms 7.3191ms 136.6282 Ops/s 136.6803 Ops/s $\color{#d91a1a}-0.04\%$
test_ddpg_speed 15.2076ms 14.3844ms 69.5198 Ops/s 71.4636 Ops/s $\color{#d91a1a}-2.72\%$
test_sac_speed 30.2432ms 29.0993ms 34.3650 Ops/s 35.9238 Ops/s $\color{#d91a1a}-4.34\%$
test_redq_speed 14.0939ms 13.2741ms 75.3346 Ops/s 85.5446 Ops/s $\textbf{\color{#d91a1a}-11.94\%}$
test_redq_deprec_speed 24.8683ms 24.0333ms 41.6090 Ops/s 40.7389 Ops/s $\color{#35bf28}+2.14\%$
test_td3_speed 30.1581ms 19.7793ms 50.5578 Ops/s 53.8262 Ops/s $\textbf{\color{#d91a1a}-6.07\%}$
test_cql_speed 83.2227ms 82.7787ms 12.0804 Ops/s 13.4098 Ops/s $\textbf{\color{#d91a1a}-9.91\%}$
test_a2c_speed 28.5647ms 26.9680ms 37.0810 Ops/s 39.3069 Ops/s $\textbf{\color{#d91a1a}-5.66\%}$
test_ppo_speed 28.1511ms 26.9653ms 37.0848 Ops/s 36.3508 Ops/s $\color{#35bf28}+2.02\%$
test_reinforce_speed 27.3178ms 26.1004ms 38.3135 Ops/s 41.6603 Ops/s $\textbf{\color{#d91a1a}-8.03\%}$
test_iql_speed 57.6643ms 57.0190ms 17.5380 Ops/s 20.0085 Ops/s $\textbf{\color{#d91a1a}-12.35\%}$
test_rb_sample[TensorDictReplayBuffer-ListStorage-RandomSampler-4000] 3.8330ms 3.6122ms 276.8423 Ops/s 274.5159 Ops/s $\color{#35bf28}+0.85\%$
test_rb_sample[TensorDictReplayBuffer-LazyMemmapStorage-RandomSampler-10000] 0.9887ms 0.8436ms 1.1855 KOps/s 1.8058 KOps/s $\textbf{\color{#d91a1a}-34.35\%}$
test_rb_sample[TensorDictReplayBuffer-LazyTensorStorage-RandomSampler-10000] 0.9968ms 0.8201ms 1.2193 KOps/s 1.8945 KOps/s $\textbf{\color{#d91a1a}-35.64\%}$
test_rb_sample[TensorDictReplayBuffer-ListStorage-SamplerWithoutReplacement-4000] 3.9332ms 3.6304ms 275.4502 Ops/s 271.6599 Ops/s $\color{#35bf28}+1.40\%$
test_rb_sample[TensorDictReplayBuffer-LazyMemmapStorage-SamplerWithoutReplacement-10000] 0.9861ms 0.8320ms 1.2019 KOps/s 1.8303 KOps/s $\textbf{\color{#d91a1a}-34.33\%}$
test_rb_sample[TensorDictReplayBuffer-LazyTensorStorage-SamplerWithoutReplacement-10000] 1.0192ms 0.8091ms 1.2359 KOps/s 1.9117 KOps/s $\textbf{\color{#d91a1a}-35.35\%}$
test_rb_sample[TensorDictPrioritizedReplayBuffer-ListStorage-None-4000] 3.5159ms 3.3093ms 302.1816 Ops/s 301.5922 Ops/s $\color{#35bf28}+0.20\%$
test_rb_sample[TensorDictPrioritizedReplayBuffer-LazyMemmapStorage-None-10000] 1.1380ms 0.9634ms 1.0380 KOps/s 1.4690 KOps/s $\textbf{\color{#d91a1a}-29.34\%}$
test_rb_sample[TensorDictPrioritizedReplayBuffer-LazyTensorStorage-None-10000] 1.1035ms 0.9394ms 1.0645 KOps/s 1.5267 KOps/s $\textbf{\color{#d91a1a}-30.27\%}$
test_rb_iterate[TensorDictReplayBuffer-ListStorage-RandomSampler-4000] 3.7527ms 3.5883ms 278.6822 Ops/s 274.4776 Ops/s $\color{#35bf28}+1.53\%$
test_rb_iterate[TensorDictReplayBuffer-LazyMemmapStorage-RandomSampler-10000] 1.0172ms 0.8440ms 1.1848 KOps/s 1.7972 KOps/s $\textbf{\color{#d91a1a}-34.08\%}$
test_rb_iterate[TensorDictReplayBuffer-LazyTensorStorage-RandomSampler-10000] 0.9688ms 0.8232ms 1.2148 KOps/s 1.8959 KOps/s $\textbf{\color{#d91a1a}-35.93\%}$
test_rb_iterate[TensorDictReplayBuffer-ListStorage-SamplerWithoutReplacement-4000] 4.0220ms 3.6501ms 273.9622 Ops/s 272.0410 Ops/s $\color{#35bf28}+0.71\%$
test_rb_iterate[TensorDictReplayBuffer-LazyMemmapStorage-SamplerWithoutReplacement-10000] 1.0160ms 0.8344ms 1.1984 KOps/s 1.8082 KOps/s $\textbf{\color{#d91a1a}-33.72\%}$
test_rb_iterate[TensorDictReplayBuffer-LazyTensorStorage-SamplerWithoutReplacement-10000] 0.9704ms 0.8113ms 1.2325 KOps/s 1.9063 KOps/s $\textbf{\color{#d91a1a}-35.35\%}$
test_rb_iterate[TensorDictPrioritizedReplayBuffer-ListStorage-None-4000] 3.9198ms 3.3676ms 296.9430 Ops/s 299.9092 Ops/s $\color{#d91a1a}-0.99\%$
test_rb_iterate[TensorDictPrioritizedReplayBuffer-LazyMemmapStorage-None-10000] 1.1050ms 0.9628ms 1.0386 KOps/s 1.4703 KOps/s $\textbf{\color{#d91a1a}-29.36\%}$
test_rb_iterate[TensorDictPrioritizedReplayBuffer-LazyTensorStorage-None-10000] 1.0805ms 0.9388ms 1.0652 KOps/s 1.5293 KOps/s $\textbf{\color{#d91a1a}-30.35\%}$
test_rb_populate[TensorDictReplayBuffer-ListStorage-RandomSampler-400] 0.1221s 10.1091ms 98.9209 Ops/s 93.4577 Ops/s $\textbf{\color{#35bf28}+5.85\%}$
test_rb_populate[TensorDictReplayBuffer-LazyMemmapStorage-RandomSampler-400] 0.1348s 16.2949ms 61.3690 Ops/s 73.2538 Ops/s $\textbf{\color{#d91a1a}-16.22\%}$
test_rb_populate[TensorDictReplayBuffer-LazyTensorStorage-RandomSampler-400] 6.1640ms 3.3669ms 297.0054 Ops/s 325.5392 Ops/s $\textbf{\color{#d91a1a}-8.77\%}$
test_rb_populate[TensorDictReplayBuffer-ListStorage-SamplerWithoutReplacement-400] 0.1227s 10.1774ms 98.2572 Ops/s 96.5467 Ops/s $\color{#35bf28}+1.77\%$
test_rb_populate[TensorDictReplayBuffer-LazyMemmapStorage-SamplerWithoutReplacement-400] 15.8891ms 13.8263ms 72.3259 Ops/s 62.4839 Ops/s $\textbf{\color{#35bf28}+15.75\%}$
test_rb_populate[TensorDictReplayBuffer-LazyTensorStorage-SamplerWithoutReplacement-400] 6.6459ms 3.3717ms 296.5834 Ops/s 325.2233 Ops/s $\textbf{\color{#d91a1a}-8.81\%}$
test_rb_populate[TensorDictPrioritizedReplayBuffer-ListStorage-None-400] 0.1244s 10.3792ms 96.3467 Ops/s 94.7108 Ops/s $\color{#35bf28}+1.73\%$
test_rb_populate[TensorDictPrioritizedReplayBuffer-LazyMemmapStorage-None-400] 16.6171ms 14.0527ms 71.1608 Ops/s 72.4090 Ops/s $\color{#d91a1a}-1.72\%$
test_rb_populate[TensorDictPrioritizedReplayBuffer-LazyTensorStorage-None-400] 7.4011ms 3.5766ms 279.5922 Ops/s 302.8701 Ops/s $\textbf{\color{#d91a1a}-7.69\%}$

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Reviewers
No reviews
Assignees
No one assigned
Labels
CI Has to do with CI setup (e.g. wheels & builds, tests...) CLA Signed This label is managed by the Facebook bot. Authors need to sign the CLA before a PR can be reviewed.
Projects
None yet
Milestone
No milestone
Development

Successfully merging this pull request may close these issues.
2 participants
@vmoens @facebook-github-bot