Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

[CI] Remove stable GPU tests from CI #1792

Merged
merged 7 commits into from
Jan 12, 2024
Merged

[CI] Remove stable GPU tests from CI #1792

merged 7 commits into from
Jan 12, 2024

Conversation

vmoens
Copy link
Contributor

@vmoens vmoens commented Jan 12, 2024 •
edited
Loading

No description provided.

@pytorch-bot PyTorch Bot
Copy link

pytorch-bot bot commented Jan 12, 2024 •
edited
Loading

🔗 Helpful Links

🧪 See artifacts and rendered test results at hud.pytorch.org/pr/pytorch/rl/1792

Note: Links to docs will display an error until the docs builds have been completed.

⏳ 1 Pending, 10 Unrelated Failures

As of commit fd75161 with merge base 6667dd8 (image):

FLAKY - The following jobs failed but were likely due to flakiness present on trunk:

BROKEN TRUNK - The following job failed but were present on the merge base:

👉 Rebase onto the `viable/strict` branch to avoid these failures

This comment was automatically generated by Dr. CI and updates every 15 minutes.

@facebook-github-bot facebook-github-bot added the CLA Signed This label is managed by the Facebook bot. Authors need to sign the CLA before a PR can be reviewed. label Jan 12, 2024
@vmoens vmoens added the CI Has to do with CI setup (e.g. wheels & builds, tests...) label Jan 12, 2024
@vmoens vmoens merged commit 1e508b5 into main Jan 12, 2024
39 of 48 checks passed
@vmoens vmoens deleted the refactor-tests2 branch January 12, 2024 17:29
@github-actions GitHub Actions
Copy link

$\color{#D29922}\textsf{\Large⚠\kern{0.2cm}\normalsize Warning}$ Result of CPU Benchmark Tests

Total Benchmarks: 89. Improved: $\large\color{#35bf28}2$. Worsened: $\large\color{#d91a1a}3$.

Expand to view detailed results
Name Max Mean Ops Ops on Repo HEAD Change
test_single 63.3349ms 63.0313ms 15.8651 Ops/s 15.3688 Ops/s $\color{#35bf28}+3.23\%$
test_sync 42.0006ms 35.3610ms 28.2798 Ops/s 28.5828 Ops/s $\color{#d91a1a}-1.06\%$
test_async 0.1324s 34.6647ms 28.8478 Ops/s 28.2276 Ops/s $\color{#35bf28}+2.20\%$
test_simple 0.5056s 0.4457s 2.2438 Ops/s 2.1825 Ops/s $\color{#35bf28}+2.80\%$
test_transformed 0.6817s 0.6122s 1.6335 Ops/s 1.6065 Ops/s $\color{#35bf28}+1.68\%$
test_serial 1.4265s 1.3622s 0.7341 Ops/s 0.7121 Ops/s $\color{#35bf28}+3.08\%$
test_parallel 1.4251s 1.3592s 0.7357 Ops/s 0.7353 Ops/s $\color{#35bf28}+0.06\%$
test_step_mdp_speed[True-True-True-True-True] 52.9790μs 22.0299μs 45.3928 KOps/s 45.8597 KOps/s $\color{#d91a1a}-1.02\%$
test_step_mdp_speed[True-True-True-True-False] 44.3430μs 13.5383μs 73.8643 KOps/s 73.3210 KOps/s $\color{#35bf28}+0.74\%$
test_step_mdp_speed[True-True-True-False-True] 62.3170μs 12.9188μs 77.4065 KOps/s 77.8166 KOps/s $\color{#d91a1a}-0.53\%$
test_step_mdp_speed[True-True-True-False-False] 39.2540μs 7.9565μs 125.6828 KOps/s 126.6725 KOps/s $\color{#d91a1a}-0.78\%$
test_step_mdp_speed[True-True-False-True-True] 71.9120μs 23.4251μs 42.6893 KOps/s 43.2829 KOps/s $\color{#d91a1a}-1.37\%$
test_step_mdp_speed[True-True-False-True-False] 80.3610μs 14.8360μs 67.4036 KOps/s 68.4480 KOps/s $\color{#d91a1a}-1.53\%$
test_step_mdp_speed[True-True-False-False-True] 56.2260μs 14.2235μs 70.3060 KOps/s 70.7169 KOps/s $\color{#d91a1a}-0.58\%$
test_step_mdp_speed[True-True-False-False-False] 42.4290μs 9.2261μs 108.3881 KOps/s 108.9794 KOps/s $\color{#d91a1a}-0.54\%$
test_step_mdp_speed[True-False-True-True-True] 44.9440μs 24.7442μs 40.4135 KOps/s 40.2768 KOps/s $\color{#35bf28}+0.34\%$
test_step_mdp_speed[True-False-True-True-False] 44.2730μs 16.1343μs 61.9799 KOps/s 62.3133 KOps/s $\color{#d91a1a}-0.54\%$
test_step_mdp_speed[True-False-True-False-True] 39.6040μs 14.3821μs 69.5308 KOps/s 69.6557 KOps/s $\color{#d91a1a}-0.18\%$
test_step_mdp_speed[True-False-True-False-False] 38.2910μs 9.2725μs 107.8454 KOps/s 109.7252 KOps/s $\color{#d91a1a}-1.71\%$
test_step_mdp_speed[True-False-False-True-True] 80.3110μs 26.2205μs 38.1381 KOps/s 38.4967 KOps/s $\color{#d91a1a}-0.93\%$
test_step_mdp_speed[True-False-False-True-False] 77.9160μs 17.4690μs 57.2442 KOps/s 57.2739 KOps/s $\color{#d91a1a}-0.05\%$
test_step_mdp_speed[True-False-False-False-True] 52.3680μs 15.3427μs 65.1775 KOps/s 65.2207 KOps/s $\color{#d91a1a}-0.07\%$
test_step_mdp_speed[True-False-False-False-False] 35.0760μs 10.4351μs 95.8301 KOps/s 95.7314 KOps/s $\color{#35bf28}+0.10\%$
test_step_mdp_speed[False-True-True-True-True] 58.2590μs 25.0584μs 39.9068 KOps/s 40.3070 KOps/s $\color{#d91a1a}-0.99\%$
test_step_mdp_speed[False-True-True-True-False] 50.2540μs 16.1706μs 61.8405 KOps/s 62.0573 KOps/s $\color{#d91a1a}-0.35\%$
test_step_mdp_speed[False-True-True-False-True] 2.8583ms 16.9876μs 58.8665 KOps/s 60.4819 KOps/s $\color{#d91a1a}-2.67\%$
test_step_mdp_speed[False-True-True-False-False] 38.0710μs 10.5457μs 94.8251 KOps/s 95.6810 KOps/s $\color{#d91a1a}-0.89\%$
test_step_mdp_speed[False-True-False-True-True] 61.3950μs 25.9904μs 38.4757 KOps/s 38.7417 KOps/s $\color{#d91a1a}-0.69\%$
test_step_mdp_speed[False-True-False-True-False] 41.6180μs 17.3899μs 57.5045 KOps/s 57.7886 KOps/s $\color{#d91a1a}-0.49\%$
test_step_mdp_speed[False-True-False-False-True] 47.6190μs 17.6854μs 56.5440 KOps/s 56.1795 KOps/s $\color{#35bf28}+0.65\%$
test_step_mdp_speed[False-True-False-False-False] 32.8520μs 11.6461μs 85.8656 KOps/s 86.0120 KOps/s $\color{#d91a1a}-0.17\%$
test_step_mdp_speed[False-False-True-True-True] 73.8680μs 27.2966μs 36.6346 KOps/s 36.2448 KOps/s $\color{#35bf28}+1.08\%$
test_step_mdp_speed[False-False-True-True-False] 54.5720μs 18.7239μs 53.4075 KOps/s 53.7396 KOps/s $\color{#d91a1a}-0.62\%$
test_step_mdp_speed[False-False-True-False-True] 45.2950μs 17.9013μs 55.8620 KOps/s 55.5853 KOps/s $\color{#35bf28}+0.50\%$
test_step_mdp_speed[False-False-True-False-False] 37.3700μs 11.7078μs 85.4134 KOps/s 86.1429 KOps/s $\color{#d91a1a}-0.85\%$
test_step_mdp_speed[False-False-False-True-True] 57.6580μs 28.2317μs 35.4211 KOps/s 35.6515 KOps/s $\color{#d91a1a}-0.65\%$
test_step_mdp_speed[False-False-False-True-False] 54.1720μs 19.6021μs 51.0148 KOps/s 51.3406 KOps/s $\color{#d91a1a}-0.63\%$
test_step_mdp_speed[False-False-False-False-True] 45.6750μs 18.7918μs 53.2148 KOps/s 53.5057 KOps/s $\color{#d91a1a}-0.54\%$
test_step_mdp_speed[False-False-False-False-False] 36.3780μs 12.8281μs 77.9541 KOps/s 79.2469 KOps/s $\color{#d91a1a}-1.63\%$
test_values[generalized_advantage_estimate-True-True] 15.7020ms 12.1764ms 82.1262 Ops/s 81.8505 Ops/s $\color{#35bf28}+0.34\%$
test_values[vec_generalized_advantage_estimate-True-True] 35.2972ms 27.7904ms 35.9836 Ops/s 35.9063 Ops/s $\color{#35bf28}+0.22\%$
test_values[td0_return_estimate-False-False] 8.4336ms 0.3551ms 2.8160 KOps/s 5.3751 KOps/s $\textbf{\color{#d91a1a}-47.61\%}$
test_values[td1_return_estimate-False-False] 26.4984ms 25.4580ms 39.2804 Ops/s 38.3568 Ops/s $\color{#35bf28}+2.41\%$
test_values[vec_td1_return_estimate-False-False] 35.8500ms 27.9896ms 35.7276 Ops/s 35.8044 Ops/s $\color{#d91a1a}-0.21\%$
test_values[td_lambda_return_estimate-True-False] 38.8381ms 35.8087ms 27.9262 Ops/s 27.7095 Ops/s $\color{#35bf28}+0.78\%$
test_values[vec_td_lambda_return_estimate-True-False] 37.4651ms 28.0791ms 35.6137 Ops/s 35.6812 Ops/s $\color{#d91a1a}-0.19\%$
test_gae_speed[generalized_advantage_estimate-False-1-512] 8.1215ms 7.9747ms 125.3970 Ops/s 124.6463 Ops/s $\color{#35bf28}+0.60\%$
test_gae_speed[vec_generalized_advantage_estimate-True-1-512] 2.2734ms 1.8512ms 540.1981 Ops/s 530.3994 Ops/s $\color{#35bf28}+1.85\%$
test_gae_speed[vec_generalized_advantage_estimate-False-1-512] 8.6090ms 0.4393ms 2.2765 KOps/s 2.2778 KOps/s $\color{#d91a1a}-0.05\%$
test_gae_speed[vec_generalized_advantage_estimate-True-32-512] 46.3367ms 39.9075ms 25.0579 Ops/s 25.7529 Ops/s $\color{#d91a1a}-2.70\%$
test_gae_speed[vec_generalized_advantage_estimate-False-32-512] 10.9690ms 2.6368ms 379.2514 Ops/s 376.7631 Ops/s $\color{#35bf28}+0.66\%$
test_dqn_speed 85.5667ms 8.1227ms 123.1119 Ops/s 117.0026 Ops/s $\textbf{\color{#35bf28}+5.22\%}$
test_ddpg_speed 22.0817ms 14.6050ms 68.4695 Ops/s 67.3845 Ops/s $\color{#35bf28}+1.61\%$
test_sac_speed 34.8146ms 29.6597ms 33.7158 Ops/s 33.0725 Ops/s $\color{#35bf28}+1.95\%$
test_redq_speed 71.6173ms 38.6987ms 25.8407 Ops/s 27.6615 Ops/s $\textbf{\color{#d91a1a}-6.58\%}$
test_redq_deprec_speed 34.3198ms 26.2686ms 38.0683 Ops/s 37.7190 Ops/s $\color{#35bf28}+0.93\%$
test_td3_speed 29.1377ms 20.4738ms 48.8428 Ops/s 48.0293 Ops/s $\color{#35bf28}+1.69\%$
test_cql_speed 95.6935ms 88.0525ms 11.3569 Ops/s 11.1819 Ops/s $\color{#35bf28}+1.56\%$
test_a2c_speed 35.6958ms 26.9418ms 37.1171 Ops/s 36.2127 Ops/s $\color{#35bf28}+2.50\%$
test_ppo_speed 27.8791ms 27.0073ms 37.0271 Ops/s 35.9133 Ops/s $\color{#35bf28}+3.10\%$
test_reinforce_speed 27.4445ms 26.1560ms 38.2321 Ops/s 34.2458 Ops/s $\textbf{\color{#35bf28}+11.64\%}$
test_iql_speed 73.4090ms 65.1985ms 15.3378 Ops/s 15.2808 Ops/s $\color{#35bf28}+0.37\%$
test_rb_sample[TensorDictReplayBuffer-ListStorage-RandomSampler-4000] 2.2172ms 1.5199ms 657.9395 Ops/s 654.1611 Ops/s $\color{#35bf28}+0.58\%$
test_rb_sample[TensorDictReplayBuffer-LazyMemmapStorage-RandomSampler-10000] 10.0269ms 0.5195ms 1.9251 KOps/s 1.9091 KOps/s $\color{#35bf28}+0.84\%$
test_rb_sample[TensorDictReplayBuffer-LazyTensorStorage-RandomSampler-10000] 1.5706ms 0.4978ms 2.0089 KOps/s 1.9938 KOps/s $\color{#35bf28}+0.76\%$
test_rb_sample[TensorDictReplayBuffer-ListStorage-SamplerWithoutReplacement-4000] 2.1232ms 1.5291ms 653.9682 Ops/s 657.5826 Ops/s $\color{#d91a1a}-0.55\%$
test_rb_sample[TensorDictReplayBuffer-LazyMemmapStorage-SamplerWithoutReplacement-10000] 0.6253ms 0.5054ms 1.9787 KOps/s 1.9580 KOps/s $\color{#35bf28}+1.06\%$
test_rb_sample[TensorDictReplayBuffer-LazyTensorStorage-SamplerWithoutReplacement-10000] 0.1124s 0.5898ms 1.6956 KOps/s 1.6788 KOps/s $\color{#35bf28}+1.00\%$
test_rb_sample[TensorDictPrioritizedReplayBuffer-ListStorage-None-4000] 2.7430ms 1.7554ms 569.6682 Ops/s 567.2884 Ops/s $\color{#35bf28}+0.42\%$
test_rb_sample[TensorDictPrioritizedReplayBuffer-LazyMemmapStorage-None-10000] 8.9278ms 0.6525ms 1.5327 KOps/s 1.5107 KOps/s $\color{#35bf28}+1.45\%$
test_rb_sample[TensorDictPrioritizedReplayBuffer-LazyTensorStorage-None-10000] 9.1850ms 0.6392ms 1.5644 KOps/s 1.5272 KOps/s $\color{#35bf28}+2.44\%$
test_rb_iterate[TensorDictReplayBuffer-ListStorage-RandomSampler-4000] 1.9245ms 1.5344ms 651.7194 Ops/s 641.3558 Ops/s $\color{#35bf28}+1.62\%$
test_rb_iterate[TensorDictReplayBuffer-LazyMemmapStorage-RandomSampler-10000] 0.1092s 0.6439ms 1.5531 KOps/s 1.5421 KOps/s $\color{#35bf28}+0.72\%$
test_rb_iterate[TensorDictReplayBuffer-LazyTensorStorage-RandomSampler-10000] 9.8241ms 0.5101ms 1.9603 KOps/s 1.9513 KOps/s $\color{#35bf28}+0.46\%$
test_rb_iterate[TensorDictReplayBuffer-ListStorage-SamplerWithoutReplacement-4000] 2.0015ms 1.5068ms 663.6575 Ops/s 661.5109 Ops/s $\color{#35bf28}+0.32\%$
test_rb_iterate[TensorDictReplayBuffer-LazyMemmapStorage-SamplerWithoutReplacement-10000] 1.6077ms 0.5147ms 1.9428 KOps/s 1.9576 KOps/s $\color{#d91a1a}-0.75\%$
test_rb_iterate[TensorDictReplayBuffer-LazyTensorStorage-SamplerWithoutReplacement-10000] 8.8119ms 0.5101ms 1.9603 KOps/s 2.0070 KOps/s $\color{#d91a1a}-2.33\%$
test_rb_iterate[TensorDictPrioritizedReplayBuffer-ListStorage-None-4000] 2.6444ms 1.7597ms 568.2938 Ops/s 569.7295 Ops/s $\color{#d91a1a}-0.25\%$
test_rb_iterate[TensorDictPrioritizedReplayBuffer-LazyMemmapStorage-None-10000] 0.7757ms 0.6452ms 1.5500 KOps/s 1.5239 KOps/s $\color{#35bf28}+1.71\%$
test_rb_iterate[TensorDictPrioritizedReplayBuffer-LazyTensorStorage-None-10000] 8.9458ms 0.6466ms 1.5465 KOps/s 1.5388 KOps/s $\color{#35bf28}+0.50\%$
test_rb_populate[TensorDictReplayBuffer-ListStorage-RandomSampler-400] 0.1191s 17.7692ms 56.2773 Ops/s 55.9548 Ops/s $\color{#35bf28}+0.58\%$
test_rb_populate[TensorDictReplayBuffer-LazyMemmapStorage-RandomSampler-400] 15.6442ms 13.5282ms 73.9196 Ops/s 73.4965 Ops/s $\color{#35bf28}+0.58\%$
test_rb_populate[TensorDictReplayBuffer-LazyTensorStorage-RandomSampler-400] 5.4966ms 3.2610ms 306.6568 Ops/s 301.5734 Ops/s $\color{#35bf28}+1.69\%$
test_rb_populate[TensorDictReplayBuffer-ListStorage-SamplerWithoutReplacement-400] 0.1179s 17.7419ms 56.3638 Ops/s 56.1131 Ops/s $\color{#35bf28}+0.45\%$
test_rb_populate[TensorDictReplayBuffer-LazyMemmapStorage-SamplerWithoutReplacement-400] 22.8804ms 13.7085ms 72.9477 Ops/s 72.6399 Ops/s $\color{#35bf28}+0.42\%$
test_rb_populate[TensorDictReplayBuffer-LazyTensorStorage-SamplerWithoutReplacement-400] 5.7107ms 3.2637ms 306.3982 Ops/s 300.0713 Ops/s $\color{#35bf28}+2.11\%$
test_rb_populate[TensorDictPrioritizedReplayBuffer-ListStorage-None-400] 0.1193s 17.8279ms 56.0919 Ops/s 55.5747 Ops/s $\color{#35bf28}+0.93\%$
test_rb_populate[TensorDictPrioritizedReplayBuffer-LazyMemmapStorage-None-400] 0.1178s 16.0416ms 62.3380 Ops/s 71.2025 Ops/s $\textbf{\color{#d91a1a}-12.45\%}$
test_rb_populate[TensorDictPrioritizedReplayBuffer-LazyTensorStorage-None-400] 5.6689ms 3.5337ms 282.9898 Ops/s 284.3897 Ops/s $\color{#d91a1a}-0.49\%$

@github-actions GitHub Actions
Copy link

$\color{#D29922}\textsf{\Large⚠\kern{0.2cm}\normalsize Warning}$ Result of GPU Benchmark Tests

Total Benchmarks: 92. Improved: $\large\color{#35bf28}5$. Worsened: $\large\color{#d91a1a}9$.

Expand to view detailed results
Name Max Mean Ops Ops on Repo HEAD Change
test_single 0.1222s 0.1202s 8.3211 Ops/s 8.2213 Ops/s $\color{#35bf28}+1.21\%$
test_sync 0.1809s 0.1151s 8.6845 Ops/s 9.0924 Ops/s $\color{#d91a1a}-4.49\%$
test_async 0.2704s 99.0225ms 10.0987 Ops/s 10.1444 Ops/s $\color{#d91a1a}-0.45\%$
test_single_pixels 0.1315s 0.1311s 7.6266 Ops/s 7.6226 Ops/s $\color{#35bf28}+0.05\%$
test_sync_pixels 96.2275ms 94.9495ms 10.5319 Ops/s 10.5449 Ops/s $\color{#d91a1a}-0.12\%$
test_async_pixels 0.1763s 89.2993ms 11.1983 Ops/s 10.8469 Ops/s $\color{#35bf28}+3.24\%$
test_simple 0.9479s 0.8778s 1.1393 Ops/s 1.1328 Ops/s $\color{#35bf28}+0.57\%$
test_transformed 1.1949s 1.1262s 0.8880 Ops/s 0.8889 Ops/s $\color{#d91a1a}-0.10\%$
test_serial 2.5274s 2.4576s 0.4069 Ops/s 0.4026 Ops/s $\color{#35bf28}+1.07\%$
test_parallel 2.5346s 2.4674s 0.4053 Ops/s 0.4043 Ops/s $\color{#35bf28}+0.25\%$
test_step_mdp_speed[True-True-True-True-True] 0.1783ms 33.6693μs 29.7006 KOps/s 29.7721 KOps/s $\color{#d91a1a}-0.24\%$
test_step_mdp_speed[True-True-True-True-False] 42.7810μs 20.2579μs 49.3635 KOps/s 50.8491 KOps/s $\color{#d91a1a}-2.92\%$
test_step_mdp_speed[True-True-True-False-True] 38.5500μs 19.0728μs 52.4306 KOps/s 52.4871 KOps/s $\color{#d91a1a}-0.11\%$
test_step_mdp_speed[True-True-True-False-False] 31.3610μs 11.6516μs 85.8252 KOps/s 88.8768 KOps/s $\color{#d91a1a}-3.43\%$
test_step_mdp_speed[True-True-False-True-True] 67.7410μs 34.9220μs 28.6353 KOps/s 28.2359 KOps/s $\color{#35bf28}+1.41\%$
test_step_mdp_speed[True-True-False-True-False] 60.6910μs 21.7486μs 45.9799 KOps/s 45.9061 KOps/s $\color{#35bf28}+0.16\%$
test_step_mdp_speed[True-True-False-False-True] 45.0310μs 20.8021μs 48.0721 KOps/s 46.7343 KOps/s $\color{#35bf28}+2.86\%$
test_step_mdp_speed[True-True-False-False-False] 31.9800μs 13.1637μs 75.9663 KOps/s 74.4696 KOps/s $\color{#35bf28}+2.01\%$
test_step_mdp_speed[True-False-True-True-True] 58.4610μs 37.2779μs 26.8256 KOps/s 26.6878 KOps/s $\color{#35bf28}+0.52\%$
test_step_mdp_speed[True-False-True-True-False] 40.2400μs 23.4750μs 42.5985 KOps/s 41.8552 KOps/s $\color{#35bf28}+1.78\%$
test_step_mdp_speed[True-False-True-False-True] 44.5100μs 20.6391μs 48.4518 KOps/s 45.9793 KOps/s $\textbf{\color{#35bf28}+5.38\%}$
test_step_mdp_speed[True-False-True-False-False] 30.4410μs 13.1773μs 75.8881 KOps/s 75.7634 KOps/s $\color{#35bf28}+0.16\%$
test_step_mdp_speed[True-False-False-True-True] 0.1269ms 38.7460μs 25.8091 KOps/s 25.6223 KOps/s $\color{#35bf28}+0.73\%$
test_step_mdp_speed[True-False-False-True-False] 40.8210μs 25.1594μs 39.7465 KOps/s 39.1081 KOps/s $\color{#35bf28}+1.63\%$
test_step_mdp_speed[True-False-False-False-True] 41.2800μs 22.3822μs 44.6783 KOps/s 45.0413 KOps/s $\color{#d91a1a}-0.81\%$
test_step_mdp_speed[True-False-False-False-False] 68.8210μs 15.0601μs 66.4005 KOps/s 65.2266 KOps/s $\color{#35bf28}+1.80\%$
test_step_mdp_speed[False-True-True-True-True] 62.7100μs 37.3425μs 26.7791 KOps/s 26.5733 KOps/s $\color{#35bf28}+0.77\%$
test_step_mdp_speed[False-True-True-True-False] 41.3700μs 23.5985μs 42.3756 KOps/s 42.4272 KOps/s $\color{#d91a1a}-0.12\%$
test_step_mdp_speed[False-True-True-False-True] 70.8810μs 26.1641μs 38.2203 KOps/s 40.5288 KOps/s $\textbf{\color{#d91a1a}-5.70\%}$
test_step_mdp_speed[False-True-True-False-False] 33.4410μs 14.9819μs 66.7474 KOps/s 67.1419 KOps/s $\color{#d91a1a}-0.59\%$
test_step_mdp_speed[False-True-False-True-True] 0.1298ms 38.9181μs 25.6950 KOps/s 25.4157 KOps/s $\color{#35bf28}+1.10\%$
test_step_mdp_speed[False-True-False-True-False] 50.8400μs 25.4337μs 39.3180 KOps/s 38.8559 KOps/s $\color{#35bf28}+1.19\%$
test_step_mdp_speed[False-True-False-False-True] 47.3710μs 26.5153μs 37.7141 KOps/s 37.2099 KOps/s $\color{#35bf28}+1.36\%$
test_step_mdp_speed[False-True-False-False-False] 35.3700μs 16.7443μs 59.7219 KOps/s 59.4431 KOps/s $\color{#35bf28}+0.47\%$
test_step_mdp_speed[False-False-True-True-True] 63.2110μs 40.6281μs 24.6135 KOps/s 24.0002 KOps/s $\color{#35bf28}+2.56\%$
test_step_mdp_speed[False-False-True-True-False] 46.2710μs 27.4469μs 36.4340 KOps/s 36.1148 KOps/s $\color{#35bf28}+0.88\%$
test_step_mdp_speed[False-False-True-False-True] 51.9510μs 26.4330μs 37.8314 KOps/s 37.5479 KOps/s $\color{#35bf28}+0.76\%$
test_step_mdp_speed[False-False-True-False-False] 40.6600μs 16.8762μs 59.2549 KOps/s 59.5087 KOps/s $\color{#d91a1a}-0.43\%$
test_step_mdp_speed[False-False-False-True-True] 0.2359ms 41.8474μs 23.8963 KOps/s 23.5258 KOps/s $\color{#35bf28}+1.58\%$
test_step_mdp_speed[False-False-False-True-False] 0.1985ms 29.2904μs 34.1409 KOps/s 33.9531 KOps/s $\color{#35bf28}+0.55\%$
test_step_mdp_speed[False-False-False-False-True] 0.2029ms 27.7678μs 36.0129 KOps/s 35.7370 KOps/s $\color{#35bf28}+0.77\%$
test_step_mdp_speed[False-False-False-False-False] 0.1364ms 18.6527μs 53.6115 KOps/s 54.2533 KOps/s $\color{#d91a1a}-1.18\%$
test_values[generalized_advantage_estimate-True-True] 25.7194ms 25.3879ms 39.3889 Ops/s 43.3776 Ops/s $\textbf{\color{#d91a1a}-9.20\%}$
test_values[vec_generalized_advantage_estimate-True-True] 84.3842ms 3.2471ms 307.9681 Ops/s 291.1421 Ops/s $\textbf{\color{#35bf28}+5.78\%}$
test_values[td0_return_estimate-False-False] 96.3020μs 61.7848μs 16.1852 KOps/s 16.6285 KOps/s $\color{#d91a1a}-2.67\%$
test_values[td1_return_estimate-False-False] 54.9621ms 54.4213ms 18.3752 Ops/s 20.0999 Ops/s $\textbf{\color{#d91a1a}-8.58\%}$
test_values[vec_td1_return_estimate-False-False] 2.0687ms 1.7605ms 568.0263 Ops/s 574.6022 Ops/s $\color{#d91a1a}-1.14\%$
test_values[td_lambda_return_estimate-True-False] 87.6227ms 87.1977ms 11.4682 Ops/s 12.5155 Ops/s $\textbf{\color{#d91a1a}-8.37\%}$
test_values[vec_td_lambda_return_estimate-True-False] 2.1244ms 1.7784ms 562.3175 Ops/s 576.0941 Ops/s $\color{#d91a1a}-2.39\%$
test_gae_speed[generalized_advantage_estimate-False-1-512] 24.3000ms 23.9755ms 41.7093 Ops/s 45.9551 Ops/s $\textbf{\color{#d91a1a}-9.24\%}$
test_gae_speed[vec_generalized_advantage_estimate-True-1-512] 0.8589ms 0.6916ms 1.4460 KOps/s 1.4731 KOps/s $\color{#d91a1a}-1.84\%$
test_gae_speed[vec_generalized_advantage_estimate-False-1-512] 0.8247ms 0.6578ms 1.5202 KOps/s 1.5865 KOps/s $\color{#d91a1a}-4.18\%$
test_gae_speed[vec_generalized_advantage_estimate-True-32-512] 1.6602ms 1.4637ms 683.2168 Ops/s 697.6726 Ops/s $\color{#d91a1a}-2.07\%$
test_gae_speed[vec_generalized_advantage_estimate-False-32-512] 0.9465ms 0.6685ms 1.4960 KOps/s 1.5392 KOps/s $\color{#d91a1a}-2.81\%$
test_dqn_speed 7.6755ms 7.4066ms 135.0154 Ops/s 133.5213 Ops/s $\color{#35bf28}+1.12\%$
test_ddpg_speed 15.1856ms 14.3882ms 69.5016 Ops/s 67.3697 Ops/s $\color{#35bf28}+3.16\%$
test_sac_speed 29.8601ms 29.0518ms 34.4212 Ops/s 31.0374 Ops/s $\textbf{\color{#35bf28}+10.90\%}$
test_redq_speed 35.9447ms 35.1444ms 28.4540 Ops/s 28.3818 Ops/s $\color{#35bf28}+0.25\%$
test_redq_deprec_speed 25.1800ms 24.2562ms 41.2265 Ops/s 41.5098 Ops/s $\color{#d91a1a}-0.68\%$
test_td3_speed 29.2394ms 19.7650ms 50.5946 Ops/s 50.0020 Ops/s $\color{#35bf28}+1.19\%$
test_cql_speed 84.0137ms 83.5577ms 11.9678 Ops/s 11.8450 Ops/s $\color{#35bf28}+1.04\%$
test_a2c_speed 30.0005ms 26.9800ms 37.0646 Ops/s 36.9390 Ops/s $\color{#35bf28}+0.34\%$
test_ppo_speed 28.0307ms 27.1590ms 36.8202 Ops/s 36.4794 Ops/s $\color{#35bf28}+0.93\%$
test_reinforce_speed 27.1511ms 26.1315ms 38.2680 Ops/s 38.1439 Ops/s $\color{#35bf28}+0.33\%$
test_iql_speed 0.1586s 63.3935ms 15.7745 Ops/s 17.2677 Ops/s $\textbf{\color{#d91a1a}-8.65\%}$
test_rb_sample[TensorDictReplayBuffer-ListStorage-RandomSampler-4000] 2.4846ms 1.8945ms 527.8475 Ops/s 530.3400 Ops/s $\color{#d91a1a}-0.47\%$
test_rb_sample[TensorDictReplayBuffer-LazyMemmapStorage-RandomSampler-10000] 0.1206s 0.9612ms 1.0404 KOps/s 1.1980 KOps/s $\textbf{\color{#d91a1a}-13.15\%}$
test_rb_sample[TensorDictReplayBuffer-LazyTensorStorage-RandomSampler-10000] 0.9846ms 0.8243ms 1.2132 KOps/s 1.2147 KOps/s $\color{#d91a1a}-0.12\%$
test_rb_sample[TensorDictReplayBuffer-ListStorage-SamplerWithoutReplacement-4000] 2.5202ms 1.8685ms 535.1779 Ops/s 537.0105 Ops/s $\color{#d91a1a}-0.34\%$
test_rb_sample[TensorDictReplayBuffer-LazyMemmapStorage-SamplerWithoutReplacement-10000] 1.0032ms 0.8251ms 1.2120 KOps/s 1.2139 KOps/s $\color{#d91a1a}-0.16\%$
test_rb_sample[TensorDictReplayBuffer-LazyTensorStorage-SamplerWithoutReplacement-10000] 1.0051ms 0.8139ms 1.2286 KOps/s 1.2266 KOps/s $\color{#35bf28}+0.17\%$
test_rb_sample[TensorDictPrioritizedReplayBuffer-ListStorage-None-4000] 3.1865ms 2.1746ms 459.8599 Ops/s 466.1413 Ops/s $\color{#d91a1a}-1.35\%$
test_rb_sample[TensorDictPrioritizedReplayBuffer-LazyMemmapStorage-None-10000] 1.0946ms 0.9513ms 1.0511 KOps/s 1.0457 KOps/s $\color{#35bf28}+0.52\%$
test_rb_sample[TensorDictPrioritizedReplayBuffer-LazyTensorStorage-None-10000] 1.1208ms 0.9411ms 1.0626 KOps/s 1.0573 KOps/s $\color{#35bf28}+0.50\%$
test_rb_iterate[TensorDictReplayBuffer-ListStorage-RandomSampler-4000] 0.1130s 2.1308ms 469.3055 Ops/s 528.4174 Ops/s $\textbf{\color{#d91a1a}-11.19\%}$
test_rb_iterate[TensorDictReplayBuffer-LazyMemmapStorage-RandomSampler-10000] 1.0045ms 0.8364ms 1.1955 KOps/s 1.1978 KOps/s $\color{#d91a1a}-0.19\%$
test_rb_iterate[TensorDictReplayBuffer-LazyTensorStorage-RandomSampler-10000] 1.0258ms 0.8253ms 1.2117 KOps/s 1.2136 KOps/s $\color{#d91a1a}-0.16\%$
test_rb_iterate[TensorDictReplayBuffer-ListStorage-SamplerWithoutReplacement-4000] 2.0677ms 1.8525ms 539.8071 Ops/s 531.9838 Ops/s $\color{#35bf28}+1.47\%$
test_rb_iterate[TensorDictReplayBuffer-LazyMemmapStorage-SamplerWithoutReplacement-10000] 0.9836ms 0.8243ms 1.2132 KOps/s 1.2095 KOps/s $\color{#35bf28}+0.31\%$
test_rb_iterate[TensorDictReplayBuffer-LazyTensorStorage-SamplerWithoutReplacement-10000] 1.0723ms 0.8156ms 1.2261 KOps/s 1.2266 KOps/s $\color{#d91a1a}-0.04\%$
test_rb_iterate[TensorDictPrioritizedReplayBuffer-ListStorage-None-4000] 0.1267s 2.4380ms 410.1797 Ops/s 459.3873 Ops/s $\textbf{\color{#d91a1a}-10.71\%}$
test_rb_iterate[TensorDictPrioritizedReplayBuffer-LazyMemmapStorage-None-10000] 1.1209ms 0.9516ms 1.0508 KOps/s 1.0433 KOps/s $\color{#35bf28}+0.72\%$
test_rb_iterate[TensorDictPrioritizedReplayBuffer-LazyTensorStorage-None-10000] 1.1046ms 0.9430ms 1.0605 KOps/s 1.0536 KOps/s $\color{#35bf28}+0.65\%$
test_rb_populate[TensorDictReplayBuffer-ListStorage-RandomSampler-400] 0.1755s 19.7090ms 50.7382 Ops/s 52.9028 Ops/s $\color{#d91a1a}-4.09\%$
test_rb_populate[TensorDictReplayBuffer-LazyMemmapStorage-RandomSampler-400] 15.6937ms 13.5850ms 73.6104 Ops/s 60.6642 Ops/s $\textbf{\color{#35bf28}+21.34\%}$
test_rb_populate[TensorDictReplayBuffer-LazyTensorStorage-RandomSampler-400] 6.9316ms 3.3073ms 302.3611 Ops/s 303.4222 Ops/s $\color{#d91a1a}-0.35\%$
test_rb_populate[TensorDictReplayBuffer-ListStorage-SamplerWithoutReplacement-400] 0.1308s 18.5668ms 53.8595 Ops/s 53.2804 Ops/s $\color{#35bf28}+1.09\%$
test_rb_populate[TensorDictReplayBuffer-LazyMemmapStorage-SamplerWithoutReplacement-400] 16.1338ms 13.6599ms 73.2071 Ops/s 72.7634 Ops/s $\color{#35bf28}+0.61\%$
test_rb_populate[TensorDictReplayBuffer-LazyTensorStorage-SamplerWithoutReplacement-400] 4.8487ms 3.2241ms 310.1615 Ops/s 302.6815 Ops/s $\color{#35bf28}+2.47\%$
test_rb_populate[TensorDictPrioritizedReplayBuffer-ListStorage-None-400] 0.1264s 18.2828ms 54.6961 Ops/s 51.5661 Ops/s $\textbf{\color{#35bf28}+6.07\%}$
test_rb_populate[TensorDictPrioritizedReplayBuffer-LazyMemmapStorage-None-400] 16.4509ms 13.8442ms 72.2327 Ops/s 71.6036 Ops/s $\color{#35bf28}+0.88\%$
test_rb_populate[TensorDictPrioritizedReplayBuffer-LazyTensorStorage-None-400] 4.9621ms 3.4134ms 292.9637 Ops/s 286.3831 Ops/s $\color{#35bf28}+2.30\%$

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Reviewers
No reviews
Assignees
No one assigned
Labels
CI Has to do with CI setup (e.g. wheels & builds, tests...) CLA Signed This label is managed by the Facebook bot. Authors need to sign the CLA before a PR can be reviewed.
Projects
None yet
Milestone
No milestone
Development

Successfully merging this pull request may close these issues.
2 participants
@vmoens @facebook-github-bot