[BugFix,CI] Fix sporadically failing tests in CI

[vmoens]

Description

Describe your changes in detail.

Motivation and Context

Why is this change required? What problem does it solve?
If it fixes an open issue, please link to the issue here.
You can use the syntax close #15213 if this solves the issue #15213

• I have raised an issue to propose this change (required for new features and bug fixes)

Types of changes

What types of changes does your code introduce? Remove all that do not apply:

• Bug fix (non-breaking change which fixes an issue)
• New feature (non-breaking change which adds core functionality)
• Breaking change (fix or feature that would cause existing functionality to change)
• Documentation (update in the documentation)
• Example (update in the folder of examples)

Checklist

Go over all the following points, and put an x in all the boxes that apply.
If you are unsure about any of these, don't hesitate to ask. We are here to help!

• I have read the CONTRIBUTION guide (required)
• My change requires a change to the documentation.
• I have updated the tests accordingly (required for a bug fix or a new feature).
• I have updated the documentation accordingly.

[pytorch-bot]

🔗 Helpful Links

🧪 See artifacts and rendered test results at hud.pytorch.org/pr/pytorch/rl/2098

• 📄 Preview Python docs built from this PR

Note: Links to docs will display an error until the docs builds have been completed.

❌ 3 New Failures, 1 Unrelated Failure

As of commit a593d27 with merge base 32533a5 ():

NEW FAILURES - The following jobs have failed:

• Habitat Tests on Linux / tests (3.9, 12.1) / linux-job (gh)
  RuntimeError: Command docker exec -t e4f11aaffe24b7420ce6e666d3ff81b696eb33925161373c5cd45fc60c9fbaad /exec failed with exit code 139
• Unit-tests on Linux / tests-optdeps (3.10, 12.1) / linux-job (gh)
  test/test_trainer.py::TestOptimizer::test_optimizer_multiple_hooks
• Unit-tests on Windows / unittests-cpu / windows-job (gh)
  The process 'C:\Program Files\Git\cmd\git.exe' failed with exit code 128

BROKEN TRUNK - The following job failed but were present on the merge base:

👉 Rebase onto the `viable/strict` branch to avoid these failures

• Examples Tests on Linux / tests (3.9, 12.1) / linux-job (gh)
  RuntimeError: This process waited for 1000.0 seconds without receiving a command from main. Consider increasing the maximum idle count if this is expected via the environment variable MAX_IDLE_COUNT (current value is 1000).

This comment was automatically generated by Dr. CI and updates every 15 minutes.

[github-actions]

$\color{#D29922}\textsf{\Large⚠\kern{0.2cm}\normalsize Warning}$ Result of CPU Benchmark Tests

Total Benchmarks: 91. Improved: $\large\color{#35bf28}1$. Worsened: $\large\color{#d91a1a}5$.

Expand to view detailed results

Name	Max	Mean	Ops	Ops on Repo HEAD	Change
test_single	52.7837ms	51.7877ms	19.3096 Ops/s	18.2095 Ops/s	$\textbf{\color{#35bf28}+6.04\%}$
test_sync	41.3684ms	31.6493ms	31.5963 Ops/s	35.2071 Ops/s	$\textbf{\color{#d91a1a}-10.26\%}$
test_async	58.7681ms	26.2781ms	38.0544 Ops/s	37.0571 Ops/s	$\color{#35bf28}+2.69\%$
test_simple	0.3859s	0.3420s	2.9240 Ops/s	3.0107 Ops/s	$\color{#d91a1a}-2.88\%$
test_transformed	0.4767s	0.4737s	2.1110 Ops/s	2.0202 Ops/s	$\color{#35bf28}+4.50\%$
test_serial	1.2168s	1.1785s	0.8485 Ops/s	0.8303 Ops/s	$\color{#35bf28}+2.19\%$
test_parallel	1.0219s	0.9880s	1.0121 Ops/s	1.0214 Ops/s	$\color{#d91a1a}-0.91\%$
test_step_mdp_speed[True-True-True-True-True]	0.1086ms	21.5797μs	46.3399 KOps/s	47.1161 KOps/s	$\color{#d91a1a}-1.65\%$
test_step_mdp_speed[True-True-True-True-False]	38.0810μs	13.0596μs	76.5722 KOps/s	77.2553 KOps/s	$\color{#d91a1a}-0.88\%$
test_step_mdp_speed[True-True-True-False-True]	30.6570μs	12.6460μs	79.0762 KOps/s	80.6600 KOps/s	$\color{#d91a1a}-1.96\%$
test_step_mdp_speed[True-True-True-False-False]	37.6280μs	7.5573μs	132.3216 KOps/s	132.9660 KOps/s	$\color{#d91a1a}-0.48\%$
test_step_mdp_speed[True-True-False-True-True]	58.2080μs	22.8473μs	43.7689 KOps/s	44.2846 KOps/s	$\color{#d91a1a}-1.16\%$
test_step_mdp_speed[True-True-False-True-False]	56.7890μs	14.4300μs	69.2999 KOps/s	70.8389 KOps/s	$\color{#d91a1a}-2.17\%$
test_step_mdp_speed[True-True-False-False-True]	40.2360μs	13.8770μs	72.0618 KOps/s	74.6750 KOps/s	$\color{#d91a1a}-3.50\%$
test_step_mdp_speed[True-True-False-False-False]	29.0140μs	8.8583μs	112.8881 KOps/s	115.0066 KOps/s	$\color{#d91a1a}-1.84\%$
test_step_mdp_speed[True-False-True-True-True]	83.2360μs	23.7969μs	42.0223 KOps/s	42.0862 KOps/s	$\color{#d91a1a}-0.15\%$
test_step_mdp_speed[True-False-True-True-False]	35.3450μs	15.4761μs	64.6159 KOps/s	65.0376 KOps/s	$\color{#d91a1a}-0.65\%$
test_step_mdp_speed[True-False-True-False-True]	60.6030μs	13.5696μs	73.6942 KOps/s	72.7065 KOps/s	$\color{#35bf28}+1.36\%$
test_step_mdp_speed[True-False-True-False-False]	44.9640μs	8.7087μs	114.8280 KOps/s	114.7080 KOps/s	$\color{#35bf28}+0.10\%$
test_step_mdp_speed[True-False-False-True-True]	54.4420μs	25.0243μs	39.9612 KOps/s	39.6629 KOps/s	$\color{#35bf28}+0.75\%$
test_step_mdp_speed[True-False-False-True-False]	56.9470μs	16.5384μs	60.4654 KOps/s	59.9626 KOps/s	$\color{#35bf28}+0.84\%$
test_step_mdp_speed[True-False-False-False-True]	49.9440μs	14.5574μs	68.6934 KOps/s	68.9201 KOps/s	$\color{#d91a1a}-0.33\%$
test_step_mdp_speed[True-False-False-False-False]	39.2100μs	9.8005μs	102.0359 KOps/s	100.9711 KOps/s	$\color{#35bf28}+1.05\%$
test_step_mdp_speed[False-True-True-True-True]	58.9100μs	24.1333μs	41.4366 KOps/s	41.8218 KOps/s	$\color{#d91a1a}-0.92\%$
test_step_mdp_speed[False-True-True-True-False]	39.4030μs	15.7207μs	63.6104 KOps/s	64.3091 KOps/s	$\color{#d91a1a}-1.09\%$
test_step_mdp_speed[False-True-True-False-True]	50.5340μs	15.9475μs	62.7059 KOps/s	62.1923 KOps/s	$\color{#35bf28}+0.83\%$
test_step_mdp_speed[False-True-True-False-False]	30.1770μs	9.7709μs	102.3445 KOps/s	97.8218 KOps/s	$\color{#35bf28}+4.62\%$
test_step_mdp_speed[False-True-False-True-True]	56.1450μs	25.8545μs	38.6780 KOps/s	38.8046 KOps/s	$\color{#d91a1a}-0.33\%$
test_step_mdp_speed[False-True-False-True-False]	45.1440μs	16.7172μs	59.8186 KOps/s	59.0947 KOps/s	$\color{#35bf28}+1.22\%$
test_step_mdp_speed[False-True-False-False-True]	54.0110μs	17.0683μs	58.5882 KOps/s	57.7559 KOps/s	$\color{#35bf28}+1.44\%$
test_step_mdp_speed[False-True-False-False-False]	48.5310μs	11.2350μs	89.0077 KOps/s	88.6064 KOps/s	$\color{#35bf28}+0.45\%$
test_step_mdp_speed[False-False-True-True-True]	56.5260μs	26.7519μs	37.3805 KOps/s	37.6615 KOps/s	$\color{#d91a1a}-0.75\%$
test_step_mdp_speed[False-False-True-True-False]	51.8170μs	18.1241μs	55.1750 KOps/s	55.1692 KOps/s	$\color{#35bf28}+0.01\%$
test_step_mdp_speed[False-False-True-False-True]	51.8670μs	17.1813μs	58.2027 KOps/s	58.0479 KOps/s	$\color{#35bf28}+0.27\%$
test_step_mdp_speed[False-False-True-False-False]	36.3270μs	11.0818μs	90.2377 KOps/s	89.0777 KOps/s	$\color{#35bf28}+1.30\%$
test_step_mdp_speed[False-False-False-True-True]	56.7360μs	27.7641μs	36.0177 KOps/s	36.0404 KOps/s	$\color{#d91a1a}-0.06\%$
test_step_mdp_speed[False-False-False-True-False]	54.5720μs	19.2382μs	51.9798 KOps/s	52.5508 KOps/s	$\color{#d91a1a}-1.09\%$
test_step_mdp_speed[False-False-False-False-True]	55.6830μs	18.1551μs	55.0809 KOps/s	54.5488 KOps/s	$\color{#35bf28}+0.98\%$
test_step_mdp_speed[False-False-False-False-False]	42.3090μs	12.2419μs	81.6864 KOps/s	81.3519 KOps/s	$\color{#35bf28}+0.41\%$
test_values[generalized_advantage_estimate-True-True]	11.4505ms	9.4337ms	106.0033 Ops/s	108.6966 Ops/s	$\color{#d91a1a}-2.48\%$
test_values[vec_generalized_advantage_estimate-True-True]	35.9347ms	34.8206ms	28.7186 Ops/s	28.6997 Ops/s	$\color{#35bf28}+0.07\%$
test_values[td0_return_estimate-False-False]	0.2265ms	0.1652ms	6.0517 KOps/s	6.0651 KOps/s	$\color{#d91a1a}-0.22\%$
test_values[td1_return_estimate-False-False]	24.0049ms	23.5488ms	42.4649 Ops/s	43.9413 Ops/s	$\color{#d91a1a}-3.36\%$
test_values[vec_td1_return_estimate-False-False]	38.7526ms	35.0526ms	28.5285 Ops/s	28.6640 Ops/s	$\color{#d91a1a}-0.47\%$
test_values[td_lambda_return_estimate-True-False]	36.6093ms	34.2028ms	29.2373 Ops/s	30.3064 Ops/s	$\color{#d91a1a}-3.53\%$
test_values[vec_td_lambda_return_estimate-True-False]	36.2589ms	34.8172ms	28.7214 Ops/s	28.5592 Ops/s	$\color{#35bf28}+0.57\%$
test_gae_speed[generalized_advantage_estimate-False-1-512]	8.7503ms	8.2606ms	121.0569 Ops/s	121.6256 Ops/s	$\color{#d91a1a}-0.47\%$
test_gae_speed[vec_generalized_advantage_estimate-True-1-512]	2.2990ms	2.0000ms	500.0120 Ops/s	510.4523 Ops/s	$\color{#d91a1a}-2.05\%$
test_gae_speed[vec_generalized_advantage_estimate-False-1-512]	0.4087ms	0.3450ms	2.8988 KOps/s	2.8952 KOps/s	$\color{#35bf28}+0.13\%$
test_gae_speed[vec_generalized_advantage_estimate-True-32-512]	47.8210ms	45.6867ms	21.8882 Ops/s	23.1136 Ops/s	$\textbf{\color{#d91a1a}-5.30\%}$
test_gae_speed[vec_generalized_advantage_estimate-False-32-512]	4.0769ms	3.0228ms	330.8179 Ops/s	332.1692 Ops/s	$\color{#d91a1a}-0.41\%$
test_dqn_speed	1.8820ms	1.3361ms	748.4425 Ops/s	746.4941 Ops/s	$\color{#35bf28}+0.26\%$
test_ddpg_speed	2.9904ms	2.6888ms	371.9066 Ops/s	372.0022 Ops/s	$\color{#d91a1a}-0.03\%$
test_sac_speed	9.3338ms	8.1186ms	123.1747 Ops/s	122.1119 Ops/s	$\color{#35bf28}+0.87\%$
test_redq_speed	15.0253ms	12.8196ms	78.0057 Ops/s	78.2500 Ops/s	$\color{#d91a1a}-0.31\%$
test_redq_deprec_speed	14.3475ms	12.8080ms	78.0765 Ops/s	78.1317 Ops/s	$\color{#d91a1a}-0.07\%$
test_td3_speed	15.2450ms	8.0972ms	123.4988 Ops/s	124.1409 Ops/s	$\color{#d91a1a}-0.52\%$
test_cql_speed	40.4284ms	36.1900ms	27.6319 Ops/s	27.6965 Ops/s	$\color{#d91a1a}-0.23\%$
test_a2c_speed	8.1770ms	7.2963ms	137.0561 Ops/s	136.8158 Ops/s	$\color{#35bf28}+0.18\%$
test_ppo_speed	8.1967ms	7.5313ms	132.7796 Ops/s	132.9014 Ops/s	$\color{#d91a1a}-0.09\%$
test_reinforce_speed	7.7410ms	6.5142ms	153.5117 Ops/s	153.6493 Ops/s	$\color{#d91a1a}-0.09\%$
test_iql_speed	33.7393ms	32.2534ms	31.0045 Ops/s	31.4354 Ops/s	$\color{#d91a1a}-1.37\%$
test_rb_sample[TensorDictReplayBuffer-ListStorage-RandomSampler-4000]	3.1776ms	2.0847ms	479.6934 Ops/s	480.4273 Ops/s	$\color{#d91a1a}-0.15\%$
test_rb_sample[TensorDictReplayBuffer-LazyMemmapStorage-RandomSampler-10000]	80.7862ms	0.5477ms	1.8259 KOps/s	2.0150 KOps/s	$\textbf{\color{#d91a1a}-9.38\%}$
test_rb_sample[TensorDictReplayBuffer-LazyTensorStorage-RandomSampler-10000]	0.6294ms	0.4662ms	2.1449 KOps/s	2.1422 KOps/s	$\color{#35bf28}+0.13\%$
test_rb_sample[TensorDictReplayBuffer-ListStorage-SamplerWithoutReplacement-4000]	3.2358ms	2.0533ms	487.0310 Ops/s	481.1013 Ops/s	$\color{#35bf28}+1.23\%$
test_rb_sample[TensorDictReplayBuffer-LazyMemmapStorage-SamplerWithoutReplacement-10000]	0.9626ms	0.4878ms	2.0499 KOps/s	2.0609 KOps/s	$\color{#d91a1a}-0.54\%$
test_rb_sample[TensorDictReplayBuffer-LazyTensorStorage-SamplerWithoutReplacement-10000]	0.7850ms	0.4610ms	2.1693 KOps/s	2.1674 KOps/s	$\color{#35bf28}+0.09\%$
test_rb_sample[TensorDictReplayBuffer-LazyMemmapStorage-sampler6-10000]	1.8174ms	1.2183ms	820.8245 Ops/s	813.7317 Ops/s	$\color{#35bf28}+0.87\%$
test_rb_sample[TensorDictReplayBuffer-LazyTensorStorage-sampler7-10000]	1.6069ms	1.1474ms	871.5122 Ops/s	861.0251 Ops/s	$\color{#35bf28}+1.22\%$
test_rb_sample[TensorDictPrioritizedReplayBuffer-ListStorage-None-4000]	2.3292ms	2.1856ms	457.5494 Ops/s	455.9073 Ops/s	$\color{#35bf28}+0.36\%$
test_rb_sample[TensorDictPrioritizedReplayBuffer-LazyMemmapStorage-None-10000]	1.0316ms	0.6097ms	1.6401 KOps/s	1.6423 KOps/s	$\color{#d91a1a}-0.13\%$
test_rb_sample[TensorDictPrioritizedReplayBuffer-LazyTensorStorage-None-10000]	0.8072ms	0.5796ms	1.7252 KOps/s	1.7216 KOps/s	$\color{#35bf28}+0.21\%$
test_rb_iterate[TensorDictReplayBuffer-ListStorage-RandomSampler-4000]	3.1714ms	2.0760ms	481.6916 Ops/s	489.2718 Ops/s	$\color{#d91a1a}-1.55\%$
test_rb_iterate[TensorDictReplayBuffer-LazyMemmapStorage-RandomSampler-10000]	1.0448ms	0.4931ms	2.0282 KOps/s	2.0110 KOps/s	$\color{#35bf28}+0.86\%$
test_rb_iterate[TensorDictReplayBuffer-LazyTensorStorage-RandomSampler-10000]	0.7152ms	0.4692ms	2.1312 KOps/s	2.1708 KOps/s	$\color{#d91a1a}-1.82\%$
test_rb_iterate[TensorDictReplayBuffer-ListStorage-SamplerWithoutReplacement-4000]	3.1996ms	2.0526ms	487.1834 Ops/s	485.9169 Ops/s	$\color{#35bf28}+0.26\%$
test_rb_iterate[TensorDictReplayBuffer-LazyMemmapStorage-SamplerWithoutReplacement-10000]	0.6033ms	0.4866ms	2.0552 KOps/s	2.0455 KOps/s	$\color{#35bf28}+0.47\%$
test_rb_iterate[TensorDictReplayBuffer-LazyTensorStorage-SamplerWithoutReplacement-10000]	3.5077ms	0.4665ms	2.1436 KOps/s	2.1548 KOps/s	$\color{#d91a1a}-0.52\%$
test_rb_iterate[TensorDictPrioritizedReplayBuffer-ListStorage-None-4000]	2.5166ms	2.1980ms	454.9671 Ops/s	463.8901 Ops/s	$\color{#d91a1a}-1.92\%$
test_rb_iterate[TensorDictPrioritizedReplayBuffer-LazyMemmapStorage-None-10000]	0.9714ms	0.6128ms	1.6319 KOps/s	1.6457 KOps/s	$\color{#d91a1a}-0.83\%$
test_rb_iterate[TensorDictPrioritizedReplayBuffer-LazyTensorStorage-None-10000]	0.9599ms	0.5840ms	1.7124 KOps/s	1.7214 KOps/s	$\color{#d91a1a}-0.53\%$
test_rb_populate[TensorDictReplayBuffer-ListStorage-RandomSampler-400]	0.1009s	7.3504ms	136.0477 Ops/s	138.8588 Ops/s	$\color{#d91a1a}-2.02\%$
test_rb_populate[TensorDictReplayBuffer-LazyMemmapStorage-RandomSampler-400]	14.2702ms	11.8577ms	84.3336 Ops/s	84.4383 Ops/s	$\color{#d91a1a}-0.12\%$
test_rb_populate[TensorDictReplayBuffer-LazyTensorStorage-RandomSampler-400]	1.7669ms	1.0802ms	925.7951 Ops/s	981.3718 Ops/s	$\textbf{\color{#d91a1a}-5.66\%}$
test_rb_populate[TensorDictReplayBuffer-ListStorage-SamplerWithoutReplacement-400]	91.2018ms	5.4665ms	182.9323 Ops/s	187.4052 Ops/s	$\color{#d91a1a}-2.39\%$
test_rb_populate[TensorDictReplayBuffer-LazyMemmapStorage-SamplerWithoutReplacement-400]	13.6358ms	11.7100ms	85.3972 Ops/s	84.8305 Ops/s	$\color{#35bf28}+0.67\%$
test_rb_populate[TensorDictReplayBuffer-LazyTensorStorage-SamplerWithoutReplacement-400]	1.5871ms	1.0341ms	967.0155 Ops/s	952.6432 Ops/s	$\color{#35bf28}+1.51\%$
test_rb_populate[TensorDictPrioritizedReplayBuffer-ListStorage-None-400]	91.8007ms	7.5038ms	133.2651 Ops/s	136.0220 Ops/s	$\color{#d91a1a}-2.03\%$
test_rb_populate[TensorDictPrioritizedReplayBuffer-LazyMemmapStorage-None-400]	14.3664ms	12.0904ms	82.7105 Ops/s	83.4768 Ops/s	$\color{#d91a1a}-0.92\%$
test_rb_populate[TensorDictPrioritizedReplayBuffer-LazyTensorStorage-None-400]	3.9612ms	1.4201ms	704.1546 Ops/s	763.2052 Ops/s	$\textbf{\color{#d91a1a}-7.74\%}$

[github-actions]

$\color{#D29922}\textsf{\Large⚠\kern{0.2cm}\normalsize Warning}$ Result of GPU Benchmark Tests

Total Benchmarks: 94. Improved: $\large\color{#35bf28}5$. Worsened: $\large\color{#d91a1a}6$.

Expand to view detailed results

Name	Max	Mean	Ops	Ops on Repo HEAD	Change
test_single	0.1010s	0.1007s	9.9280 Ops/s	9.4751 Ops/s	$\color{#35bf28}+4.78\%$
test_sync	87.2269ms	86.1878ms	11.6026 Ops/s	11.3969 Ops/s	$\color{#35bf28}+1.80\%$
test_async	0.1641s	71.7170ms	13.9437 Ops/s	11.9255 Ops/s	$\textbf{\color{#35bf28}+16.92\%}$
test_single_pixels	0.1116s	0.1113s	8.9819 Ops/s	9.0678 Ops/s	$\color{#d91a1a}-0.95\%$
test_sync_pixels	68.8475ms	66.7821ms	14.9741 Ops/s	14.7213 Ops/s	$\color{#35bf28}+1.72\%$
test_async_pixels	0.1236s	55.6784ms	17.9603 Ops/s	17.7743 Ops/s	$\color{#35bf28}+1.05\%$
test_simple	0.7034s	0.6808s	1.4688 Ops/s	1.4730 Ops/s	$\color{#d91a1a}-0.28\%$
test_transformed	0.9134s	0.9076s	1.1018 Ops/s	1.1104 Ops/s	$\color{#d91a1a}-0.78\%$
test_serial	2.1654s	2.1271s	0.4701 Ops/s	0.4831 Ops/s	$\color{#d91a1a}-2.69\%$
test_parallel	1.8067s	1.7766s	0.5629 Ops/s	0.5534 Ops/s	$\color{#35bf28}+1.71\%$
test_step_mdp_speed[True-True-True-True-True]	85.3000μs	33.5810μs	29.7787 KOps/s	31.1459 KOps/s	$\color{#d91a1a}-4.39\%$
test_step_mdp_speed[True-True-True-True-False]	53.9200μs	19.8335μs	50.4197 KOps/s	51.3891 KOps/s	$\color{#d91a1a}-1.89\%$
test_step_mdp_speed[True-True-True-False-True]	47.3100μs	18.8523μs	53.0438 KOps/s	54.2117 KOps/s	$\color{#d91a1a}-2.15\%$
test_step_mdp_speed[True-True-True-False-False]	38.8600μs	11.4807μs	87.1023 KOps/s	89.8901 KOps/s	$\color{#d91a1a}-3.10\%$
test_step_mdp_speed[True-True-False-True-True]	64.4500μs	34.8996μs	28.6536 KOps/s	29.4246 KOps/s	$\color{#d91a1a}-2.62\%$
test_step_mdp_speed[True-True-False-True-False]	52.5700μs	21.4918μs	46.5294 KOps/s	47.3810 KOps/s	$\color{#d91a1a}-1.80\%$
test_step_mdp_speed[True-True-False-False-True]	48.8200μs	20.6935μs	48.3244 KOps/s	49.1475 KOps/s	$\color{#d91a1a}-1.67\%$
test_step_mdp_speed[True-True-False-False-False]	30.8500μs	13.2895μs	75.2471 KOps/s	76.8209 KOps/s	$\color{#d91a1a}-2.05\%$
test_step_mdp_speed[True-False-True-True-True]	65.2700μs	36.6945μs	27.2521 KOps/s	27.8379 KOps/s	$\color{#d91a1a}-2.10\%$
test_step_mdp_speed[True-False-True-True-False]	44.2000μs	23.8866μs	41.8645 KOps/s	43.0608 KOps/s	$\color{#d91a1a}-2.78\%$
test_step_mdp_speed[True-False-True-False-True]	47.8400μs	20.4474μs	48.9060 KOps/s	49.1948 KOps/s	$\color{#d91a1a}-0.59\%$
test_step_mdp_speed[True-False-True-False-False]	44.5810μs	13.4327μs	74.4454 KOps/s	76.7841 KOps/s	$\color{#d91a1a}-3.05\%$
test_step_mdp_speed[True-False-False-True-True]	67.1700μs	38.5278μs	25.9553 KOps/s	26.3476 KOps/s	$\color{#d91a1a}-1.49\%$
test_step_mdp_speed[True-False-False-True-False]	52.6810μs	25.4549μs	39.2851 KOps/s	40.3953 KOps/s	$\color{#d91a1a}-2.75\%$
test_step_mdp_speed[True-False-False-False-True]	45.3700μs	22.2207μs	45.0031 KOps/s	45.6395 KOps/s	$\color{#d91a1a}-1.39\%$
test_step_mdp_speed[True-False-False-False-False]	36.9000μs	15.0574μs	66.4127 KOps/s	67.6922 KOps/s	$\color{#d91a1a}-1.89\%$
test_step_mdp_speed[False-True-True-True-True]	68.2410μs	36.3638μs	27.4999 KOps/s	27.5980 KOps/s	$\color{#d91a1a}-0.36\%$
test_step_mdp_speed[False-True-True-True-False]	49.3600μs	23.8085μs	42.0018 KOps/s	43.0800 KOps/s	$\color{#d91a1a}-2.50\%$
test_step_mdp_speed[False-True-True-False-True]	47.3420μs	24.7358μs	40.4273 KOps/s	41.3930 KOps/s	$\color{#d91a1a}-2.33\%$
test_step_mdp_speed[False-True-True-False-False]	35.7000μs	15.1662μs	65.9360 KOps/s	67.9185 KOps/s	$\color{#d91a1a}-2.92\%$
test_step_mdp_speed[False-True-False-True-True]	75.9410μs	38.9951μs	25.6442 KOps/s	26.4679 KOps/s	$\color{#d91a1a}-3.11\%$
test_step_mdp_speed[False-True-False-True-False]	48.4010μs	25.9635μs	38.5156 KOps/s	39.9265 KOps/s	$\color{#d91a1a}-3.53\%$
test_step_mdp_speed[False-True-False-False-True]	56.6110μs	26.5677μs	37.6397 KOps/s	39.3504 KOps/s	$\color{#d91a1a}-4.35\%$
test_step_mdp_speed[False-True-False-False-False]	44.8600μs	16.7744μs	59.6147 KOps/s	60.8580 KOps/s	$\color{#d91a1a}-2.04\%$
test_step_mdp_speed[False-False-True-True-True]	67.9800μs	40.3211μs	24.8009 KOps/s	25.0403 KOps/s	$\color{#d91a1a}-0.96\%$
test_step_mdp_speed[False-False-True-True-False]	64.2310μs	27.7444μs	36.0433 KOps/s	37.2154 KOps/s	$\color{#d91a1a}-3.15\%$
test_step_mdp_speed[False-False-True-False-True]	51.5000μs	26.2279μs	38.1274 KOps/s	39.1117 KOps/s	$\color{#d91a1a}-2.52\%$
test_step_mdp_speed[False-False-True-False-False]	46.7900μs	16.5925μs	60.2680 KOps/s	60.7240 KOps/s	$\color{#d91a1a}-0.75\%$
test_step_mdp_speed[False-False-False-True-True]	68.9210μs	41.7144μs	23.9725 KOps/s	24.1190 KOps/s	$\color{#d91a1a}-0.61\%$
test_step_mdp_speed[False-False-False-True-False]	60.6300μs	29.5681μs	33.8202 KOps/s	34.8302 KOps/s	$\color{#d91a1a}-2.90\%$
test_step_mdp_speed[False-False-False-False-True]	56.0510μs	28.0951μs	35.5934 KOps/s	36.9015 KOps/s	$\color{#d91a1a}-3.54\%$
test_step_mdp_speed[False-False-False-False-False]	42.6110μs	18.2425μs	54.8169 KOps/s	54.5902 KOps/s	$\color{#35bf28}+0.42\%$
test_values[generalized_advantage_estimate-True-True]	24.8768ms	24.2737ms	41.1969 Ops/s	41.3613 Ops/s	$\color{#d91a1a}-0.40\%$
test_values[vec_generalized_advantage_estimate-True-True]	80.5880ms	3.1798ms	314.4849 Ops/s	298.7767 Ops/s	$\textbf{\color{#35bf28}+5.26\%}$
test_values[td0_return_estimate-False-False]	92.4710μs	64.1411μs	15.5906 KOps/s	15.7630 KOps/s	$\color{#d91a1a}-1.09\%$
test_values[td1_return_estimate-False-False]	52.6594ms	52.2320ms	19.1454 Ops/s	19.0917 Ops/s	$\color{#35bf28}+0.28\%$
test_values[vec_td1_return_estimate-False-False]	2.0824ms	1.7687ms	565.3799 Ops/s	569.0034 Ops/s	$\color{#d91a1a}-0.64\%$
test_values[td_lambda_return_estimate-True-False]	84.5509ms	83.6187ms	11.9590 Ops/s	12.1084 Ops/s	$\color{#d91a1a}-1.23\%$
test_values[vec_td_lambda_return_estimate-True-False]	2.1353ms	1.7600ms	568.1859 Ops/s	569.2308 Ops/s	$\color{#d91a1a}-0.18\%$
test_gae_speed[generalized_advantage_estimate-False-1-512]	24.3509ms	23.4054ms	42.7251 Ops/s	43.3999 Ops/s	$\color{#d91a1a}-1.55\%$
test_gae_speed[vec_generalized_advantage_estimate-True-1-512]	0.9262ms	0.7311ms	1.3677 KOps/s	1.4423 KOps/s	$\textbf{\color{#d91a1a}-5.17\%}$
test_gae_speed[vec_generalized_advantage_estimate-False-1-512]	0.7084ms	0.6515ms	1.5350 KOps/s	1.5541 KOps/s	$\color{#d91a1a}-1.22\%$
test_gae_speed[vec_generalized_advantage_estimate-True-32-512]	1.4928ms	1.4522ms	688.5964 Ops/s	691.7078 Ops/s	$\color{#d91a1a}-0.45\%$
test_gae_speed[vec_generalized_advantage_estimate-False-32-512]	1.0188ms	0.6717ms	1.4888 KOps/s	1.4995 KOps/s	$\color{#d91a1a}-0.71\%$
test_dqn_speed	8.0024ms	1.4453ms	691.8783 Ops/s	698.9126 Ops/s	$\color{#d91a1a}-1.01\%$
test_ddpg_speed	3.2859ms	2.7624ms	362.0035 Ops/s	370.0568 Ops/s	$\color{#d91a1a}-2.18\%$
test_sac_speed	12.5654ms	8.2069ms	121.8493 Ops/s	113.2610 Ops/s	$\textbf{\color{#35bf28}+7.58\%}$
test_redq_speed	0.1000s	11.4851ms	87.0692 Ops/s	97.4975 Ops/s	$\textbf{\color{#d91a1a}-10.70\%}$
test_redq_deprec_speed	11.4458ms	11.0605ms	90.4116 Ops/s	89.8424 Ops/s	$\color{#35bf28}+0.63\%$
test_td3_speed	8.1662ms	8.0324ms	124.4951 Ops/s	125.0633 Ops/s	$\color{#d91a1a}-0.45\%$
test_cql_speed	25.4381ms	25.0258ms	39.9587 Ops/s	40.3242 Ops/s	$\color{#d91a1a}-0.91\%$
test_a2c_speed	6.4717ms	5.5678ms	179.6041 Ops/s	180.2142 Ops/s	$\color{#d91a1a}-0.34\%$
test_ppo_speed	6.2561ms	5.8785ms	170.1124 Ops/s	173.1264 Ops/s	$\color{#d91a1a}-1.74\%$
test_reinforce_speed	4.7259ms	4.4965ms	222.3937 Ops/s	224.5203 Ops/s	$\color{#d91a1a}-0.95\%$
test_iql_speed	20.1098ms	19.5382ms	51.1818 Ops/s	52.9534 Ops/s	$\color{#d91a1a}-3.35\%$
test_rb_sample[TensorDictReplayBuffer-ListStorage-RandomSampler-4000]	3.0562ms	2.8972ms	345.1573 Ops/s	352.3222 Ops/s	$\color{#d91a1a}-2.03\%$
test_rb_sample[TensorDictReplayBuffer-LazyMemmapStorage-RandomSampler-10000]	1.4360ms	0.5430ms	1.8415 KOps/s	1.8570 KOps/s	$\color{#d91a1a}-0.84\%$
test_rb_sample[TensorDictReplayBuffer-LazyTensorStorage-RandomSampler-10000]	0.7142ms	0.5207ms	1.9204 KOps/s	1.9318 KOps/s	$\color{#d91a1a}-0.59\%$
test_rb_sample[TensorDictReplayBuffer-ListStorage-SamplerWithoutReplacement-4000]	3.1169ms	2.9167ms	342.8563 Ops/s	350.3459 Ops/s	$\color{#d91a1a}-2.14\%$
test_rb_sample[TensorDictReplayBuffer-LazyMemmapStorage-SamplerWithoutReplacement-10000]	1.3296ms	0.5358ms	1.8665 KOps/s	1.8974 KOps/s	$\color{#d91a1a}-1.63\%$
test_rb_sample[TensorDictReplayBuffer-LazyTensorStorage-SamplerWithoutReplacement-10000]	0.6375ms	0.5123ms	1.9520 KOps/s	1.9630 KOps/s	$\color{#d91a1a}-0.56\%$
test_rb_sample[TensorDictReplayBuffer-LazyMemmapStorage-sampler6-10000]	3.7414ms	1.4402ms	694.3711 Ops/s	704.4634 Ops/s	$\color{#d91a1a}-1.43\%$
test_rb_sample[TensorDictReplayBuffer-LazyTensorStorage-sampler7-10000]	1.5118ms	1.3708ms	729.4930 Ops/s	737.7835 Ops/s	$\color{#d91a1a}-1.12\%$
test_rb_sample[TensorDictPrioritizedReplayBuffer-ListStorage-None-4000]	3.1639ms	3.0190ms	331.2399 Ops/s	337.8960 Ops/s	$\color{#d91a1a}-1.97\%$
test_rb_sample[TensorDictPrioritizedReplayBuffer-LazyMemmapStorage-None-10000]	0.8054ms	0.6649ms	1.5041 KOps/s	1.3198 KOps/s	$\textbf{\color{#35bf28}+13.96\%}$
test_rb_sample[TensorDictPrioritizedReplayBuffer-LazyTensorStorage-None-10000]	4.2659ms	0.6446ms	1.5514 KOps/s	1.5886 KOps/s	$\color{#d91a1a}-2.34\%$
test_rb_iterate[TensorDictReplayBuffer-ListStorage-RandomSampler-4000]	3.0564ms	2.8982ms	345.0409 Ops/s	349.8704 Ops/s	$\color{#d91a1a}-1.38\%$
test_rb_iterate[TensorDictReplayBuffer-LazyMemmapStorage-RandomSampler-10000]	0.6611ms	0.5417ms	1.8462 KOps/s	1.8614 KOps/s	$\color{#d91a1a}-0.82\%$
test_rb_iterate[TensorDictReplayBuffer-LazyTensorStorage-RandomSampler-10000]	4.2052ms	0.5252ms	1.9039 KOps/s	1.9540 KOps/s	$\color{#d91a1a}-2.56\%$
test_rb_iterate[TensorDictReplayBuffer-ListStorage-SamplerWithoutReplacement-4000]	3.2294ms	2.9181ms	342.6930 Ops/s	351.9430 Ops/s	$\color{#d91a1a}-2.63\%$
test_rb_iterate[TensorDictReplayBuffer-LazyMemmapStorage-SamplerWithoutReplacement-10000]	1.4296ms	0.5711ms	1.7509 KOps/s	1.8842 KOps/s	$\textbf{\color{#d91a1a}-7.08\%}$
test_rb_iterate[TensorDictReplayBuffer-LazyTensorStorage-SamplerWithoutReplacement-10000]	0.7200ms	0.5535ms	1.8067 KOps/s	1.9503 KOps/s	$\textbf{\color{#d91a1a}-7.36\%}$
test_rb_iterate[TensorDictPrioritizedReplayBuffer-ListStorage-None-4000]	3.2221ms	3.0272ms	330.3347 Ops/s	332.7181 Ops/s	$\color{#d91a1a}-0.72\%$
test_rb_iterate[TensorDictPrioritizedReplayBuffer-LazyMemmapStorage-None-10000]	0.8788ms	0.7061ms	1.4163 KOps/s	1.5186 KOps/s	$\textbf{\color{#d91a1a}-6.74\%}$
test_rb_iterate[TensorDictPrioritizedReplayBuffer-LazyTensorStorage-None-10000]	5.1210ms	0.6984ms	1.4319 KOps/s	1.5742 KOps/s	$\textbf{\color{#d91a1a}-9.04\%}$
test_rb_populate[TensorDictReplayBuffer-ListStorage-RandomSampler-400]	0.1340s	7.5345ms	132.7237 Ops/s	135.6335 Ops/s	$\color{#d91a1a}-2.15\%$
test_rb_populate[TensorDictReplayBuffer-LazyMemmapStorage-RandomSampler-400]	0.1300s	17.2180ms	58.0788 Ops/s	60.4872 Ops/s	$\color{#d91a1a}-3.98\%$
test_rb_populate[TensorDictReplayBuffer-LazyTensorStorage-RandomSampler-400]	2.2960ms	1.1290ms	885.7778 Ops/s	923.8988 Ops/s	$\color{#d91a1a}-4.13\%$
test_rb_populate[TensorDictReplayBuffer-ListStorage-SamplerWithoutReplacement-400]	0.1199s	7.3412ms	136.2172 Ops/s	142.6518 Ops/s	$\color{#d91a1a}-4.51\%$
test_rb_populate[TensorDictReplayBuffer-LazyMemmapStorage-SamplerWithoutReplacement-400]	17.7061ms	14.9208ms	67.0208 Ops/s	69.7288 Ops/s	$\color{#d91a1a}-3.88\%$
test_rb_populate[TensorDictReplayBuffer-LazyTensorStorage-SamplerWithoutReplacement-400]	2.1416ms	1.1235ms	890.0377 Ops/s	830.5828 Ops/s	$\textbf{\color{#35bf28}+7.16\%}$
test_rb_populate[TensorDictPrioritizedReplayBuffer-ListStorage-None-400]	0.1191s	7.6533ms	130.6621 Ops/s	135.8135 Ops/s	$\color{#d91a1a}-3.79\%$
test_rb_populate[TensorDictPrioritizedReplayBuffer-LazyMemmapStorage-None-400]	17.8222ms	15.1134ms	66.1663 Ops/s	68.4356 Ops/s	$\color{#d91a1a}-3.32\%$
test_rb_populate[TensorDictPrioritizedReplayBuffer-LazyTensorStorage-None-400]	1.7647ms	1.5840ms	631.2998 Ops/s	650.2378 Ops/s	$\color{#d91a1a}-2.91\%$