[BugFix] done = done | truncated in collector

[vmoens]

Solved #2329

@matteobettini

[pytorch-bot]

🔗 Helpful Links

🧪 See artifacts and rendered test results at hud.pytorch.org/pr/pytorch/rl/2333

• 📄 Preview Python docs built from this PR

Note: Links to docs will display an error until the docs builds have been completed.

❌ 3 New Failures, 1 Pending

As of commit 57e4051 with merge base 0063741 ():

NEW FAILURES - The following jobs have failed:

• Habitat Tests on Linux / tests (3.9, 12.1) / linux-job (gh)
  RuntimeError: Command docker exec -t 81fade4c6c4e250701392a51c31e44ca9dccde7d122853e1f001d07ea7f81476 /exec failed with exit code 139
• Unit-tests on Linux / tests-olddeps (3.8, 11.6) / linux-job (gh)
  test/test_tensordictmodules.py::TestGRUModule::test_gru_vmap_complex_model
• Unit-tests on Windows / unittests-cpu / windows-job (gh)
  The process 'C:\Program Files\Git\cmd\git.exe' failed with exit code 128

This comment was automatically generated by Dr. CI and updates every 15 minutes.

[github-actions]

$\color{#D29922}\textsf{\Large⚠\kern{0.2cm}\normalsize Warning}$ Result of CPU Benchmark Tests

Total Benchmarks: 91. Improved: $\large\color{#35bf28}7$. Worsened: $\large\color{#d91a1a}6$.

Expand to view detailed results

Name	Max	Mean	Ops	Ops on Repo HEAD	Change
test_single	58.2682ms	57.7201ms	17.3250 Ops/s	17.3332 Ops/s	$\color{#d91a1a}-0.05\%$
test_sync	38.9921ms	32.7087ms	30.5729 Ops/s	31.0508 Ops/s	$\color{#d91a1a}-1.54\%$
test_async	54.0355ms	31.1677ms	32.0845 Ops/s	32.2214 Ops/s	$\color{#d91a1a}-0.43\%$
test_simple	0.5164s	0.4216s	2.3720 Ops/s	2.5443 Ops/s	$\textbf{\color{#d91a1a}-6.77\%}$
test_transformed	0.6449s	0.5722s	1.7477 Ops/s	1.8376 Ops/s	$\color{#d91a1a}-4.89\%$
test_serial	1.3262s	1.2627s	0.7920 Ops/s	0.7980 Ops/s	$\color{#d91a1a}-0.76\%$
test_parallel	1.1714s	1.1011s	0.9082 Ops/s	0.9054 Ops/s	$\color{#35bf28}+0.31\%$
test_step_mdp_speed[True-True-True-True-True]	0.1533ms	24.5134μs	40.7940 KOps/s	40.4697 KOps/s	$\color{#35bf28}+0.80\%$
test_step_mdp_speed[True-True-True-True-False]	48.4510μs	14.4725μs	69.0966 KOps/s	69.4328 KOps/s	$\color{#d91a1a}-0.48\%$
test_step_mdp_speed[True-True-True-False-True]	42.5410μs	14.2628μs	70.1127 KOps/s	69.9997 KOps/s	$\color{#35bf28}+0.16\%$
test_step_mdp_speed[True-True-True-False-False]	52.6600μs	8.3447μs	119.8366 KOps/s	120.2243 KOps/s	$\color{#d91a1a}-0.32\%$
test_step_mdp_speed[True-True-False-True-True]	58.8910μs	26.1916μs	38.1802 KOps/s	38.0864 KOps/s	$\color{#35bf28}+0.25\%$
test_step_mdp_speed[True-True-False-True-False]	43.3510μs	16.0956μs	62.1289 KOps/s	62.4919 KOps/s	$\color{#d91a1a}-0.58\%$
test_step_mdp_speed[True-True-False-False-True]	52.7800μs	15.6647μs	63.8377 KOps/s	63.6273 KOps/s	$\color{#35bf28}+0.33\%$
test_step_mdp_speed[True-True-False-False-False]	36.0680μs	9.8333μs	101.6951 KOps/s	102.1530 KOps/s	$\color{#d91a1a}-0.45\%$
test_step_mdp_speed[True-False-True-True-True]	63.8910μs	27.6803μs	36.1268 KOps/s	36.2084 KOps/s	$\color{#d91a1a}-0.23\%$
test_step_mdp_speed[True-False-True-True-False]	54.0020μs	17.5289μs	57.0485 KOps/s	57.0801 KOps/s	$\color{#d91a1a}-0.06\%$
test_step_mdp_speed[True-False-True-False-True]	42.0500μs	15.7255μs	63.5909 KOps/s	63.3003 KOps/s	$\color{#35bf28}+0.46\%$
test_step_mdp_speed[True-False-True-False-False]	35.0260μs	9.8736μs	101.2806 KOps/s	96.3453 KOps/s	$\textbf{\color{#35bf28}+5.12\%}$
test_step_mdp_speed[True-False-False-True-True]	58.9510μs	29.0794μs	34.3885 KOps/s	33.7138 KOps/s	$\color{#35bf28}+2.00\%$
test_step_mdp_speed[True-False-False-True-False]	51.1360μs	18.8699μs	52.9945 KOps/s	52.7756 KOps/s	$\color{#35bf28}+0.41\%$
test_step_mdp_speed[True-False-False-False-True]	53.0100μs	17.0346μs	58.7041 KOps/s	57.9211 KOps/s	$\color{#35bf28}+1.35\%$
test_step_mdp_speed[True-False-False-False-False]	34.6050μs	11.2528μs	88.8671 KOps/s	89.6627 KOps/s	$\color{#d91a1a}-0.89\%$
test_step_mdp_speed[False-True-True-True-True]	56.7570μs	27.7607μs	36.0221 KOps/s	35.6061 KOps/s	$\color{#35bf28}+1.17\%$
test_step_mdp_speed[False-True-True-True-False]	57.7590μs	17.4229μs	57.3957 KOps/s	56.5673 KOps/s	$\color{#35bf28}+1.46\%$
test_step_mdp_speed[False-True-True-False-True]	46.0560μs	18.7682μs	53.2817 KOps/s	54.5428 KOps/s	$\color{#d91a1a}-2.31\%$
test_step_mdp_speed[False-True-True-False-False]	63.4910μs	11.1461μs	89.7174 KOps/s	87.6156 KOps/s	$\color{#35bf28}+2.40\%$
test_step_mdp_speed[False-True-False-True-True]	66.1340μs	29.1966μs	34.2505 KOps/s	34.2388 KOps/s	$\color{#35bf28}+0.03\%$
test_step_mdp_speed[False-True-False-True-False]	46.0370μs	18.9839μs	52.6761 KOps/s	53.2415 KOps/s	$\color{#d91a1a}-1.06\%$
test_step_mdp_speed[False-True-False-False-True]	67.6670μs	19.9676μs	50.0811 KOps/s	51.1037 KOps/s	$\color{#d91a1a}-2.00\%$
test_step_mdp_speed[False-True-False-False-False]	40.8170μs	12.5893μs	79.4324 KOps/s	81.6348 KOps/s	$\color{#d91a1a}-2.70\%$
test_step_mdp_speed[False-False-True-True-True]	3.1869ms	30.8881μs	32.3749 KOps/s	30.7064 KOps/s	$\textbf{\color{#35bf28}+5.43\%}$
test_step_mdp_speed[False-False-True-True-False]	54.1720μs	20.3621μs	49.1108 KOps/s	49.1508 KOps/s	$\color{#d91a1a}-0.08\%$
test_step_mdp_speed[False-False-True-False-True]	83.5980μs	19.7299μs	50.6845 KOps/s	51.3579 KOps/s	$\color{#d91a1a}-1.31\%$
test_step_mdp_speed[False-False-True-False-False]	38.6130μs	12.5734μs	79.5331 KOps/s	72.8458 KOps/s	$\textbf{\color{#35bf28}+9.18\%}$
test_step_mdp_speed[False-False-False-True-True]	86.4930μs	31.7795μs	31.4669 KOps/s	29.5483 KOps/s	$\textbf{\color{#35bf28}+6.49\%}$
test_step_mdp_speed[False-False-False-True-False]	46.8890μs	21.5778μs	46.3439 KOps/s	46.0473 KOps/s	$\color{#35bf28}+0.64\%$
test_step_mdp_speed[False-False-False-False-True]	87.9060μs	21.0941μs	47.4067 KOps/s	47.6777 KOps/s	$\color{#d91a1a}-0.57\%$
test_step_mdp_speed[False-False-False-False-False]	49.6330μs	13.6880μs	73.0568 KOps/s	72.8544 KOps/s	$\color{#35bf28}+0.28\%$
test_values[generalized_advantage_estimate-True-True]	10.0118ms	9.5491ms	104.7215 Ops/s	106.6554 Ops/s	$\color{#d91a1a}-1.81\%$
test_values[vec_generalized_advantage_estimate-True-True]	38.7313ms	35.5951ms	28.0938 Ops/s	28.0843 Ops/s	$\color{#35bf28}+0.03\%$
test_values[td0_return_estimate-False-False]	0.2004ms	0.1792ms	5.5808 KOps/s	5.9463 KOps/s	$\textbf{\color{#d91a1a}-6.15\%}$
test_values[td1_return_estimate-False-False]	39.3363ms	24.5050ms	40.8079 Ops/s	43.3879 Ops/s	$\textbf{\color{#d91a1a}-5.95\%}$
test_values[vec_td1_return_estimate-False-False]	37.3837ms	35.6434ms	28.0557 Ops/s	28.0238 Ops/s	$\color{#35bf28}+0.11\%$
test_values[td_lambda_return_estimate-True-False]	34.8344ms	34.2007ms	29.2392 Ops/s	29.9063 Ops/s	$\color{#d91a1a}-2.23\%$
test_values[vec_td_lambda_return_estimate-True-False]	38.5814ms	35.7828ms	27.9464 Ops/s	28.0594 Ops/s	$\color{#d91a1a}-0.40\%$
test_gae_speed[generalized_advantage_estimate-False-1-512]	8.4802ms	8.2883ms	120.6524 Ops/s	123.8882 Ops/s	$\color{#d91a1a}-2.61\%$
test_gae_speed[vec_generalized_advantage_estimate-True-1-512]	2.4801ms	2.0209ms	494.8219 Ops/s	498.7956 Ops/s	$\color{#d91a1a}-0.80\%$
test_gae_speed[vec_generalized_advantage_estimate-False-1-512]	0.4251ms	0.3535ms	2.8291 KOps/s	2.8889 KOps/s	$\color{#d91a1a}-2.07\%$
test_gae_speed[vec_generalized_advantage_estimate-True-32-512]	49.9546ms	47.3934ms	21.1000 Ops/s	21.5967 Ops/s	$\color{#d91a1a}-2.30\%$
test_gae_speed[vec_generalized_advantage_estimate-False-32-512]	3.8790ms	3.0298ms	330.0519 Ops/s	329.1609 Ops/s	$\color{#35bf28}+0.27\%$
test_dqn_speed	1.7689ms	1.2972ms	770.8745 Ops/s	764.3089 Ops/s	$\color{#35bf28}+0.86\%$
test_ddpg_speed	3.4857ms	2.7020ms	370.0926 Ops/s	362.4203 Ops/s	$\color{#35bf28}+2.12\%$
test_sac_speed	9.5527ms	7.9834ms	125.2600 Ops/s	126.8541 Ops/s	$\color{#d91a1a}-1.26\%$
test_redq_speed	16.5662ms	12.8989ms	77.5259 Ops/s	79.1867 Ops/s	$\color{#d91a1a}-2.10\%$
test_redq_deprec_speed	13.6242ms	12.5082ms	79.9475 Ops/s	79.2655 Ops/s	$\color{#35bf28}+0.86\%$
test_td3_speed	8.0768ms	7.8435ms	127.4937 Ops/s	129.2277 Ops/s	$\color{#d91a1a}-1.34\%$
test_cql_speed	36.4411ms	35.2359ms	28.3801 Ops/s	28.3911 Ops/s	$\color{#d91a1a}-0.04\%$
test_a2c_speed	9.2594ms	7.2515ms	137.9018 Ops/s	138.1512 Ops/s	$\color{#d91a1a}-0.18\%$
test_ppo_speed	8.1775ms	7.4792ms	133.7044 Ops/s	134.4606 Ops/s	$\color{#d91a1a}-0.56\%$
test_reinforce_speed	15.9634ms	6.6718ms	149.8836 Ops/s	156.1048 Ops/s	$\color{#d91a1a}-3.99\%$
test_iql_speed	33.5061ms	31.7971ms	31.4494 Ops/s	31.3829 Ops/s	$\color{#35bf28}+0.21\%$
test_rb_sample[TensorDictReplayBuffer-ListStorage-RandomSampler-4000]	7.5619ms	4.7770ms	209.3361 Ops/s	208.7312 Ops/s	$\color{#35bf28}+0.29\%$
test_rb_sample[TensorDictReplayBuffer-LazyMemmapStorage-RandomSampler-10000]	93.4754ms	0.5310ms	1.8833 KOps/s	2.1066 KOps/s	$\textbf{\color{#d91a1a}-10.60\%}$
test_rb_sample[TensorDictReplayBuffer-LazyTensorStorage-RandomSampler-10000]	0.6590ms	0.4474ms	2.2349 KOps/s	2.1096 KOps/s	$\textbf{\color{#35bf28}+5.94\%}$
test_rb_sample[TensorDictReplayBuffer-ListStorage-SamplerWithoutReplacement-4000]	7.4543ms	4.7052ms	212.5319 Ops/s	210.3604 Ops/s	$\color{#35bf28}+1.03\%$
test_rb_sample[TensorDictReplayBuffer-LazyMemmapStorage-SamplerWithoutReplacement-10000]	1.3913ms	0.4649ms	2.1509 KOps/s	2.1500 KOps/s	$\color{#35bf28}+0.04\%$
test_rb_sample[TensorDictReplayBuffer-LazyTensorStorage-SamplerWithoutReplacement-10000]	1.3619ms	0.4426ms	2.2592 KOps/s	2.2350 KOps/s	$\color{#35bf28}+1.08\%$
test_rb_sample[TensorDictReplayBuffer-LazyMemmapStorage-sampler6-10000]	1.8422ms	1.6662ms	600.1608 Ops/s	602.7514 Ops/s	$\color{#d91a1a}-0.43\%$
test_rb_sample[TensorDictReplayBuffer-LazyTensorStorage-sampler7-10000]	2.2798ms	1.5900ms	628.9285 Ops/s	635.9370 Ops/s	$\color{#d91a1a}-1.10\%$
test_rb_sample[TensorDictPrioritizedReplayBuffer-ListStorage-None-4000]	5.1389ms	4.8717ms	205.2688 Ops/s	203.0860 Ops/s	$\color{#35bf28}+1.07\%$
test_rb_sample[TensorDictPrioritizedReplayBuffer-LazyMemmapStorage-None-10000]	1.4456ms	0.6103ms	1.6385 KOps/s	1.6494 KOps/s	$\color{#d91a1a}-0.66\%$
test_rb_sample[TensorDictPrioritizedReplayBuffer-LazyTensorStorage-None-10000]	0.8024ms	0.5857ms	1.7073 KOps/s	1.7264 KOps/s	$\color{#d91a1a}-1.10\%$
test_rb_iterate[TensorDictReplayBuffer-ListStorage-RandomSampler-4000]	7.4454ms	4.8148ms	207.6935 Ops/s	208.4881 Ops/s	$\color{#d91a1a}-0.38\%$
test_rb_iterate[TensorDictReplayBuffer-LazyMemmapStorage-RandomSampler-10000]	1.3792ms	0.4731ms	2.1136 KOps/s	2.0951 KOps/s	$\color{#35bf28}+0.88\%$
test_rb_iterate[TensorDictReplayBuffer-LazyTensorStorage-RandomSampler-10000]	0.6176ms	0.4516ms	2.2143 KOps/s	2.1986 KOps/s	$\color{#35bf28}+0.71\%$
test_rb_iterate[TensorDictReplayBuffer-ListStorage-SamplerWithoutReplacement-4000]	6.9827ms	4.7814ms	209.1434 Ops/s	209.3904 Ops/s	$\color{#d91a1a}-0.12\%$
test_rb_iterate[TensorDictReplayBuffer-LazyMemmapStorage-SamplerWithoutReplacement-10000]	0.9239ms	0.4685ms	2.1343 KOps/s	2.1245 KOps/s	$\color{#35bf28}+0.46\%$
test_rb_iterate[TensorDictReplayBuffer-LazyTensorStorage-SamplerWithoutReplacement-10000]	0.6170ms	0.4489ms	2.2277 KOps/s	2.1927 KOps/s	$\color{#35bf28}+1.60\%$
test_rb_iterate[TensorDictPrioritizedReplayBuffer-ListStorage-None-4000]	6.0518ms	4.9947ms	200.2106 Ops/s	204.4546 Ops/s	$\color{#d91a1a}-2.08\%$
test_rb_iterate[TensorDictPrioritizedReplayBuffer-LazyMemmapStorage-None-10000]	1.5261ms	0.6148ms	1.6266 KOps/s	1.6378 KOps/s	$\color{#d91a1a}-0.68\%$
test_rb_iterate[TensorDictPrioritizedReplayBuffer-LazyTensorStorage-None-10000]	0.8607ms	0.5839ms	1.7125 KOps/s	1.5953 KOps/s	$\textbf{\color{#35bf28}+7.35\%}$
test_rb_populate[TensorDictReplayBuffer-ListStorage-RandomSampler-400]	0.1397s	6.4513ms	155.0077 Ops/s	159.7449 Ops/s	$\color{#d91a1a}-2.97\%$
test_rb_populate[TensorDictReplayBuffer-LazyMemmapStorage-RandomSampler-400]	16.8486ms	12.8197ms	78.0048 Ops/s	77.8088 Ops/s	$\color{#35bf28}+0.25\%$
test_rb_populate[TensorDictReplayBuffer-LazyTensorStorage-RandomSampler-400]	5.2566ms	1.1993ms	833.7925 Ops/s	907.4908 Ops/s	$\textbf{\color{#d91a1a}-8.12\%}$
test_rb_populate[TensorDictReplayBuffer-ListStorage-SamplerWithoutReplacement-400]	0.1100s	7.9706ms	125.4618 Ops/s	174.8311 Ops/s	$\textbf{\color{#d91a1a}-28.24\%}$
test_rb_populate[TensorDictReplayBuffer-LazyMemmapStorage-SamplerWithoutReplacement-400]	22.6082ms	12.9894ms	76.9860 Ops/s	78.5196 Ops/s	$\color{#d91a1a}-1.95\%$
test_rb_populate[TensorDictReplayBuffer-LazyTensorStorage-SamplerWithoutReplacement-400]	4.6928ms	1.2049ms	829.9340 Ops/s	813.4756 Ops/s	$\color{#35bf28}+2.02\%$
test_rb_populate[TensorDictPrioritizedReplayBuffer-ListStorage-None-400]	0.1085s	5.9980ms	166.7234 Ops/s	122.3864 Ops/s	$\textbf{\color{#35bf28}+36.23\%}$
test_rb_populate[TensorDictPrioritizedReplayBuffer-LazyMemmapStorage-None-400]	17.3861ms	13.0238ms	76.7824 Ops/s	78.0535 Ops/s	$\color{#d91a1a}-1.63\%$
test_rb_populate[TensorDictPrioritizedReplayBuffer-LazyTensorStorage-None-400]	2.0036ms	1.2734ms	785.2990 Ops/s	802.7707 Ops/s	$\color{#d91a1a}-2.18\%$

[github-actions]

$\color{#D29922}\textsf{\Large⚠\kern{0.2cm}\normalsize Warning}$ Result of GPU Benchmark Tests

Total Benchmarks: 94. Improved: $\large\color{#35bf28}1$. Worsened: $\large\color{#d91a1a}3$.

Expand to view detailed results

Name	Max	Mean	Ops	Ops on Repo HEAD	Change
test_single	0.1071s	0.1063s	9.4080 Ops/s	9.2346 Ops/s	$\color{#35bf28}+1.88\%$
test_sync	95.8157ms	95.3427ms	10.4885 Ops/s	10.5148 Ops/s	$\color{#d91a1a}-0.25\%$
test_async	0.1795s	90.8605ms	11.0059 Ops/s	11.1272 Ops/s	$\color{#d91a1a}-1.09\%$
test_single_pixels	0.1176s	0.1170s	8.5472 Ops/s	8.5528 Ops/s	$\color{#d91a1a}-0.06\%$
test_sync_pixels	76.2023ms	73.2369ms	13.6543 Ops/s	13.2133 Ops/s	$\color{#35bf28}+3.34\%$
test_async_pixels	0.2288s	72.0934ms	13.8709 Ops/s	14.2168 Ops/s	$\color{#d91a1a}-2.43\%$
test_simple	0.7655s	0.7644s	1.3083 Ops/s	1.3055 Ops/s	$\color{#35bf28}+0.21\%$
test_transformed	1.1005s	1.0165s	0.9838 Ops/s	0.9865 Ops/s	$\color{#d91a1a}-0.28\%$
test_serial	2.2776s	2.2006s	0.4544 Ops/s	0.4550 Ops/s	$\color{#d91a1a}-0.13\%$
test_parallel	2.0116s	1.9607s	0.5100 Ops/s	0.5120 Ops/s	$\color{#d91a1a}-0.38\%$
test_step_mdp_speed[True-True-True-True-True]	0.2195ms	36.1838μs	27.6367 KOps/s	27.3084 KOps/s	$\color{#35bf28}+1.20\%$
test_step_mdp_speed[True-True-True-True-False]	0.2131ms	20.3721μs	49.0867 KOps/s	48.6954 KOps/s	$\color{#35bf28}+0.80\%$
test_step_mdp_speed[True-True-True-False-True]	0.2150ms	19.9546μs	50.1137 KOps/s	48.5481 KOps/s	$\color{#35bf28}+3.22\%$
test_step_mdp_speed[True-True-True-False-False]	32.7210μs	11.5235μs	86.7794 KOps/s	85.6085 KOps/s	$\color{#35bf28}+1.37\%$
test_step_mdp_speed[True-True-False-True-True]	99.5920μs	37.6585μs	26.5544 KOps/s	26.0437 KOps/s	$\color{#35bf28}+1.96\%$
test_step_mdp_speed[True-True-False-True-False]	0.2227ms	22.5782μs	44.2905 KOps/s	44.2848 KOps/s	$\color{#35bf28}+0.01\%$
test_step_mdp_speed[True-True-False-False-True]	41.6610μs	22.1207μs	45.2065 KOps/s	43.3706 KOps/s	$\color{#35bf28}+4.23\%$
test_step_mdp_speed[True-True-False-False-False]	32.2000μs	13.6518μs	73.2504 KOps/s	72.2638 KOps/s	$\color{#35bf28}+1.37\%$
test_step_mdp_speed[True-False-True-True-True]	73.4710μs	39.9728μs	25.0170 KOps/s	24.6370 KOps/s	$\color{#35bf28}+1.54\%$
test_step_mdp_speed[True-False-True-True-False]	41.5800μs	24.8978μs	40.1641 KOps/s	40.2794 KOps/s	$\color{#d91a1a}-0.29\%$
test_step_mdp_speed[True-False-True-False-True]	87.8220μs	22.1144μs	45.2194 KOps/s	43.8972 KOps/s	$\color{#35bf28}+3.01\%$
test_step_mdp_speed[True-False-True-False-False]	37.1710μs	13.6391μs	73.3187 KOps/s	71.7995 KOps/s	$\color{#35bf28}+2.12\%$
test_step_mdp_speed[True-False-False-True-True]	73.0710μs	41.7556μs	23.9489 KOps/s	23.3529 KOps/s	$\color{#35bf28}+2.55\%$
test_step_mdp_speed[True-False-False-True-False]	51.4510μs	26.8641μs	37.2244 KOps/s	37.0504 KOps/s	$\color{#35bf28}+0.47\%$
test_step_mdp_speed[True-False-False-False-True]	0.2009ms	24.0593μs	41.5639 KOps/s	40.0056 KOps/s	$\color{#35bf28}+3.90\%$
test_step_mdp_speed[True-False-False-False-False]	72.3110μs	15.7708μs	63.4083 KOps/s	62.5053 KOps/s	$\color{#35bf28}+1.44\%$
test_step_mdp_speed[False-True-True-True-True]	67.1210μs	39.9590μs	25.0257 KOps/s	24.6529 KOps/s	$\color{#35bf28}+1.51\%$
test_step_mdp_speed[False-True-True-True-False]	42.6410μs	24.6531μs	40.5628 KOps/s	40.4785 KOps/s	$\color{#35bf28}+0.21\%$
test_step_mdp_speed[False-True-True-False-True]	50.6310μs	26.5312μs	37.6915 KOps/s	36.8709 KOps/s	$\color{#35bf28}+2.23\%$
test_step_mdp_speed[False-True-True-False-False]	0.1223ms	15.6992μs	63.6975 KOps/s	63.0433 KOps/s	$\color{#35bf28}+1.04\%$
test_step_mdp_speed[False-True-False-True-True]	66.9010μs	42.0162μs	23.8003 KOps/s	23.4188 KOps/s	$\color{#35bf28}+1.63\%$
test_step_mdp_speed[False-True-False-True-False]	45.2610μs	27.0456μs	36.9746 KOps/s	37.2539 KOps/s	$\color{#d91a1a}-0.75\%$
test_step_mdp_speed[False-True-False-False-True]	54.9810μs	28.6258μs	34.9335 KOps/s	34.7142 KOps/s	$\color{#35bf28}+0.63\%$
test_step_mdp_speed[False-True-False-False-False]	36.5010μs	17.7656μs	56.2884 KOps/s	55.5137 KOps/s	$\color{#35bf28}+1.40\%$
test_step_mdp_speed[False-False-True-True-True]	4.3861ms	44.9561μs	22.2439 KOps/s	22.2065 KOps/s	$\color{#35bf28}+0.17\%$
test_step_mdp_speed[False-False-True-True-False]	55.0410μs	29.4658μs	33.9376 KOps/s	34.2954 KOps/s	$\color{#d91a1a}-1.04\%$
test_step_mdp_speed[False-False-True-False-True]	85.8120μs	28.4378μs	35.1645 KOps/s	34.5637 KOps/s	$\color{#35bf28}+1.74\%$
test_step_mdp_speed[False-False-True-False-False]	41.5110μs	17.8377μs	56.0609 KOps/s	54.7608 KOps/s	$\color{#35bf28}+2.37\%$
test_step_mdp_speed[False-False-False-True-True]	64.5510μs	46.0800μs	21.7014 KOps/s	21.6514 KOps/s	$\color{#35bf28}+0.23\%$
test_step_mdp_speed[False-False-False-True-False]	70.9810μs	30.8750μs	32.3887 KOps/s	32.1790 KOps/s	$\color{#35bf28}+0.65\%$
test_step_mdp_speed[False-False-False-False-True]	0.1317ms	30.3142μs	32.9879 KOps/s	32.8030 KOps/s	$\color{#35bf28}+0.56\%$
test_step_mdp_speed[False-False-False-False-False]	40.0300μs	19.6086μs	50.9980 KOps/s	49.7584 KOps/s	$\color{#35bf28}+2.49\%$
test_values[generalized_advantage_estimate-True-True]	24.3704ms	23.7408ms	42.1215 Ops/s	41.9824 Ops/s	$\color{#35bf28}+0.33\%$
test_values[vec_generalized_advantage_estimate-True-True]	91.1016ms	2.7150ms	368.3268 Ops/s	353.1535 Ops/s	$\color{#35bf28}+4.30\%$
test_values[td0_return_estimate-False-False]	89.8220μs	64.7722μs	15.4387 KOps/s	15.6809 KOps/s	$\color{#d91a1a}-1.54\%$
test_values[td1_return_estimate-False-False]	54.0862ms	53.1975ms	18.7979 Ops/s	18.8854 Ops/s	$\color{#d91a1a}-0.46\%$
test_values[vec_td1_return_estimate-False-False]	1.3939ms	1.0764ms	928.9844 Ops/s	927.2865 Ops/s	$\color{#35bf28}+0.18\%$
test_values[td_lambda_return_estimate-True-False]	85.9313ms	84.8368ms	11.7873 Ops/s	11.7385 Ops/s	$\color{#35bf28}+0.42\%$
test_values[vec_td_lambda_return_estimate-True-False]	1.4030ms	1.0817ms	924.4521 Ops/s	934.5505 Ops/s	$\color{#d91a1a}-1.08\%$
test_gae_speed[generalized_advantage_estimate-False-1-512]	24.5101ms	23.7497ms	42.1058 Ops/s	42.3697 Ops/s	$\color{#d91a1a}-0.62\%$
test_gae_speed[vec_generalized_advantage_estimate-True-1-512]	0.9553ms	0.7116ms	1.4054 KOps/s	1.3706 KOps/s	$\color{#35bf28}+2.54\%$
test_gae_speed[vec_generalized_advantage_estimate-False-1-512]	0.8229ms	0.6632ms	1.5079 KOps/s	1.5166 KOps/s	$\color{#d91a1a}-0.57\%$
test_gae_speed[vec_generalized_advantage_estimate-True-32-512]	1.6675ms	1.4562ms	686.6987 Ops/s	686.5824 Ops/s	$\color{#35bf28}+0.02\%$
test_gae_speed[vec_generalized_advantage_estimate-False-32-512]	0.8444ms	0.6735ms	1.4849 KOps/s	1.4892 KOps/s	$\color{#d91a1a}-0.29\%$
test_dqn_speed	8.1540ms	1.3764ms	726.5411 Ops/s	733.7462 Ops/s	$\color{#d91a1a}-0.98\%$
test_ddpg_speed	3.1092ms	2.7652ms	361.6407 Ops/s	365.1733 Ops/s	$\color{#d91a1a}-0.97\%$
test_sac_speed	8.2861ms	7.9035ms	126.5258 Ops/s	126.0114 Ops/s	$\color{#35bf28}+0.41\%$
test_redq_speed	12.4911ms	10.6664ms	93.7526 Ops/s	94.8880 Ops/s	$\color{#d91a1a}-1.20\%$
test_redq_deprec_speed	11.4264ms	10.8077ms	92.5267 Ops/s	88.9475 Ops/s	$\color{#35bf28}+4.02\%$
test_td3_speed	7.9473ms	7.8803ms	126.8991 Ops/s	127.4314 Ops/s	$\color{#d91a1a}-0.42\%$
test_cql_speed	27.5194ms	25.9982ms	38.4642 Ops/s	38.3066 Ops/s	$\color{#35bf28}+0.41\%$
test_a2c_speed	6.0266ms	5.6549ms	176.8383 Ops/s	177.4844 Ops/s	$\color{#d91a1a}-0.36\%$
test_ppo_speed	6.7246ms	5.9817ms	167.1762 Ops/s	168.2243 Ops/s	$\color{#d91a1a}-0.62\%$
test_reinforce_speed	4.9117ms	4.5227ms	221.1072 Ops/s	219.9169 Ops/s	$\color{#35bf28}+0.54\%$
test_iql_speed	20.7447ms	20.0521ms	49.8702 Ops/s	50.1454 Ops/s	$\color{#d91a1a}-0.55\%$
test_rb_sample[TensorDictReplayBuffer-ListStorage-RandomSampler-4000]	6.9823ms	6.5893ms	151.7619 Ops/s	149.5230 Ops/s	$\color{#35bf28}+1.50\%$
test_rb_sample[TensorDictReplayBuffer-LazyMemmapStorage-RandomSampler-10000]	1.1753ms	0.5290ms	1.8905 KOps/s	1.8583 KOps/s	$\color{#35bf28}+1.73\%$
test_rb_sample[TensorDictReplayBuffer-LazyTensorStorage-RandomSampler-10000]	0.8393ms	0.5095ms	1.9627 KOps/s	1.9297 KOps/s	$\color{#35bf28}+1.71\%$
test_rb_sample[TensorDictReplayBuffer-ListStorage-SamplerWithoutReplacement-4000]	6.9628ms	6.5387ms	152.9347 Ops/s	152.5440 Ops/s	$\color{#35bf28}+0.26\%$
test_rb_sample[TensorDictReplayBuffer-LazyMemmapStorage-SamplerWithoutReplacement-10000]	1.9910ms	0.5238ms	1.9093 KOps/s	1.8975 KOps/s	$\color{#35bf28}+0.62\%$
test_rb_sample[TensorDictReplayBuffer-LazyTensorStorage-SamplerWithoutReplacement-10000]	0.7005ms	0.5028ms	1.9887 KOps/s	1.9707 KOps/s	$\color{#35bf28}+0.92\%$
test_rb_sample[TensorDictReplayBuffer-LazyMemmapStorage-sampler6-10000]	2.3009ms	2.0228ms	494.3678 Ops/s	495.0281 Ops/s	$\color{#d91a1a}-0.13\%$
test_rb_sample[TensorDictReplayBuffer-LazyTensorStorage-sampler7-10000]	2.2142ms	1.9241ms	519.7203 Ops/s	514.9528 Ops/s	$\color{#35bf28}+0.93\%$
test_rb_sample[TensorDictPrioritizedReplayBuffer-ListStorage-None-4000]	7.0410ms	6.7481ms	148.1900 Ops/s	147.9929 Ops/s	$\color{#35bf28}+0.13\%$
test_rb_sample[TensorDictPrioritizedReplayBuffer-LazyMemmapStorage-None-10000]	0.1366s	0.8023ms	1.2464 KOps/s	1.2455 KOps/s	$\color{#35bf28}+0.08\%$
test_rb_sample[TensorDictPrioritizedReplayBuffer-LazyTensorStorage-None-10000]	1.0166ms	0.6886ms	1.4523 KOps/s	1.4967 KOps/s	$\color{#d91a1a}-2.97\%$
test_rb_iterate[TensorDictReplayBuffer-ListStorage-RandomSampler-4000]	7.0381ms	6.6104ms	151.2775 Ops/s	146.3002 Ops/s	$\color{#35bf28}+3.40\%$
test_rb_iterate[TensorDictReplayBuffer-LazyMemmapStorage-RandomSampler-10000]	1.2700ms	0.5319ms	1.8800 KOps/s	1.8813 KOps/s	$\color{#d91a1a}-0.07\%$
test_rb_iterate[TensorDictReplayBuffer-LazyTensorStorage-RandomSampler-10000]	0.7368ms	0.5163ms	1.9368 KOps/s	1.9557 KOps/s	$\color{#d91a1a}-0.97\%$
test_rb_iterate[TensorDictReplayBuffer-ListStorage-SamplerWithoutReplacement-4000]	7.2319ms	6.5732ms	152.1335 Ops/s	152.5530 Ops/s	$\color{#d91a1a}-0.28\%$
test_rb_iterate[TensorDictReplayBuffer-LazyMemmapStorage-SamplerWithoutReplacement-10000]	0.1452s	0.7296ms	1.3707 KOps/s	1.8872 KOps/s	$\textbf{\color{#d91a1a}-27.37\%}$
test_rb_iterate[TensorDictReplayBuffer-LazyTensorStorage-SamplerWithoutReplacement-10000]	0.7720ms	0.5053ms	1.9789 KOps/s	1.9393 KOps/s	$\color{#35bf28}+2.04\%$
test_rb_iterate[TensorDictPrioritizedReplayBuffer-ListStorage-None-4000]	7.2556ms	6.7942ms	147.1837 Ops/s	146.2944 Ops/s	$\color{#35bf28}+0.61\%$
test_rb_iterate[TensorDictPrioritizedReplayBuffer-LazyMemmapStorage-None-10000]	0.8675ms	0.6864ms	1.4570 KOps/s	1.4685 KOps/s	$\color{#d91a1a}-0.79\%$
test_rb_iterate[TensorDictPrioritizedReplayBuffer-LazyTensorStorage-None-10000]	0.8971ms	0.6710ms	1.4903 KOps/s	1.4941 KOps/s	$\color{#d91a1a}-0.26\%$
test_rb_populate[TensorDictReplayBuffer-ListStorage-RandomSampler-400]	0.1364s	7.8758ms	126.9717 Ops/s	121.4874 Ops/s	$\color{#35bf28}+4.51\%$
test_rb_populate[TensorDictReplayBuffer-LazyMemmapStorage-RandomSampler-400]	18.4505ms	15.8457ms	63.1087 Ops/s	62.4643 Ops/s	$\color{#35bf28}+1.03\%$
test_rb_populate[TensorDictReplayBuffer-LazyTensorStorage-RandomSampler-400]	7.7046ms	1.4145ms	706.9424 Ops/s	779.2424 Ops/s	$\textbf{\color{#d91a1a}-9.28\%}$
test_rb_populate[TensorDictReplayBuffer-ListStorage-SamplerWithoutReplacement-400]	0.1398s	10.6092ms	94.2582 Ops/s	125.6071 Ops/s	$\textbf{\color{#d91a1a}-24.96\%}$
test_rb_populate[TensorDictReplayBuffer-LazyMemmapStorage-SamplerWithoutReplacement-400]	18.5954ms	15.8572ms	63.0627 Ops/s	53.4797 Ops/s	$\textbf{\color{#35bf28}+17.92\%}$
test_rb_populate[TensorDictReplayBuffer-LazyTensorStorage-SamplerWithoutReplacement-400]	2.4351ms	1.2131ms	824.3160 Ops/s	793.4239 Ops/s	$\color{#35bf28}+3.89\%$
test_rb_populate[TensorDictPrioritizedReplayBuffer-ListStorage-None-400]	0.1367s	8.0768ms	123.8109 Ops/s	122.9382 Ops/s	$\color{#35bf28}+0.71\%$
test_rb_populate[TensorDictPrioritizedReplayBuffer-LazyMemmapStorage-None-400]	18.9675ms	16.0323ms	62.3743 Ops/s	61.6939 Ops/s	$\color{#35bf28}+1.10\%$
test_rb_populate[TensorDictPrioritizedReplayBuffer-LazyTensorStorage-None-400]	7.7326ms	1.5524ms	644.1825 Ops/s	661.1605 Ops/s	$\color{#d91a1a}-2.57\%$