[BugFix] Fix schemes and refactor collectors to make them readable

[vmoens]

Stack from ghstack (oldest at bottom):

• -> [BugFix] Fix schemes and refactor collectors to make them readable #3226
• [Refactor] Non-daemonic processes in PEnv #3225

[pytorch-bot]

🔗 Helpful Links

🧪 See artifacts and rendered test results at hud.pytorch.org/pr/pytorch/rl/3226

• 📄 Preview Python docs built from this PR

Note: Links to docs will display an error until the docs builds have been completed.

❌ 8 New Failures, 3 Cancelled Jobs, 4 Unrelated Failures

As of commit 87047ec with merge base 13434eb ():

NEW FAILURES - The following jobs have failed:

• Continuous Benchmark (PR) / GPU Pytest benchmark (gh)
  Process completed with exit code 139.
• Habitat Tests on Linux / tests (3.9, 12.8) / linux-job (gh)
  RuntimeError: Command docker exec -t 0948a12619d89160cf595a8a4fa75e065293f8789f5683f7d64d4ae64e14b8dd /exec failed with exit code 1
• Libs Tests on Linux / unittests-gym (3.9, 12.8) / linux-job (gh)
  RuntimeError: Command docker exec -t e469e3d0c51eb6f2a8ba237077ac33e3abb7cde708f4905cbe59614764fba995 /exec failed with exit code 1
• Push Binary Nightly / build-wheel-unix (macos, macos-latest, 3.9, cp39-cp39, cpu, cpu) (gh)
• Push Binary Nightly / test-wheel-windows (3.9, 3.9) (gh)
  Process completed with exit code 1.
• SOTA Tests on Linux / tests (3.9, 12.8) / linux-job (gh)
  RuntimeError: Command docker exec -t 8e0f10d23261151b4ab3bf6dbd240acd17b40c5eb95f150b2f3b4ced3a88682d /exec failed with exit code 1
• Unit-tests on Linux / tests-cpu (3.10) / linux-job (gh)
  test/test_env.py::TestNonTensorEnv::test_parallel[False-True]
• Unit-tests on Linux / tests-optdeps (3.11, 12.8) / linux-job (gh)
  test/test_collector.py::TestLibThreading::test_num_threads

CANCELLED JOBS - The following jobs were cancelled. Please retry:

• Unit-tests on Linux / tests-gpu (3.11, 12.8) / linux-job (gh)
  ##[error]The operation was canceled.
• Unit-tests on Linux / tests-olddeps (3.9, 11.6) / linux-job (gh)
  ##[error]The operation was canceled.
• Unit-tests on Linux / tests-stable-gpu (3.10, 11.8) / linux-job (gh)
  ##[error]The operation was canceled.

BROKEN TRUNK - The following jobs failed but were present on the merge base:

👉 Rebase onto the `viable/strict` branch to avoid these failures

• Push Binary Nightly / build-wheel-unix (linux, ubuntu-22.04, 3.9, cp39-cp39, cpu, cpu) (gh) (trunk failure)
• Push Binary Nightly / build-wheel-windows (3.9, 3.9) (gh) (trunk failure)
  Process completed with exit code 1.
• Push Binary Nightly / test-wheel-unix (linux, ubuntu-22.04, 3.9, cp39-cp39, cpu, cpu) (gh) (trunk failure)
  Process completed with exit code 1.
• Push Binary Nightly / test-wheel-unix (macos, macos-latest, 3.9, cp39-cp39, cpu, cpu) (gh) (trunk failure)

This comment was automatically generated by Dr. CI and updates every 15 minutes.

[github-actions]

$\color{#D29922}\textsf{\Large⚠\kern{0.2cm}\normalsize Warning}$ Result of CPU Benchmark Tests

Total Benchmarks: 154. Improved: $\large\color{#35bf28}31$. Worsened: $\large\color{#d91a1a}4$.

Expand to view detailed results

Name	Max	Mean	Ops	Ops on Repo HEAD	Change
test_tensor_to_bytestream_speed[pickle]	85.9891μs	84.4846μs	11.8365 KOps/s	11.6079 KOps/s	$\color{#35bf28}+1.97\%$
test_tensor_to_bytestream_speed[torch.save]	0.1458ms	0.1444ms	6.9250 KOps/s	6.9523 KOps/s	$\color{#d91a1a}-0.39\%$
test_tensor_to_bytestream_speed[untyped_storage]	0.1261s	0.1249s	8.0080 Ops/s	7.9074 Ops/s	$\color{#35bf28}+1.27\%$
test_tensor_to_bytestream_speed[numpy]	2.8079μs	2.7951μs	357.7750 KOps/s	347.3740 KOps/s	$\color{#35bf28}+2.99\%$
test_tensor_to_bytestream_speed[safetensors]	43.8415μs	43.5802μs	22.9462 KOps/s	22.2101 KOps/s	$\color{#35bf28}+3.31\%$
test_simple	0.5606s	0.5529s	1.8086 Ops/s	1.7198 Ops/s	$\textbf{\color{#35bf28}+5.16\%}$
test_transformed	1.2135s	1.1258s	0.8883 Ops/s	0.8750 Ops/s	$\color{#35bf28}+1.51\%$
test_serial	1.6805s	1.6696s	0.5990 Ops/s	0.5884 Ops/s	$\color{#35bf28}+1.79\%$
test_parallel	1.1769s	1.0915s	0.9162 Ops/s	0.9198 Ops/s	$\color{#d91a1a}-0.40\%$
test_step_mdp_speed[True-True-True-True-True]	0.1820ms	44.2679μs	22.5897 KOps/s	22.1163 KOps/s	$\color{#35bf28}+2.14\%$
test_step_mdp_speed[True-True-True-True-False]	51.9510μs	24.6021μs	40.6470 KOps/s	39.9479 KOps/s	$\color{#35bf28}+1.75\%$
test_step_mdp_speed[True-True-True-False-True]	0.1068ms	24.9539μs	40.0739 KOps/s	39.5909 KOps/s	$\color{#35bf28}+1.22\%$
test_step_mdp_speed[True-True-True-False-False]	0.1047ms	13.7335μs	72.8144 KOps/s	72.1307 KOps/s	$\color{#35bf28}+0.95\%$
test_step_mdp_speed[True-True-False-True-True]	0.1522ms	47.5295μs	21.0396 KOps/s	20.8241 KOps/s	$\color{#35bf28}+1.03\%$
test_step_mdp_speed[True-True-False-True-False]	63.6410μs	27.7102μs	36.0878 KOps/s	36.0901 KOps/s	$-0.01\%$
test_step_mdp_speed[True-True-False-False-True]	0.1233ms	27.7146μs	36.0821 KOps/s	35.3421 KOps/s	$\color{#35bf28}+2.09\%$
test_step_mdp_speed[True-True-False-False-False]	96.4410μs	16.2790μs	61.4287 KOps/s	60.4232 KOps/s	$\color{#35bf28}+1.66\%$
test_step_mdp_speed[True-False-True-True-True]	91.1910μs	49.9700μs	20.0120 KOps/s	19.8826 KOps/s	$\color{#35bf28}+0.65\%$
test_step_mdp_speed[True-False-True-True-False]	0.1121ms	30.3601μs	32.9379 KOps/s	32.6124 KOps/s	$\color{#35bf28}+1.00\%$
test_step_mdp_speed[True-False-True-False-True]	0.1298ms	27.7027μs	36.0975 KOps/s	35.1945 KOps/s	$\color{#35bf28}+2.57\%$
test_step_mdp_speed[True-False-True-False-False]	0.1106ms	16.3446μs	61.1822 KOps/s	60.6716 KOps/s	$\color{#35bf28}+0.84\%$
test_step_mdp_speed[True-False-False-True-True]	0.1614ms	52.6304μs	19.0004 KOps/s	18.8705 KOps/s	$\color{#35bf28}+0.69\%$
test_step_mdp_speed[True-False-False-True-False]	66.9110μs	33.2869μs	30.0418 KOps/s	30.1991 KOps/s	$\color{#d91a1a}-0.52\%$
test_step_mdp_speed[True-False-False-False-True]	0.1230ms	30.3963μs	32.8988 KOps/s	32.8714 KOps/s	$\color{#35bf28}+0.08\%$
test_step_mdp_speed[True-False-False-False-False]	0.1172ms	19.1219μs	52.2960 KOps/s	52.6606 KOps/s	$\color{#d91a1a}-0.69\%$
test_step_mdp_speed[False-True-True-True-True]	98.1620μs	50.3086μs	19.8773 KOps/s	19.9105 KOps/s	$\color{#d91a1a}-0.17\%$
test_step_mdp_speed[False-True-True-True-False]	0.1086ms	29.9916μs	33.3427 KOps/s	33.2213 KOps/s	$\color{#35bf28}+0.37\%$
test_step_mdp_speed[False-True-True-False-True]	2.4470ms	31.7562μs	31.4899 KOps/s	31.0316 KOps/s	$\color{#35bf28}+1.48\%$
test_step_mdp_speed[False-True-True-False-False]	99.7920μs	18.0010μs	55.5526 KOps/s	53.9147 KOps/s	$\color{#35bf28}+3.04\%$
test_step_mdp_speed[False-True-False-True-True]	87.3410μs	52.7818μs	18.9459 KOps/s	18.5174 KOps/s	$\color{#35bf28}+2.31\%$
test_step_mdp_speed[False-True-False-True-False]	0.1322ms	32.7661μs	30.5194 KOps/s	29.9650 KOps/s	$\color{#35bf28}+1.85\%$
test_step_mdp_speed[False-True-False-False-True]	0.1093ms	33.8843μs	29.5122 KOps/s	28.8692 KOps/s	$\color{#35bf28}+2.23\%$
test_step_mdp_speed[False-True-False-False-False]	96.4420μs	20.6068μs	48.5278 KOps/s	47.8780 KOps/s	$\color{#35bf28}+1.36\%$
test_step_mdp_speed[False-False-True-True-True]	88.6720μs	55.4607μs	18.0308 KOps/s	17.9099 KOps/s	$\color{#35bf28}+0.67\%$
test_step_mdp_speed[False-False-True-True-False]	72.8510μs	35.5601μs	28.1214 KOps/s	28.0485 KOps/s	$\color{#35bf28}+0.26\%$
test_step_mdp_speed[False-False-True-False-True]	0.1131ms	34.2341μs	29.2107 KOps/s	29.2278 KOps/s	$\color{#d91a1a}-0.06\%$
test_step_mdp_speed[False-False-True-False-False]	0.1023ms	20.8456μs	47.9718 KOps/s	48.6005 KOps/s	$\color{#d91a1a}-1.29\%$
test_step_mdp_speed[False-False-False-True-True]	0.1323ms	57.5449μs	17.3777 KOps/s	17.0527 KOps/s	$\color{#35bf28}+1.91\%$
test_step_mdp_speed[False-False-False-True-False]	0.1076ms	38.2976μs	26.1113 KOps/s	25.9051 KOps/s	$\color{#35bf28}+0.80\%$
test_step_mdp_speed[False-False-False-False-True]	0.4349ms	36.4287μs	27.4509 KOps/s	27.1319 KOps/s	$\color{#35bf28}+1.18\%$
test_step_mdp_speed[False-False-False-False-False]	0.1021ms	23.0537μs	43.3769 KOps/s	43.4124 KOps/s	$\color{#d91a1a}-0.08\%$
test_values[generalized_advantage_estimate-True-True]	10.6754ms	10.2006ms	98.0330 Ops/s	102.9945 Ops/s	$\color{#d91a1a}-4.82\%$
test_values[vec_generalized_advantage_estimate-True-True]	15.2123ms	11.2494ms	88.8938 Ops/s	90.1273 Ops/s	$\color{#d91a1a}-1.37\%$
test_values[td0_return_estimate-False-False]	0.2655ms	0.1380ms	7.2444 KOps/s	8.1135 KOps/s	$\textbf{\color{#d91a1a}-10.71\%}$
test_values[td1_return_estimate-False-False]	27.9881ms	27.4020ms	36.4937 Ops/s	37.0711 Ops/s	$\color{#d91a1a}-1.56\%$
test_values[vec_td1_return_estimate-False-False]	11.6170ms	11.3036ms	88.4673 Ops/s	89.7956 Ops/s	$\color{#d91a1a}-1.48\%$
test_values[td_lambda_return_estimate-True-False]	41.9574ms	41.4036ms	24.1525 Ops/s	24.3406 Ops/s	$\color{#d91a1a}-0.77\%$
test_values[vec_td_lambda_return_estimate-True-False]	11.4917ms	11.2608ms	88.8038 Ops/s	90.3583 Ops/s	$\color{#d91a1a}-1.72\%$
test_gae_speed[generalized_advantage_estimate-False-1-512]	9.1889ms	8.7690ms	114.0380 Ops/s	118.9182 Ops/s	$\color{#d91a1a}-4.10\%$
test_gae_speed[vec_generalized_advantage_estimate-True-1-512]	1.8898ms	1.5582ms	641.7682 Ops/s	666.1672 Ops/s	$\color{#d91a1a}-3.66\%$
test_gae_speed[vec_generalized_advantage_estimate-False-1-512]	0.8784ms	0.4191ms	2.3859 KOps/s	2.4336 KOps/s	$\color{#d91a1a}-1.96\%$
test_gae_speed[vec_generalized_advantage_estimate-True-32-512]	34.7217ms	29.9762ms	33.3598 Ops/s	31.1664 Ops/s	$\textbf{\color{#35bf28}+7.04\%}$
test_gae_speed[vec_generalized_advantage_estimate-False-32-512]	2.0297ms	1.7311ms	577.6579 Ops/s	581.7910 Ops/s	$\color{#d91a1a}-0.71\%$
test_dqn_speed[False-None]	6.7861ms	1.4591ms	685.3403 Ops/s	669.9520 Ops/s	$\color{#35bf28}+2.30\%$
test_dqn_speed[False-backward]	2.0665ms	1.9676ms	508.2363 Ops/s	499.2472 Ops/s	$\color{#35bf28}+1.80\%$
test_dqn_speed[True-None]	0.8352ms	0.5258ms	1.9018 KOps/s	1.8935 KOps/s	$\color{#35bf28}+0.44\%$
test_dqn_speed[True-backward]	1.0775ms	0.9756ms	1.0250 KOps/s	924.0461 Ops/s	$\textbf{\color{#35bf28}+10.92\%}$
test_dqn_speed[reduce-overhead-None]	0.6738ms	0.5206ms	1.9209 KOps/s	1.9029 KOps/s	$\color{#35bf28}+0.95\%$
test_dqn_speed[reduce-overhead-backward]	1.0572ms	0.9749ms	1.0257 KOps/s	955.4282 Ops/s	$\textbf{\color{#35bf28}+7.36\%}$
test_ddpg_speed[False-None]	3.6009ms	2.9924ms	334.1746 Ops/s	324.9151 Ops/s	$\color{#35bf28}+2.85\%$
test_ddpg_speed[False-backward]	4.4631ms	4.1661ms	240.0324 Ops/s	235.1170 Ops/s	$\color{#35bf28}+2.09\%$
test_ddpg_speed[True-None]	1.7710ms	1.4050ms	711.7268 Ops/s	698.9978 Ops/s	$\color{#35bf28}+1.82\%$
test_ddpg_speed[True-backward]	2.5178ms	2.4003ms	416.6099 Ops/s	356.4180 Ops/s	$\textbf{\color{#35bf28}+16.89\%}$
test_ddpg_speed[reduce-overhead-None]	1.6412ms	1.3941ms	717.3000 Ops/s	710.7299 Ops/s	$\color{#35bf28}+0.92\%$
test_ddpg_speed[reduce-overhead-backward]	2.5267ms	2.3701ms	421.9251 Ops/s	422.3323 Ops/s	$\color{#d91a1a}-0.10\%$
test_sac_speed[False-None]	8.4214ms	8.0071ms	124.8890 Ops/s	122.8011 Ops/s	$\color{#35bf28}+1.70\%$
test_sac_speed[False-backward]	11.6940ms	11.2420ms	88.9524 Ops/s	87.3607 Ops/s	$\color{#35bf28}+1.82\%$
test_sac_speed[True-None]	2.6298ms	2.1273ms	470.0811 Ops/s	471.8855 Ops/s	$\color{#d91a1a}-0.38\%$
test_sac_speed[True-backward]	4.0567ms	3.9638ms	252.2814 Ops/s	250.5325 Ops/s	$\color{#35bf28}+0.70\%$
test_sac_speed[reduce-overhead-None]	2.4805ms	2.0807ms	480.6067 Ops/s	461.4857 Ops/s	$\color{#35bf28}+4.14\%$
test_sac_speed[reduce-overhead-backward]	4.1973ms	4.0277ms	248.2791 Ops/s	215.8129 Ops/s	$\textbf{\color{#35bf28}+15.04\%}$
test_redq_speed[False-None]	10.8629ms	10.3738ms	96.3963 Ops/s	91.6989 Ops/s	$\textbf{\color{#35bf28}+5.12\%}$
test_redq_speed[False-backward]	18.9576ms	17.9611ms	55.6759 Ops/s	54.6106 Ops/s	$\color{#35bf28}+1.95\%$
test_redq_speed[True-None]	5.7084ms	4.4570ms	224.3671 Ops/s	222.6495 Ops/s	$\color{#35bf28}+0.77\%$
test_redq_speed[True-backward]	10.0916ms	9.8600ms	101.4203 Ops/s	102.1035 Ops/s	$\color{#d91a1a}-0.67\%$
test_redq_speed[reduce-overhead-None]	4.7119ms	4.3779ms	228.4204 Ops/s	224.3057 Ops/s	$\color{#35bf28}+1.83\%$
test_redq_speed[reduce-overhead-backward]	10.9153ms	10.1330ms	98.6870 Ops/s	101.4423 Ops/s	$\color{#d91a1a}-2.72\%$
test_redq_deprec_speed[False-None]	11.5066ms	11.0520ms	90.4810 Ops/s	88.4869 Ops/s	$\color{#35bf28}+2.25\%$
test_redq_deprec_speed[False-backward]	16.5396ms	15.9487ms	62.7008 Ops/s	63.0564 Ops/s	$\color{#d91a1a}-0.56\%$
test_redq_deprec_speed[True-None]	3.9361ms	3.5576ms	281.0887 Ops/s	282.0457 Ops/s	$\color{#d91a1a}-0.34\%$
test_redq_deprec_speed[True-backward]	7.9424ms	7.6143ms	131.3312 Ops/s	117.1760 Ops/s	$\textbf{\color{#35bf28}+12.08\%}$
test_redq_deprec_speed[reduce-overhead-None]	3.8701ms	3.5470ms	281.9278 Ops/s	277.3518 Ops/s	$\color{#35bf28}+1.65\%$
test_redq_deprec_speed[reduce-overhead-backward]	7.8958ms	7.6426ms	130.8452 Ops/s	127.3099 Ops/s	$\color{#35bf28}+2.78\%$
test_td3_speed[False-None]	8.1385ms	7.9634ms	125.5753 Ops/s	116.7607 Ops/s	$\textbf{\color{#35bf28}+7.55\%}$
test_td3_speed[False-backward]	11.7822ms	11.0084ms	90.8396 Ops/s	89.6751 Ops/s	$\color{#35bf28}+1.30\%$
test_td3_speed[True-None]	1.8425ms	1.7921ms	558.0041 Ops/s	562.7840 Ops/s	$\color{#d91a1a}-0.85\%$
test_td3_speed[True-backward]	3.7043ms	3.5542ms	281.3554 Ops/s	274.8206 Ops/s	$\color{#35bf28}+2.38\%$
test_td3_speed[reduce-overhead-None]	2.1587ms	1.7811ms	561.4473 Ops/s	565.0199 Ops/s	$\color{#d91a1a}-0.63\%$
test_td3_speed[reduce-overhead-backward]	3.6965ms	3.5932ms	278.3072 Ops/s	250.2246 Ops/s	$\textbf{\color{#35bf28}+11.22\%}$
test_cql_speed[False-None]	29.6203ms	26.2863ms	38.0426 Ops/s	35.3638 Ops/s	$\textbf{\color{#35bf28}+7.58\%}$
test_cql_speed[False-backward]	39.4609ms	35.5468ms	28.1320 Ops/s	27.0365 Ops/s	$\color{#35bf28}+4.05\%$
test_cql_speed[True-None]	13.3921ms	12.4407ms	80.3815 Ops/s	82.3392 Ops/s	$\color{#d91a1a}-2.38\%$
test_cql_speed[True-backward]	18.9290ms	18.4706ms	54.1400 Ops/s	53.9758 Ops/s	$\color{#35bf28}+0.30\%$
test_cql_speed[reduce-overhead-None]	13.5759ms	12.4850ms	80.0963 Ops/s	79.9189 Ops/s	$\color{#35bf28}+0.22\%$
test_cql_speed[reduce-overhead-backward]	18.8198ms	18.3963ms	54.3587 Ops/s	55.4599 Ops/s	$\color{#d91a1a}-1.99\%$
test_a2c_speed[False-None]	5.9762ms	5.5142ms	181.3498 Ops/s	156.1716 Ops/s	$\textbf{\color{#35bf28}+16.12\%}$
test_a2c_speed[False-backward]	12.1761ms	11.8988ms	84.0422 Ops/s	76.9346 Ops/s	$\textbf{\color{#35bf28}+9.24\%}$
test_a2c_speed[True-None]	3.8904ms	3.7266ms	268.3406 Ops/s	270.4562 Ops/s	$\color{#d91a1a}-0.78\%$
test_a2c_speed[True-backward]	8.9064ms	8.6387ms	115.7588 Ops/s	113.2243 Ops/s	$\color{#35bf28}+2.24\%$
test_a2c_speed[reduce-overhead-None]	4.1464ms	3.7214ms	268.7190 Ops/s	267.7727 Ops/s	$\color{#35bf28}+0.35\%$
test_a2c_speed[reduce-overhead-backward]	9.2151ms	8.9819ms	111.3356 Ops/s	103.8752 Ops/s	$\textbf{\color{#35bf28}+7.18\%}$
test_ppo_speed[False-None]	6.4246ms	5.9607ms	167.7666 Ops/s	142.5805 Ops/s	$\textbf{\color{#35bf28}+17.66\%}$
test_ppo_speed[False-backward]	12.9874ms	12.6366ms	79.1355 Ops/s	73.0921 Ops/s	$\textbf{\color{#35bf28}+8.27\%}$
test_ppo_speed[True-None]	4.0099ms	3.6478ms	274.1365 Ops/s	267.5210 Ops/s	$\color{#35bf28}+2.47\%$
test_ppo_speed[True-backward]	8.9018ms	8.5686ms	116.7049 Ops/s	111.0824 Ops/s	$\textbf{\color{#35bf28}+5.06\%}$
test_ppo_speed[reduce-overhead-None]	4.2400ms	3.6796ms	271.7671 Ops/s	272.4782 Ops/s	$\color{#d91a1a}-0.26\%$
test_ppo_speed[reduce-overhead-backward]	8.9830ms	8.7402ms	114.4140 Ops/s	106.9443 Ops/s	$\textbf{\color{#35bf28}+6.98\%}$
test_reinforce_speed[False-None]	5.0122ms	4.6731ms	213.9920 Ops/s	175.4221 Ops/s	$\textbf{\color{#35bf28}+21.99\%}$
test_reinforce_speed[False-backward]	7.8025ms	7.4858ms	133.5866 Ops/s	117.2315 Ops/s	$\textbf{\color{#35bf28}+13.95\%}$
test_reinforce_speed[True-None]	3.3291ms	2.9319ms	341.0759 Ops/s	340.2605 Ops/s	$\color{#35bf28}+0.24\%$
test_reinforce_speed[True-backward]	8.0259ms	7.7800ms	128.5349 Ops/s	117.5381 Ops/s	$\textbf{\color{#35bf28}+9.36\%}$
test_reinforce_speed[reduce-overhead-None]	3.1722ms	2.9120ms	343.4058 Ops/s	338.8358 Ops/s	$\color{#35bf28}+1.35\%$
test_reinforce_speed[reduce-overhead-backward]	8.3807ms	8.0485ms	124.2462 Ops/s	114.1823 Ops/s	$\textbf{\color{#35bf28}+8.81\%}$
test_iql_speed[False-None]	21.0955ms	20.3159ms	49.2225 Ops/s	43.5873 Ops/s	$\textbf{\color{#35bf28}+12.93\%}$
test_iql_speed[False-backward]	37.4348ms	31.3153ms	31.9333 Ops/s	30.2972 Ops/s	$\textbf{\color{#35bf28}+5.40\%}$
test_iql_speed[True-None]	9.1169ms	8.5471ms	116.9983 Ops/s	116.5692 Ops/s	$\color{#35bf28}+0.37\%$
test_iql_speed[True-backward]	17.6938ms	16.9436ms	59.0194 Ops/s	59.4965 Ops/s	$\color{#d91a1a}-0.80\%$
test_iql_speed[reduce-overhead-None]	8.9879ms	8.6120ms	116.1172 Ops/s	112.9483 Ops/s	$\color{#35bf28}+2.81\%$
test_iql_speed[reduce-overhead-backward]	17.8576ms	17.4265ms	57.3838 Ops/s	57.0521 Ops/s	$\color{#35bf28}+0.58\%$
test_rb_sample[TensorDictReplayBuffer-ListStorage-RandomSampler-4000]	7.4163ms	6.0995ms	163.9480 Ops/s	166.3173 Ops/s	$\color{#d91a1a}-1.42\%$
test_rb_sample[TensorDictReplayBuffer-LazyMemmapStorage-RandomSampler-10000]	0.5661ms	0.2898ms	3.4505 KOps/s	3.2513 KOps/s	$\textbf{\color{#35bf28}+6.13\%}$
test_rb_sample[TensorDictReplayBuffer-LazyTensorStorage-RandomSampler-10000]	0.5414ms	0.2826ms	3.5390 KOps/s	3.5773 KOps/s	$\color{#d91a1a}-1.07\%$
test_rb_sample[TensorDictReplayBuffer-ListStorage-SamplerWithoutReplacement-4000]	6.1118ms	5.8084ms	172.1631 Ops/s	174.7541 Ops/s	$\color{#d91a1a}-1.48\%$
test_rb_sample[TensorDictReplayBuffer-LazyMemmapStorage-SamplerWithoutReplacement-10000]	2.2392ms	0.2743ms	3.6457 KOps/s	3.3228 KOps/s	$\textbf{\color{#35bf28}+9.72\%}$
test_rb_sample[TensorDictReplayBuffer-LazyTensorStorage-SamplerWithoutReplacement-10000]	0.6451ms	0.2909ms	3.4372 KOps/s	3.8418 KOps/s	$\textbf{\color{#d91a1a}-10.53\%}$
test_rb_sample[TensorDictReplayBuffer-LazyMemmapStorage-sampler6-10000]	1.7225ms	1.2504ms	799.7132 Ops/s	726.0487 Ops/s	$\textbf{\color{#35bf28}+10.15\%}$
test_rb_sample[TensorDictReplayBuffer-LazyTensorStorage-sampler7-10000]	1.4935ms	1.2175ms	821.3299 Ops/s	769.4340 Ops/s	$\textbf{\color{#35bf28}+6.74\%}$
test_rb_sample[TensorDictPrioritizedReplayBuffer-ListStorage-None-4000]	10.0166ms	6.1533ms	162.5153 Ops/s	170.7375 Ops/s	$\color{#d91a1a}-4.82\%$
test_rb_sample[TensorDictPrioritizedReplayBuffer-LazyMemmapStorage-None-10000]	2.0525ms	0.4401ms	2.2720 KOps/s	2.0881 KOps/s	$\textbf{\color{#35bf28}+8.81\%}$
test_rb_sample[TensorDictPrioritizedReplayBuffer-LazyTensorStorage-None-10000]	0.7568ms	0.4076ms	2.4537 KOps/s	2.3373 KOps/s	$\color{#35bf28}+4.98\%$
test_rb_iterate[TensorDictReplayBuffer-ListStorage-RandomSampler-4000]	6.0188ms	5.8473ms	171.0190 Ops/s	173.1312 Ops/s	$\color{#d91a1a}-1.22\%$
test_rb_iterate[TensorDictReplayBuffer-LazyMemmapStorage-RandomSampler-10000]	1.7275ms	0.2855ms	3.5021 KOps/s	809.1864 Ops/s	$\textbf{\color{#35bf28}+332.79\%}$
test_rb_iterate[TensorDictReplayBuffer-LazyTensorStorage-RandomSampler-10000]	0.6722ms	0.3022ms	3.3094 KOps/s	3.8543 KOps/s	$\textbf{\color{#d91a1a}-14.14\%}$
test_rb_iterate[TensorDictReplayBuffer-ListStorage-SamplerWithoutReplacement-4000]	6.3678ms	5.7807ms	172.9890 Ops/s	173.6965 Ops/s	$\color{#d91a1a}-0.41\%$
test_rb_iterate[TensorDictReplayBuffer-LazyMemmapStorage-SamplerWithoutReplacement-10000]	0.5371ms	0.2804ms	3.5666 KOps/s	3.5991 KOps/s	$\color{#d91a1a}-0.90\%$
test_rb_iterate[TensorDictReplayBuffer-LazyTensorStorage-SamplerWithoutReplacement-10000]	0.5670ms	0.2939ms	3.4030 KOps/s	3.9115 KOps/s	$\textbf{\color{#d91a1a}-13.00\%}$
test_rb_iterate[TensorDictPrioritizedReplayBuffer-ListStorage-None-4000]	6.1502ms	5.9854ms	167.0738 Ops/s	167.4019 Ops/s	$\color{#d91a1a}-0.20\%$
test_rb_iterate[TensorDictPrioritizedReplayBuffer-LazyMemmapStorage-None-10000]	0.7980ms	0.4288ms	2.3323 KOps/s	2.3295 KOps/s	$\color{#35bf28}+0.12\%$
test_rb_iterate[TensorDictPrioritizedReplayBuffer-LazyTensorStorage-None-10000]	0.7681ms	0.4065ms	2.4603 KOps/s	2.4456 KOps/s	$\color{#35bf28}+0.60\%$
test_rb_populate[TensorDictReplayBuffer-ListStorage-RandomSampler-400]	6.7005ms	5.1122ms	195.6118 Ops/s	193.8949 Ops/s	$\color{#35bf28}+0.89\%$
test_rb_populate[TensorDictReplayBuffer-LazyMemmapStorage-RandomSampler-400]	9.2703ms	2.3274ms	429.6649 Ops/s	430.4075 Ops/s	$\color{#d91a1a}-0.17\%$
test_rb_populate[TensorDictReplayBuffer-LazyTensorStorage-RandomSampler-400]	7.4537ms	1.1877ms	841.9556 Ops/s	864.6971 Ops/s	$\color{#d91a1a}-2.63\%$
test_rb_populate[TensorDictReplayBuffer-ListStorage-SamplerWithoutReplacement-400]	0.4994s	15.4267ms	64.8228 Ops/s	54.6060 Ops/s	$\textbf{\color{#35bf28}+18.71\%}$
test_rb_populate[TensorDictReplayBuffer-LazyMemmapStorage-SamplerWithoutReplacement-400]	11.6462ms	1.9220ms	520.2946 Ops/s	510.3709 Ops/s	$\color{#35bf28}+1.94\%$
test_rb_populate[TensorDictReplayBuffer-LazyTensorStorage-SamplerWithoutReplacement-400]	6.6248ms	1.1600ms	862.0729 Ops/s	852.6127 Ops/s	$\color{#35bf28}+1.11\%$
test_rb_populate[TensorDictPrioritizedReplayBuffer-ListStorage-None-400]	6.9442ms	5.2827ms	189.2965 Ops/s	184.4555 Ops/s	$\color{#35bf28}+2.62\%$
test_rb_populate[TensorDictPrioritizedReplayBuffer-LazyMemmapStorage-None-400]	11.0174ms	2.2498ms	444.4801 Ops/s	445.2596 Ops/s	$\color{#d91a1a}-0.18\%$
test_rb_populate[TensorDictPrioritizedReplayBuffer-LazyTensorStorage-None-400]	3.7038ms	1.2647ms	790.7145 Ops/s	814.3871 Ops/s	$\color{#d91a1a}-2.91\%$
test_rb_extend_sample[ReplayBuffer-LazyTensorStorage-RandomSampler-10000-10000-100-True]	36.2785ms	33.3008ms	30.0293 Ops/s	29.8671 Ops/s	$\color{#35bf28}+0.54\%$
test_rb_extend_sample[ReplayBuffer-LazyTensorStorage-RandomSampler-10000-10000-100-False]	19.0454ms	17.4155ms	57.4202 Ops/s	55.9386 Ops/s	$\color{#35bf28}+2.65\%$
test_rb_extend_sample[ReplayBuffer-LazyTensorStorage-RandomSampler-100000-10000-100-True]	36.4560ms	33.9351ms	29.4680 Ops/s	28.6281 Ops/s	$\color{#35bf28}+2.93\%$
test_rb_extend_sample[ReplayBuffer-LazyTensorStorage-RandomSampler-100000-10000-100-False]	19.2691ms	17.7577ms	56.3135 Ops/s	54.8970 Ops/s	$\color{#35bf28}+2.58\%$
test_rb_extend_sample[ReplayBuffer-LazyTensorStorage-RandomSampler-1000000-10000-100-True]	38.3382ms	36.2984ms	27.5494 Ops/s	27.3829 Ops/s	$\color{#35bf28}+0.61\%$
test_rb_extend_sample[ReplayBuffer-LazyTensorStorage-RandomSampler-1000000-10000-100-False]	20.6786ms	19.1209ms	52.2987 Ops/s	52.2719 Ops/s	$\color{#35bf28}+0.05\%$