[BugFix] StepCounter._reset should not use output_spec

[vmoens]

No description provided.

[pytorch-bot]

🔗 Helpful Links

🧪 See artifacts and rendered test results at hud.pytorch.org/pr/pytorch/rl/3626

• 📄 Preview Python docs built from this PR

Note: Links to docs will display an error until the docs builds have been completed.

❗ 1 Active SEVs

There are 1 currently active SEVs. If your PR is affected, please view them below:

• Rolling out OSDC (ARC) runners on pull workflow for PyTorch trunk commits

❌ 1 New Failure

As of commit d570eaa with merge base 09ef76d ():

NEW FAILURE - The following job has failed:

• Continuous Benchmark (PR) / GPU Pytest benchmark (gh)
  Workflow failed! Cannot read properties of undefined (reading 'ops')

This comment was automatically generated by Dr. CI and updates every 15 minutes.

[github-actions]

$\color{#D29922}\textsf{\Large⚠\kern{0.2cm}\normalsize Warning}$ Result of CPU Benchmark Tests

Total Benchmarks: 172. Improved: $\large\color{#35bf28}13$. Worsened: $\large\color{#d91a1a}16$.

Expand to view detailed results

Name	Max	Mean	Ops	Ops on Repo HEAD	Change
test_tensor_to_bytestream_speed[pickle]	79.6611μs	77.9287μs	12.8322 KOps/s	12.6975 KOps/s	$\color{#35bf28}+1.06\%$
test_tensor_to_bytestream_speed[torch.save]	0.1362ms	0.1359ms	7.3596 KOps/s	7.2696 KOps/s	$\color{#35bf28}+1.24\%$
test_tensor_to_bytestream_speed[untyped_storage]	0.1007s	0.1004s	9.9572 Ops/s	9.9785 Ops/s	$\color{#d91a1a}-0.21\%$
test_tensor_to_bytestream_speed[numpy]	2.4139μs	2.4068μs	415.4894 KOps/s	413.8252 KOps/s	$\color{#35bf28}+0.40\%$
test_tensor_to_bytestream_speed[safetensors]	35.4645μs	35.3338μs	28.3015 KOps/s	28.3381 KOps/s	$\color{#d91a1a}-0.13\%$
test_simple	0.5375s	0.5364s	1.8644 Ops/s	1.7907 Ops/s	$\color{#35bf28}+4.12\%$
test_transformed	1.0719s	1.0661s	0.9380 Ops/s	0.9180 Ops/s	$\color{#35bf28}+2.17\%$
test_serial	1.6584s	1.6558s	0.6039 Ops/s	0.5924 Ops/s	$\color{#35bf28}+1.95\%$
test_parallel	1.0165s	1.0094s	0.9907 Ops/s	0.9767 Ops/s	$\color{#35bf28}+1.43\%$
test_step_mdp_speed[True-True-True-True-True]	0.3111ms	43.3134μs	23.0875 KOps/s	24.7110 KOps/s	$\textbf{\color{#d91a1a}-6.57\%}$
test_step_mdp_speed[True-True-True-True-False]	62.3950μs	23.2444μs	43.0212 KOps/s	44.6251 KOps/s	$\color{#d91a1a}-3.59\%$
test_step_mdp_speed[True-True-True-False-True]	61.7340μs	23.8704μs	41.8929 KOps/s	42.2310 KOps/s	$\color{#d91a1a}-0.80\%$
test_step_mdp_speed[True-True-True-False-False]	40.0930μs	12.7519μs	78.4197 KOps/s	80.1722 KOps/s	$\color{#d91a1a}-2.19\%$
test_step_mdp_speed[True-True-False-True-True]	79.6360μs	45.3263μs	22.0622 KOps/s	22.8829 KOps/s	$\color{#d91a1a}-3.59\%$
test_step_mdp_speed[True-True-False-True-False]	60.8540μs	25.4193μs	39.3401 KOps/s	40.3351 KOps/s	$\color{#d91a1a}-2.47\%$
test_step_mdp_speed[True-True-False-False-True]	67.9460μs	26.9644μs	37.0859 KOps/s	38.4244 KOps/s	$\color{#d91a1a}-3.48\%$
test_step_mdp_speed[True-True-False-False-False]	57.7540μs	16.2835μs	61.4120 KOps/s	66.1186 KOps/s	$\textbf{\color{#d91a1a}-7.12\%}$
test_step_mdp_speed[True-False-True-True-True]	80.4260μs	47.5755μs	21.0192 KOps/s	21.9740 KOps/s	$\color{#d91a1a}-4.35\%$
test_step_mdp_speed[True-False-True-True-False]	0.1147ms	29.2087μs	34.2364 KOps/s	36.1354 KOps/s	$\textbf{\color{#d91a1a}-5.26\%}$
test_step_mdp_speed[True-False-True-False-True]	65.6350μs	26.9876μs	37.0540 KOps/s	38.7739 KOps/s	$\color{#d91a1a}-4.44\%$
test_step_mdp_speed[True-False-True-False-False]	47.2530μs	15.4962μs	64.5319 KOps/s	66.1813 KOps/s	$\color{#d91a1a}-2.49\%$
test_step_mdp_speed[True-False-False-True-True]	91.0670μs	49.7880μs	20.0851 KOps/s	20.8282 KOps/s	$\color{#d91a1a}-3.57\%$
test_step_mdp_speed[True-False-False-True-False]	60.5840μs	30.9141μs	32.3477 KOps/s	33.3869 KOps/s	$\color{#d91a1a}-3.11\%$
test_step_mdp_speed[True-False-False-False-True]	73.0450μs	29.6657μs	33.7090 KOps/s	35.6091 KOps/s	$\textbf{\color{#d91a1a}-5.34\%}$
test_step_mdp_speed[True-False-False-False-False]	60.7850μs	18.4732μs	54.1324 KOps/s	56.9332 KOps/s	$\color{#d91a1a}-4.92\%$
test_step_mdp_speed[False-True-True-True-True]	87.6460μs	48.6939μs	20.5365 KOps/s	21.5736 KOps/s	$\color{#d91a1a}-4.81\%$
test_step_mdp_speed[False-True-True-True-False]	66.3350μs	29.3259μs	34.0995 KOps/s	36.2971 KOps/s	$\textbf{\color{#d91a1a}-6.05\%}$
test_step_mdp_speed[False-True-True-False-True]	2.5063ms	31.8999μs	31.3481 KOps/s	33.3808 KOps/s	$\textbf{\color{#d91a1a}-6.09\%}$
test_step_mdp_speed[False-True-True-False-False]	61.4150μs	18.3351μs	54.5403 KOps/s	60.3791 KOps/s	$\textbf{\color{#d91a1a}-9.67\%}$
test_step_mdp_speed[False-True-False-True-True]	87.4360μs	50.4050μs	19.8393 KOps/s	20.9633 KOps/s	$\textbf{\color{#d91a1a}-5.36\%}$
test_step_mdp_speed[False-True-False-True-False]	97.9180μs	30.0465μs	33.2818 KOps/s	33.1507 KOps/s	$\color{#35bf28}+0.40\%$
test_step_mdp_speed[False-True-False-False-True]	0.4563ms	32.3047μs	30.9553 KOps/s	31.4116 KOps/s	$\color{#d91a1a}-1.45\%$
test_step_mdp_speed[False-True-False-False-False]	39.9030μs	19.4467μs	51.4225 KOps/s	52.0948 KOps/s	$\color{#d91a1a}-1.29\%$
test_step_mdp_speed[False-False-True-True-True]	0.4814ms	52.7241μs	18.9667 KOps/s	19.8884 KOps/s	$\color{#d91a1a}-4.63\%$
test_step_mdp_speed[False-False-True-True-False]	0.4657ms	32.8190μs	30.4701 KOps/s	30.4567 KOps/s	$\color{#35bf28}+0.04\%$
test_step_mdp_speed[False-False-True-False-True]	0.4568ms	32.8072μs	30.4811 KOps/s	31.1933 KOps/s	$\color{#d91a1a}-2.28\%$
test_step_mdp_speed[False-False-True-False-False]	56.2740μs	18.9330μs	52.8180 KOps/s	52.9770 KOps/s	$\color{#d91a1a}-0.30\%$
test_step_mdp_speed[False-False-False-True-True]	0.4807ms	54.1963μs	18.4514 KOps/s	18.9447 KOps/s	$\color{#d91a1a}-2.60\%$
test_step_mdp_speed[False-False-False-True-False]	0.4546ms	35.7390μs	27.9806 KOps/s	28.7941 KOps/s	$\color{#d91a1a}-2.83\%$
test_step_mdp_speed[False-False-False-False-True]	52.1740μs	33.7326μs	29.6449 KOps/s	29.7232 KOps/s	$\color{#d91a1a}-0.26\%$
test_step_mdp_speed[False-False-False-False-False]	0.4430ms	21.6940μs	46.0957 KOps/s	46.7428 KOps/s	$\color{#d91a1a}-1.38\%$
test_non_tensor_env_rollout_speed[1000-single-True]	0.7083s	0.7047s	1.4191 Ops/s	1.3693 Ops/s	$\color{#35bf28}+3.63\%$
test_non_tensor_env_rollout_speed[1000-single-False]	0.6964s	0.5957s	1.6787 Ops/s	1.6750 Ops/s	$\color{#35bf28}+0.22\%$
test_non_tensor_env_rollout_speed[1000-serial-no-buffers-True]	1.6917s	1.6087s	0.6216 Ops/s	0.6191 Ops/s	$\color{#35bf28}+0.40\%$
test_non_tensor_env_rollout_speed[1000-serial-no-buffers-False]	1.4722s	1.3877s	0.7206 Ops/s	0.7176 Ops/s	$\color{#35bf28}+0.41\%$
test_non_tensor_env_rollout_speed[1000-serial-buffers-True]	1.9395s	1.8557s	0.5389 Ops/s	0.5365 Ops/s	$\color{#35bf28}+0.45\%$
test_non_tensor_env_rollout_speed[1000-serial-buffers-False]	1.7202s	1.6398s	0.6098 Ops/s	0.6133 Ops/s	$\color{#d91a1a}-0.57\%$
test_non_tensor_env_rollout_speed[1000-parallel-no-buffers-True]	4.6567s	4.5772s	0.2185 Ops/s	0.2214 Ops/s	$\color{#d91a1a}-1.32\%$
test_non_tensor_env_rollout_speed[1000-parallel-no-buffers-False]	4.3374s	4.2735s	0.2340 Ops/s	0.2297 Ops/s	$\color{#35bf28}+1.86\%$
test_non_tensor_env_rollout_speed[1000-parallel-buffers-True]	1.9435s	1.8450s	0.5420 Ops/s	0.5461 Ops/s	$\color{#d91a1a}-0.75\%$
test_non_tensor_env_rollout_speed[1000-parallel-buffers-False]	1.6489s	1.5522s	0.6442 Ops/s	0.6335 Ops/s	$\color{#35bf28}+1.69\%$
test_values[generalized_advantage_estimate-True-True]	9.9433ms	9.7692ms	102.3628 Ops/s	101.6305 Ops/s	$\color{#35bf28}+0.72\%$
test_values[vec_generalized_advantage_estimate-True-True]	17.2944ms	11.4540ms	87.3054 Ops/s	61.5886 Ops/s	$\textbf{\color{#35bf28}+41.76\%}$
test_values[td0_return_estimate-False-False]	0.1931ms	0.1241ms	8.0575 KOps/s	7.9169 KOps/s	$\color{#35bf28}+1.78\%$
test_values[td1_return_estimate-False-False]	28.1315ms	27.3486ms	36.5650 Ops/s	37.1257 Ops/s	$\color{#d91a1a}-1.51\%$
test_values[vec_td1_return_estimate-False-False]	17.6326ms	12.2419ms	81.6866 Ops/s	59.6248 Ops/s	$\textbf{\color{#35bf28}+37.00\%}$
test_values[td_lambda_return_estimate-True-False]	42.0570ms	41.0596ms	24.3549 Ops/s	25.1020 Ops/s	$\color{#d91a1a}-2.98\%$
test_values[vec_td_lambda_return_estimate-True-False]	17.8687ms	12.4072ms	80.5982 Ops/s	57.9940 Ops/s	$\textbf{\color{#35bf28}+38.98\%}$
test_gae_speed[generalized_advantage_estimate-False-1-512]	8.7323ms	8.6187ms	116.0268 Ops/s	115.5019 Ops/s	$\color{#35bf28}+0.45\%$
test_gae_speed[vec_generalized_advantage_estimate-True-1-512]	1.8319ms	1.5441ms	647.6165 Ops/s	672.1757 Ops/s	$\color{#d91a1a}-3.65\%$
test_gae_speed[vec_generalized_advantage_estimate-False-1-512]	0.4648ms	0.4253ms	2.3515 KOps/s	2.4080 KOps/s	$\color{#d91a1a}-2.35\%$
test_gae_speed[vec_generalized_advantage_estimate-True-32-512]	34.5743ms	29.6872ms	33.6846 Ops/s	29.0234 Ops/s	$\textbf{\color{#35bf28}+16.06\%}$
test_gae_speed[vec_generalized_advantage_estimate-False-32-512]	1.8245ms	1.7224ms	580.5796 Ops/s	586.6390 Ops/s	$\color{#d91a1a}-1.03\%$
test_dqn_speed[False-None]	1.5029ms	1.3990ms	714.8164 Ops/s	710.3875 Ops/s	$\color{#35bf28}+0.62\%$
test_dqn_speed[False-backward]	2.0220ms	1.9115ms	523.1614 Ops/s	519.1840 Ops/s	$\color{#35bf28}+0.77\%$
test_dqn_speed[True-None]	1.1369ms	0.5746ms	1.7404 KOps/s	1.6068 KOps/s	$\textbf{\color{#35bf28}+8.32\%}$
test_dqn_speed[True-backward]	1.0987ms	1.0382ms	963.2064 Ops/s	881.8885 Ops/s	$\textbf{\color{#35bf28}+9.22\%}$
test_dqn_speed[reduce-overhead-None]	0.6637ms	0.5530ms	1.8084 KOps/s	1.7785 KOps/s	$\color{#35bf28}+1.68\%$
test_ddpg_speed[False-None]	3.1664ms	2.8305ms	353.2987 Ops/s	359.9064 Ops/s	$\color{#d91a1a}-1.84\%$
test_ddpg_speed[False-backward]	4.1427ms	4.0542ms	246.6603 Ops/s	249.3115 Ops/s	$\color{#d91a1a}-1.06\%$
test_ddpg_speed[True-None]	1.8659ms	1.4500ms	689.6474 Ops/s	676.4568 Ops/s	$\color{#35bf28}+1.95\%$
test_ddpg_speed[True-backward]	2.5562ms	2.4623ms	406.1253 Ops/s	405.3599 Ops/s	$\color{#35bf28}+0.19\%$
test_ddpg_speed[reduce-overhead-None]	1.5415ms	1.4398ms	694.5177 Ops/s	701.9187 Ops/s	$\color{#d91a1a}-1.05\%$
test_sac_speed[False-None]	8.7718ms	8.0197ms	124.6933 Ops/s	123.9731 Ops/s	$\color{#35bf28}+0.58\%$
test_sac_speed[False-backward]	12.3087ms	11.3647ms	87.9914 Ops/s	87.3215 Ops/s	$\color{#35bf28}+0.77\%$
test_sac_speed[True-None]	2.6141ms	2.2243ms	449.5792 Ops/s	453.2302 Ops/s	$\color{#d91a1a}-0.81\%$
test_sac_speed[True-backward]	4.2549ms	4.1317ms	242.0331 Ops/s	201.7516 Ops/s	$\textbf{\color{#35bf28}+19.97\%}$
test_sac_speed[reduce-overhead-None]	2.7158ms	2.2039ms	453.7333 Ops/s	451.1115 Ops/s	$\color{#35bf28}+0.58\%$
test_redq_speed[False-None]	15.6103ms	10.8479ms	92.1839 Ops/s	94.6234 Ops/s	$\color{#d91a1a}-2.58\%$
test_redq_speed[False-backward]	18.7150ms	17.9394ms	55.7432 Ops/s	55.7463 Ops/s	$-0.01\%$
test_redq_speed[True-None]	4.8549ms	4.5867ms	218.0233 Ops/s	215.9404 Ops/s	$\color{#35bf28}+0.96\%$
test_redq_speed[reduce-overhead-None]	4.8839ms	4.5978ms	217.4942 Ops/s	227.7856 Ops/s	$\color{#d91a1a}-4.52\%$
test_redq_deprec_speed[False-None]	11.7538ms	11.1850ms	89.4051 Ops/s	90.8911 Ops/s	$\color{#d91a1a}-1.63\%$
test_redq_deprec_speed[False-backward]	16.4956ms	16.1963ms	61.7426 Ops/s	63.8390 Ops/s	$\color{#d91a1a}-3.28\%$
test_redq_deprec_speed[True-None]	3.9665ms	3.6912ms	270.9176 Ops/s	278.8207 Ops/s	$\color{#d91a1a}-2.83\%$
test_redq_deprec_speed[True-backward]	7.8337ms	7.5741ms	132.0286 Ops/s	136.4587 Ops/s	$\color{#d91a1a}-3.25\%$
test_redq_deprec_speed[reduce-overhead-None]	3.8455ms	3.6548ms	273.6132 Ops/s	279.8990 Ops/s	$\color{#d91a1a}-2.25\%$
test_td3_speed[False-None]	8.3093ms	8.0669ms	123.9631 Ops/s	123.4372 Ops/s	$\color{#35bf28}+0.43\%$
test_td3_speed[False-backward]	11.4146ms	11.0385ms	90.5919 Ops/s	91.4974 Ops/s	$\color{#d91a1a}-0.99\%$
test_td3_speed[True-None]	1.9289ms	1.8635ms	536.6210 Ops/s	542.7217 Ops/s	$\color{#d91a1a}-1.12\%$
test_td3_speed[True-backward]	3.7483ms	3.6425ms	274.5394 Ops/s	244.6474 Ops/s	$\textbf{\color{#35bf28}+12.22\%}$
test_td3_speed[reduce-overhead-None]	1.8915ms	1.8281ms	547.0052 Ops/s	550.8892 Ops/s	$\color{#d91a1a}-0.71\%$
test_cql_speed[False-None]	27.5007ms	26.2575ms	38.0844 Ops/s	37.9963 Ops/s	$\color{#35bf28}+0.23\%$
test_cql_speed[False-backward]	36.2126ms	35.5376ms	28.1392 Ops/s	27.7331 Ops/s	$\color{#35bf28}+1.46\%$
test_cql_speed[True-None]	13.0077ms	12.6722ms	78.9128 Ops/s	78.2777 Ops/s	$\color{#35bf28}+0.81\%$
test_cql_speed[True-backward]	18.7125ms	18.2055ms	54.9285 Ops/s	55.8896 Ops/s	$\color{#d91a1a}-1.72\%$
test_cql_speed[reduce-overhead-None]	12.8025ms	12.5038ms	79.9756 Ops/s	78.2082 Ops/s	$\color{#35bf28}+2.26\%$
test_a2c_speed[False-None]	5.6585ms	5.4380ms	183.8924 Ops/s	187.7701 Ops/s	$\color{#d91a1a}-2.07\%$
test_a2c_speed[False-backward]	12.2858ms	11.8338ms	84.5036 Ops/s	85.3039 Ops/s	$\color{#d91a1a}-0.94\%$
test_a2c_speed[True-None]	4.2704ms	3.8143ms	262.1723 Ops/s	259.7192 Ops/s	$\color{#35bf28}+0.94\%$
test_a2c_speed[True-backward]	9.3004ms	8.8212ms	113.3630 Ops/s	105.6656 Ops/s	$\textbf{\color{#35bf28}+7.28\%}$
test_a2c_speed[reduce-overhead-None]	4.2167ms	3.7826ms	264.3666 Ops/s	257.9017 Ops/s	$\color{#35bf28}+2.51\%$
test_ppo_speed[False-None]	6.4156ms	5.9131ms	169.1173 Ops/s	168.6342 Ops/s	$\color{#35bf28}+0.29\%$
test_ppo_speed[False-backward]	12.9160ms	12.5031ms	79.9802 Ops/s	79.4686 Ops/s	$\color{#35bf28}+0.64\%$
test_ppo_speed[True-None]	4.1844ms	3.8076ms	262.6323 Ops/s	257.1017 Ops/s	$\color{#35bf28}+2.15\%$
test_ppo_speed[True-backward]	9.0041ms	8.7507ms	114.2765 Ops/s	114.1836 Ops/s	$\color{#35bf28}+0.08\%$
test_ppo_speed[reduce-overhead-None]	4.1989ms	3.7708ms	265.1952 Ops/s	265.5946 Ops/s	$\color{#d91a1a}-0.15\%$
test_reinforce_speed[False-None]	5.4055ms	4.5566ms	219.4639 Ops/s	219.5608 Ops/s	$\color{#d91a1a}-0.04\%$
test_reinforce_speed[False-backward]	7.8045ms	7.4707ms	133.8554 Ops/s	134.7283 Ops/s	$\color{#d91a1a}-0.65\%$
test_reinforce_speed[True-None]	3.2927ms	3.0951ms	323.0912 Ops/s	332.8296 Ops/s	$\color{#d91a1a}-2.93\%$
test_reinforce_speed[True-backward]	8.2672ms	8.0129ms	124.7981 Ops/s	109.6120 Ops/s	$\textbf{\color{#35bf28}+13.85\%}$
test_reinforce_speed[reduce-overhead-None]	3.1952ms	3.0041ms	332.8755 Ops/s	328.7425 Ops/s	$\color{#35bf28}+1.26\%$
test_iql_speed[False-None]	20.9808ms	19.9317ms	50.1714 Ops/s	49.7355 Ops/s	$\color{#35bf28}+0.88\%$
test_iql_speed[False-backward]	31.2054ms	30.5153ms	32.7705 Ops/s	32.8640 Ops/s	$\color{#d91a1a}-0.28\%$
test_iql_speed[True-None]	9.1459ms	8.5374ms	117.1316 Ops/s	115.7050 Ops/s	$\color{#35bf28}+1.23\%$
test_iql_speed[True-backward]	17.2755ms	16.8218ms	59.4467 Ops/s	57.0631 Ops/s	$\color{#35bf28}+4.18\%$
test_iql_speed[reduce-overhead-None]	9.0045ms	8.6399ms	115.7420 Ops/s	115.3793 Ops/s	$\color{#35bf28}+0.31\%$
test_rb_sample[TensorDictReplayBuffer-ListStorage-RandomSampler-4000]	6.0336ms	5.8758ms	170.1891 Ops/s	168.1534 Ops/s	$\color{#35bf28}+1.21\%$
test_rb_sample[TensorDictReplayBuffer-LazyMemmapStorage-RandomSampler-10000]	3.0064ms	0.3683ms	2.7153 KOps/s	3.2048 KOps/s	$\textbf{\color{#d91a1a}-15.27\%}$
test_rb_sample[TensorDictReplayBuffer-LazyTensorStorage-RandomSampler-10000]	0.5876ms	0.3173ms	3.1521 KOps/s	3.4270 KOps/s	$\textbf{\color{#d91a1a}-8.02\%}$
test_rb_sample[TensorDictReplayBuffer-ListStorage-SamplerWithoutReplacement-4000]	5.9330ms	5.6540ms	176.8659 Ops/s	175.9680 Ops/s	$\color{#35bf28}+0.51\%$
test_rb_sample[TensorDictReplayBuffer-LazyMemmapStorage-SamplerWithoutReplacement-10000]	0.5321ms	0.2861ms	3.4958 KOps/s	3.5221 KOps/s	$\color{#d91a1a}-0.75\%$
test_rb_sample[TensorDictReplayBuffer-LazyTensorStorage-SamplerWithoutReplacement-10000]	0.5229ms	0.2864ms	3.4918 KOps/s	3.7443 KOps/s	$\textbf{\color{#d91a1a}-6.74\%}$
test_rb_sample[TensorDictReplayBuffer-LazyMemmapStorage-sampler6-10000]	1.5364ms	1.2999ms	769.3166 Ops/s	785.7778 Ops/s	$\color{#d91a1a}-2.09\%$
test_rb_sample[TensorDictReplayBuffer-LazyTensorStorage-sampler7-10000]	1.4214ms	1.1843ms	844.3581 Ops/s	837.0857 Ops/s	$\color{#35bf28}+0.87\%$
test_rb_sample[TensorDictPrioritizedReplayBuffer-ListStorage-None-4000]	9.1606ms	5.8545ms	170.8085 Ops/s	172.0058 Ops/s	$\color{#d91a1a}-0.70\%$
test_rb_sample[TensorDictPrioritizedReplayBuffer-LazyMemmapStorage-None-10000]	1.7480ms	0.4717ms	2.1199 KOps/s	2.1204 KOps/s	$\color{#d91a1a}-0.02\%$
test_rb_sample[TensorDictPrioritizedReplayBuffer-LazyTensorStorage-None-10000]	0.7383ms	0.4579ms	2.1838 KOps/s	2.2142 KOps/s	$\color{#d91a1a}-1.37\%$
test_rb_iterate[TensorDictReplayBuffer-ListStorage-RandomSampler-4000]	5.8530ms	5.5944ms	178.7487 Ops/s	175.4915 Ops/s	$\color{#35bf28}+1.86\%$
test_rb_iterate[TensorDictReplayBuffer-LazyMemmapStorage-RandomSampler-10000]	0.8161ms	0.3405ms	2.9368 KOps/s	3.0340 KOps/s	$\color{#d91a1a}-3.20\%$
test_rb_iterate[TensorDictReplayBuffer-LazyTensorStorage-RandomSampler-10000]	0.5710ms	0.3241ms	3.0851 KOps/s	3.4450 KOps/s	$\textbf{\color{#d91a1a}-10.45\%}$
test_rb_iterate[TensorDictReplayBuffer-ListStorage-SamplerWithoutReplacement-4000]	5.8013ms	5.5421ms	180.4371 Ops/s	176.9574 Ops/s	$\color{#35bf28}+1.97\%$
test_rb_iterate[TensorDictReplayBuffer-LazyMemmapStorage-SamplerWithoutReplacement-10000]	0.7258ms	0.3344ms	2.9901 KOps/s	3.2043 KOps/s	$\textbf{\color{#d91a1a}-6.69\%}$
test_rb_iterate[TensorDictReplayBuffer-LazyTensorStorage-SamplerWithoutReplacement-10000]	0.5011ms	0.3250ms	3.0772 KOps/s	3.0000 KOps/s	$\color{#35bf28}+2.58\%$
test_rb_iterate[TensorDictPrioritizedReplayBuffer-ListStorage-None-4000]	5.8347ms	5.7224ms	174.7527 Ops/s	171.8886 Ops/s	$\color{#35bf28}+1.67\%$
test_rb_iterate[TensorDictPrioritizedReplayBuffer-LazyMemmapStorage-None-10000]	2.2616ms	0.4723ms	2.1171 KOps/s	2.1007 KOps/s	$\color{#35bf28}+0.78\%$
test_rb_iterate[TensorDictPrioritizedReplayBuffer-LazyTensorStorage-None-10000]	0.6582ms	0.4582ms	2.1827 KOps/s	2.2286 KOps/s	$\color{#d91a1a}-2.06\%$
test_rb_populate[TensorDictReplayBuffer-ListStorage-RandomSampler-400]	6.3260ms	4.9356ms	202.6084 Ops/s	49.6342 Ops/s	$\textbf{\color{#35bf28}+308.20\%}$
test_rb_populate[TensorDictReplayBuffer-LazyMemmapStorage-RandomSampler-400]	7.2850ms	2.1257ms	470.4247 Ops/s	536.7006 Ops/s	$\textbf{\color{#d91a1a}-12.35\%}$
test_rb_populate[TensorDictReplayBuffer-LazyTensorStorage-RandomSampler-400]	10.9984ms	1.3149ms	760.4875 Ops/s	912.0091 Ops/s	$\textbf{\color{#d91a1a}-16.61\%}$
test_rb_populate[TensorDictReplayBuffer-ListStorage-SamplerWithoutReplacement-400]	0.6286s	17.5243ms	57.0638 Ops/s	196.5243 Ops/s	$\textbf{\color{#d91a1a}-70.96\%}$
test_rb_populate[TensorDictReplayBuffer-LazyMemmapStorage-SamplerWithoutReplacement-400]	4.0732ms	1.7566ms	569.2934 Ops/s	565.3145 Ops/s	$\color{#35bf28}+0.70\%$
test_rb_populate[TensorDictReplayBuffer-LazyTensorStorage-SamplerWithoutReplacement-400]	3.5202ms	0.9338ms	1.0709 KOps/s	763.0883 Ops/s	$\textbf{\color{#35bf28}+40.34\%}$
test_rb_populate[TensorDictPrioritizedReplayBuffer-ListStorage-None-400]	8.8114ms	5.2044ms	192.1456 Ops/s	189.2351 Ops/s	$\color{#35bf28}+1.54\%$
test_rb_populate[TensorDictPrioritizedReplayBuffer-LazyMemmapStorage-None-400]	4.3618ms	1.9569ms	511.0247 Ops/s	496.5146 Ops/s	$\color{#35bf28}+2.92\%$
test_rb_populate[TensorDictPrioritizedReplayBuffer-LazyTensorStorage-None-400]	3.4129ms	1.0613ms	942.1983 Ops/s	790.3166 Ops/s	$\textbf{\color{#35bf28}+19.22\%}$
test_rb_extend_sample[ReplayBuffer-LazyTensorStorage-RandomSampler-10000-10000-100-True]	43.1913ms	38.9659ms	25.6635 Ops/s	25.3773 Ops/s	$\color{#35bf28}+1.13\%$
test_rb_extend_sample[ReplayBuffer-LazyTensorStorage-RandomSampler-10000-10000-100-False]	20.0263ms	18.3835ms	54.3966 Ops/s	55.4148 Ops/s	$\color{#d91a1a}-1.84\%$
test_rb_extend_sample[ReplayBuffer-LazyTensorStorage-RandomSampler-100000-10000-100-True]	44.3844ms	40.0004ms	24.9997 Ops/s	24.6659 Ops/s	$\color{#35bf28}+1.35\%$
test_rb_extend_sample[ReplayBuffer-LazyTensorStorage-RandomSampler-100000-10000-100-False]	19.8886ms	18.3785ms	54.4114 Ops/s	54.4181 Ops/s	$\color{#d91a1a}-0.01\%$
test_rb_extend_sample[ReplayBuffer-LazyTensorStorage-RandomSampler-1000000-10000-100-True]	44.8375ms	42.1401ms	23.7304 Ops/s	23.1977 Ops/s	$\color{#35bf28}+2.30\%$
test_rb_extend_sample[ReplayBuffer-LazyTensorStorage-RandomSampler-1000000-10000-100-False]	20.9254ms	19.9794ms	50.0517 Ops/s	50.1584 Ops/s	$\color{#d91a1a}-0.21\%$
test_storage_write_lazystack[50-img_shape0-small]	0.8207ms	0.2240ms	4.4640 KOps/s	4.3975 KOps/s	$\color{#35bf28}+1.51\%$
test_storage_write_lazystack[100-img_shape1-atari]	1.7546ms	1.4383ms	695.2726 Ops/s	707.0743 Ops/s	$\color{#d91a1a}-1.67\%$
test_storage_write_lazystack[100-img_shape2-large_img]	2.5259ms	2.3283ms	429.5046 Ops/s	431.7134 Ops/s	$\color{#d91a1a}-0.51\%$
test_storage_write_lazystack[200-img_shape3-large_batch]	3.1727ms	2.9532ms	338.6171 Ops/s	336.3527 Ops/s	$\color{#35bf28}+0.67\%$
test_storage_write_contiguous[50-img_shape0-small]	0.5582ms	0.1322ms	7.5646 KOps/s	7.4782 KOps/s	$\color{#35bf28}+1.16\%$
test_storage_write_contiguous[100-img_shape1-atari]	0.3345ms	0.1843ms	5.4271 KOps/s	5.3070 KOps/s	$\color{#35bf28}+2.26\%$
test_storage_write_contiguous[100-img_shape2-large_img]	1.9791ms	1.7586ms	568.6310 Ops/s	554.4033 Ops/s	$\color{#35bf28}+2.57\%$
test_storage_write_contiguous[200-img_shape3-large_batch]	1.5552ms	1.3554ms	737.8046 Ops/s	742.7441 Ops/s	$\color{#d91a1a}-0.67\%$
test_collector_stack_then_write[50-img_shape0-small]	1.1983ms	1.0875ms	919.5476 Ops/s	903.6490 Ops/s	$\color{#35bf28}+1.76\%$
test_collector_stack_then_write[100-img_shape1-atari]	3.6506ms	3.5204ms	284.0582 Ops/s	284.9419 Ops/s	$\color{#d91a1a}-0.31\%$
test_collector_stack_then_write[100-img_shape2-large_img]	10.0949ms	5.6576ms	176.7535 Ops/s	177.4873 Ops/s	$\color{#d91a1a}-0.41\%$
test_collector_stack_then_write[200-img_shape3-large_batch]	13.6360ms	6.9695ms	143.4817 Ops/s	143.8114 Ops/s	$\color{#d91a1a}-0.23\%$
test_collector_lazystack_then_write[50-img_shape0-small]	0.4347ms	0.2756ms	3.6284 KOps/s	3.5824 KOps/s	$\color{#35bf28}+1.28\%$
test_collector_lazystack_then_write[100-img_shape1-atari]	1.7248ms	1.5488ms	645.6513 Ops/s	655.1417 Ops/s	$\color{#d91a1a}-1.45\%$
test_collector_lazystack_then_write[100-img_shape2-large_img]	2.7016ms	2.4438ms	409.1952 Ops/s	401.7272 Ops/s	$\color{#35bf28}+1.86\%$
test_collector_lazystack_then_write[200-img_shape3-large_batch]	3.3733ms	3.1616ms	316.3001 Ops/s	314.3595 Ops/s	$\color{#35bf28}+0.62\%$
test_collector_without_rb[100-img_shape0-atari]	33.8943ms	32.6656ms	30.6133 Ops/s	30.6490 Ops/s	$\color{#d91a1a}-0.12\%$
test_collector_without_rb[200-img_shape1-large_batch]	65.2857ms	63.9023ms	15.6489 Ops/s	15.6936 Ops/s	$\color{#d91a1a}-0.29\%$
test_collector_with_rb[100-img_shape0-atari]	38.0745ms	37.0252ms	27.0087 Ops/s	27.1663 Ops/s	$\color{#d91a1a}-0.58\%$
test_collector_with_rb[200-img_shape1-large_batch]	73.8486ms	72.5658ms	13.7806 Ops/s	13.8639 Ops/s	$\color{#d91a1a}-0.60\%$