[CI] LLM tests integration

[vmoens]

Stack from ghstack (oldest at bottom):

• [Feature] Documentation #3192
• [Feature] SAC Trainer #3191
• [Feature] PPO Trainer Updates #3190
• [Feature] Trainer Algorithms - Configuration System #3189
• [Feature] Trainer Infrastructure - Timing and Utilities #3188
• -> [CI] LLM tests integration #3216
• [Quality] Fix flaky test #3211

[pytorch-bot]

🔗 Helpful Links

🧪 See artifacts and rendered test results at hud.pytorch.org/pr/pytorch/rl/3216

• 📄 Preview Python docs built from this PR

Note: Links to docs will display an error until the docs builds have been completed.

This comment was automatically generated by Dr. CI and updates every 15 minutes.

[github-actions]

$\color{#D29922}\textsf{\Large⚠\kern{0.2cm}\normalsize Warning}$ Result of CPU Benchmark Tests

Total Benchmarks: 154. Improved: $\large\color{#35bf28}17$. Worsened: $\large\color{#d91a1a}11$.

Expand to view detailed results

Name	Max	Mean	Ops	Ops on Repo HEAD	Change
test_tensor_to_bytestream_speed[pickle]	83.3515μs	82.1121μs	12.1785 KOps/s	12.1449 KOps/s	$\color{#35bf28}+0.28\%$
test_tensor_to_bytestream_speed[torch.save]	0.1433ms	0.1422ms	7.0335 KOps/s	6.9836 KOps/s	$\color{#35bf28}+0.71\%$
test_tensor_to_bytestream_speed[untyped_storage]	0.1218s	0.1216s	8.2248 Ops/s	8.2295 Ops/s	$\color{#d91a1a}-0.06\%$
test_tensor_to_bytestream_speed[numpy]	2.8143μs	2.8091μs	355.9910 KOps/s	344.7018 KOps/s	$\color{#35bf28}+3.28\%$
test_tensor_to_bytestream_speed[safetensors]	45.1265μs	42.9519μs	23.2819 KOps/s	23.5659 KOps/s	$\color{#d91a1a}-1.21\%$
test_simple	0.5521s	0.5515s	1.8132 Ops/s	1.7296 Ops/s	$\color{#35bf28}+4.83\%$
test_transformed	1.1378s	1.1192s	0.8935 Ops/s	0.8785 Ops/s	$\color{#35bf28}+1.70\%$
test_serial	1.6951s	1.6828s	0.5942 Ops/s	0.5915 Ops/s	$\color{#35bf28}+0.46\%$
test_parallel	1.1739s	1.0985s	0.9104 Ops/s	0.9331 Ops/s	$\color{#d91a1a}-2.43\%$
test_step_mdp_speed[True-True-True-True-True]	0.2152ms	45.1847μs	22.1314 KOps/s	22.0735 KOps/s	$\color{#35bf28}+0.26\%$
test_step_mdp_speed[True-True-True-True-False]	55.2810μs	25.4209μs	39.3377 KOps/s	39.0507 KOps/s	$\color{#35bf28}+0.73\%$
test_step_mdp_speed[True-True-True-False-True]	54.2810μs	25.5265μs	39.1749 KOps/s	39.0572 KOps/s	$\color{#35bf28}+0.30\%$
test_step_mdp_speed[True-True-True-False-False]	39.6010μs	14.1605μs	70.6187 KOps/s	71.1570 KOps/s	$\color{#d91a1a}-0.76\%$
test_step_mdp_speed[True-True-False-True-True]	79.8410μs	48.9799μs	20.4166 KOps/s	20.5243 KOps/s	$\color{#d91a1a}-0.53\%$
test_step_mdp_speed[True-True-False-True-False]	55.5410μs	28.2653μs	35.3791 KOps/s	35.1173 KOps/s	$\color{#35bf28}+0.75\%$
test_step_mdp_speed[True-True-False-False-True]	56.8210μs	28.9305μs	34.5656 KOps/s	35.0463 KOps/s	$\color{#d91a1a}-1.37\%$
test_step_mdp_speed[True-True-False-False-False]	45.0310μs	17.2609μs	57.9345 KOps/s	59.1200 KOps/s	$\color{#d91a1a}-2.01\%$
test_step_mdp_speed[True-False-True-True-True]	87.8520μs	51.5164μs	19.4113 KOps/s	19.2961 KOps/s	$\color{#35bf28}+0.60\%$
test_step_mdp_speed[True-False-True-True-False]	63.3910μs	30.9036μs	32.3587 KOps/s	31.9701 KOps/s	$\color{#35bf28}+1.22\%$
test_step_mdp_speed[True-False-True-False-True]	61.7820μs	28.7570μs	34.7741 KOps/s	35.2533 KOps/s	$\color{#d91a1a}-1.36\%$
test_step_mdp_speed[True-False-True-False-False]	46.7010μs	16.9844μs	58.8774 KOps/s	59.1787 KOps/s	$\color{#d91a1a}-0.51\%$
test_step_mdp_speed[True-False-False-True-True]	96.2430μs	53.9634μs	18.5311 KOps/s	18.5383 KOps/s	$\color{#d91a1a}-0.04\%$
test_step_mdp_speed[True-False-False-True-False]	93.8320μs	33.5595μs	29.7978 KOps/s	29.3423 KOps/s	$\color{#35bf28}+1.55\%$
test_step_mdp_speed[True-False-False-False-True]	72.0120μs	31.0059μs	32.2520 KOps/s	32.4090 KOps/s	$\color{#d91a1a}-0.48\%$
test_step_mdp_speed[True-False-False-False-False]	41.1110μs	19.4688μs	51.3644 KOps/s	51.4029 KOps/s	$\color{#d91a1a}-0.07\%$
test_step_mdp_speed[False-True-True-True-True]	93.5620μs	50.9900μs	19.6117 KOps/s	19.6183 KOps/s	$\color{#d91a1a}-0.03\%$
test_step_mdp_speed[False-True-True-True-False]	71.8520μs	31.2876μs	31.9615 KOps/s	32.8253 KOps/s	$\color{#d91a1a}-2.63\%$
test_step_mdp_speed[False-True-True-False-True]	2.3410ms	32.5597μs	30.7128 KOps/s	30.8266 KOps/s	$\color{#d91a1a}-0.37\%$
test_step_mdp_speed[False-True-True-False-False]	89.4020μs	18.8396μs	53.0797 KOps/s	52.7217 KOps/s	$\color{#35bf28}+0.68\%$
test_step_mdp_speed[False-True-False-True-True]	90.6420μs	54.0499μs	18.5014 KOps/s	18.7646 KOps/s	$\color{#d91a1a}-1.40\%$
test_step_mdp_speed[False-True-False-True-False]	65.7510μs	33.8585μs	29.5347 KOps/s	29.3184 KOps/s	$\color{#35bf28}+0.74\%$
test_step_mdp_speed[False-True-False-False-True]	68.0810μs	34.8859μs	28.6649 KOps/s	28.6789 KOps/s	$\color{#d91a1a}-0.05\%$
test_step_mdp_speed[False-True-False-False-False]	53.7620μs	21.3545μs	46.8285 KOps/s	46.6579 KOps/s	$\color{#35bf28}+0.37\%$
test_step_mdp_speed[False-False-True-True-True]	93.7220μs	56.0118μs	17.8534 KOps/s	17.3486 KOps/s	$\color{#35bf28}+2.91\%$
test_step_mdp_speed[False-False-True-True-False]	71.1510μs	36.5788μs	27.3383 KOps/s	27.1550 KOps/s	$\color{#35bf28}+0.67\%$
test_step_mdp_speed[False-False-True-False-True]	69.9120μs	34.6722μs	28.8415 KOps/s	28.1791 KOps/s	$\color{#35bf28}+2.35\%$
test_step_mdp_speed[False-False-True-False-False]	53.2010μs	21.3749μs	46.7838 KOps/s	46.3567 KOps/s	$\color{#35bf28}+0.92\%$
test_step_mdp_speed[False-False-False-True-True]	92.7520μs	58.6515μs	17.0499 KOps/s	16.8663 KOps/s	$\color{#35bf28}+1.09\%$
test_step_mdp_speed[False-False-False-True-False]	84.6920μs	39.1905μs	25.5164 KOps/s	25.3463 KOps/s	$\color{#35bf28}+0.67\%$
test_step_mdp_speed[False-False-False-False-True]	67.5410μs	37.1866μs	26.8914 KOps/s	26.4411 KOps/s	$\color{#35bf28}+1.70\%$
test_step_mdp_speed[False-False-False-False-False]	52.5510μs	23.7356μs	42.1308 KOps/s	41.3323 KOps/s	$\color{#35bf28}+1.93\%$
test_values[generalized_advantage_estimate-True-True]	11.0669ms	10.2220ms	97.8281 Ops/s	98.1235 Ops/s	$\color{#d91a1a}-0.30\%$
test_values[vec_generalized_advantage_estimate-True-True]	15.1408ms	11.1370ms	89.7912 Ops/s	90.6179 Ops/s	$\color{#d91a1a}-0.91\%$
test_values[td0_return_estimate-False-False]	0.2322ms	0.1316ms	7.5978 KOps/s	7.4866 KOps/s	$\color{#35bf28}+1.48\%$
test_values[td1_return_estimate-False-False]	29.7836ms	28.4141ms	35.1938 Ops/s	36.2841 Ops/s	$\color{#d91a1a}-3.00\%$
test_values[vec_td1_return_estimate-False-False]	11.6053ms	11.1085ms	90.0208 Ops/s	89.3446 Ops/s	$\color{#35bf28}+0.76\%$
test_values[td_lambda_return_estimate-True-False]	44.2798ms	42.6692ms	23.4361 Ops/s	24.3948 Ops/s	$\color{#d91a1a}-3.93\%$
test_values[vec_td_lambda_return_estimate-True-False]	12.2655ms	11.1699ms	89.5266 Ops/s	89.6090 Ops/s	$\color{#d91a1a}-0.09\%$
test_gae_speed[generalized_advantage_estimate-False-1-512]	8.7783ms	8.7344ms	114.4896 Ops/s	114.2765 Ops/s	$\color{#35bf28}+0.19\%$
test_gae_speed[vec_generalized_advantage_estimate-True-1-512]	1.7536ms	1.5330ms	652.3090 Ops/s	642.9795 Ops/s	$\color{#35bf28}+1.45\%$
test_gae_speed[vec_generalized_advantage_estimate-False-1-512]	0.8261ms	0.4332ms	2.3081 KOps/s	2.4076 KOps/s	$\color{#d91a1a}-4.13\%$
test_gae_speed[vec_generalized_advantage_estimate-True-32-512]	29.3158ms	22.8910ms	43.6852 Ops/s	33.7217 Ops/s	$\textbf{\color{#35bf28}+29.55\%}$
test_gae_speed[vec_generalized_advantage_estimate-False-32-512]	2.1138ms	1.7273ms	578.9387 Ops/s	581.3077 Ops/s	$\color{#d91a1a}-0.41\%$
test_dqn_speed[False-None]	6.5795ms	1.4420ms	693.4637 Ops/s	701.8537 Ops/s	$\color{#d91a1a}-1.20\%$
test_dqn_speed[False-backward]	2.0714ms	1.9722ms	507.0375 Ops/s	518.2926 Ops/s	$\color{#d91a1a}-2.17\%$
test_dqn_speed[True-None]	0.6990ms	0.5170ms	1.9343 KOps/s	1.9382 KOps/s	$\color{#d91a1a}-0.20\%$
test_dqn_speed[True-backward]	0.9826ms	0.9530ms	1.0493 KOps/s	930.1028 Ops/s	$\textbf{\color{#35bf28}+12.82\%}$
test_dqn_speed[reduce-overhead-None]	0.9082ms	0.5215ms	1.9176 KOps/s	1.9474 KOps/s	$\color{#d91a1a}-1.53\%$
test_dqn_speed[reduce-overhead-backward]	0.9727ms	0.9381ms	1.0660 KOps/s	876.5477 Ops/s	$\textbf{\color{#35bf28}+21.61\%}$
test_ddpg_speed[False-None]	3.2581ms	2.8659ms	348.9366 Ops/s	335.7213 Ops/s	$\color{#35bf28}+3.94\%$
test_ddpg_speed[False-backward]	4.1714ms	4.0514ms	246.8262 Ops/s	243.8259 Ops/s	$\color{#35bf28}+1.23\%$
test_ddpg_speed[True-None]	1.7686ms	1.3661ms	731.9874 Ops/s	718.7678 Ops/s	$\color{#35bf28}+1.84\%$
test_ddpg_speed[True-backward]	2.7046ms	2.3282ms	429.5228 Ops/s	354.5025 Ops/s	$\textbf{\color{#35bf28}+21.16\%}$
test_ddpg_speed[reduce-overhead-None]	1.7617ms	1.3626ms	733.8820 Ops/s	729.7902 Ops/s	$\color{#35bf28}+0.56\%$
test_ddpg_speed[reduce-overhead-backward]	2.4631ms	2.3070ms	433.4566 Ops/s	347.9591 Ops/s	$\textbf{\color{#35bf28}+24.57\%}$
test_sac_speed[False-None]	8.3199ms	7.8818ms	126.8738 Ops/s	123.6180 Ops/s	$\color{#35bf28}+2.63\%$
test_sac_speed[False-backward]	11.5695ms	11.1272ms	89.8703 Ops/s	89.4932 Ops/s	$\color{#35bf28}+0.42\%$
test_sac_speed[True-None]	2.5087ms	2.0898ms	478.5241 Ops/s	457.3416 Ops/s	$\color{#35bf28}+4.63\%$
test_sac_speed[True-backward]	4.4146ms	3.9936ms	250.3984 Ops/s	246.9412 Ops/s	$\color{#35bf28}+1.40\%$
test_sac_speed[reduce-overhead-None]	2.2409ms	2.0888ms	478.7499 Ops/s	449.3359 Ops/s	$\textbf{\color{#35bf28}+6.55\%}$
test_sac_speed[reduce-overhead-backward]	4.4419ms	4.0062ms	249.6122 Ops/s	230.1602 Ops/s	$\textbf{\color{#35bf28}+8.45\%}$
test_redq_speed[False-None]	10.6550ms	10.1695ms	98.3336 Ops/s	98.1271 Ops/s	$\color{#35bf28}+0.21\%$
test_redq_speed[False-backward]	18.1486ms	17.4407ms	57.3373 Ops/s	57.2130 Ops/s	$\color{#35bf28}+0.22\%$
test_redq_speed[True-None]	4.5705ms	4.3126ms	231.8802 Ops/s	232.6002 Ops/s	$\color{#d91a1a}-0.31\%$
test_redq_speed[True-backward]	9.8154ms	9.5400ms	104.8221 Ops/s	100.7920 Ops/s	$\color{#35bf28}+4.00\%$
test_redq_speed[reduce-overhead-None]	4.4304ms	4.2765ms	233.8371 Ops/s	226.6851 Ops/s	$\color{#35bf28}+3.16\%$
test_redq_speed[reduce-overhead-backward]	10.1661ms	9.7675ms	102.3804 Ops/s	98.1937 Ops/s	$\color{#35bf28}+4.26\%$
test_redq_deprec_speed[False-None]	11.0532ms	10.7516ms	93.0094 Ops/s	90.7842 Ops/s	$\color{#35bf28}+2.45\%$
test_redq_deprec_speed[False-backward]	16.2043ms	15.6154ms	64.0394 Ops/s	62.8407 Ops/s	$\color{#35bf28}+1.91\%$
test_redq_deprec_speed[True-None]	3.7935ms	3.4440ms	290.3597 Ops/s	274.0395 Ops/s	$\textbf{\color{#35bf28}+5.96\%}$
test_redq_deprec_speed[True-backward]	7.5407ms	7.1917ms	139.0497 Ops/s	135.6744 Ops/s	$\color{#35bf28}+2.49\%$
test_redq_deprec_speed[reduce-overhead-None]	3.7812ms	3.4394ms	290.7481 Ops/s	282.7471 Ops/s	$\color{#35bf28}+2.83\%$
test_redq_deprec_speed[reduce-overhead-backward]	7.5138ms	7.2463ms	138.0006 Ops/s	114.6330 Ops/s	$\textbf{\color{#35bf28}+20.38\%}$
test_td3_speed[False-None]	8.1969ms	7.9373ms	125.9868 Ops/s	124.6586 Ops/s	$\color{#35bf28}+1.07\%$
test_td3_speed[False-backward]	11.3940ms	10.8238ms	92.3890 Ops/s	91.5943 Ops/s	$\color{#35bf28}+0.87\%$
test_td3_speed[True-None]	1.8236ms	1.7671ms	565.8842 Ops/s	554.0888 Ops/s	$\color{#35bf28}+2.13\%$
test_td3_speed[True-backward]	3.7654ms	3.5414ms	282.3726 Ops/s	274.4501 Ops/s	$\color{#35bf28}+2.89\%$
test_td3_speed[reduce-overhead-None]	1.7629ms	1.7302ms	577.9617 Ops/s	564.3090 Ops/s	$\color{#35bf28}+2.42\%$
test_td3_speed[reduce-overhead-backward]	3.9259ms	3.5707ms	280.0544 Ops/s	274.5392 Ops/s	$\color{#35bf28}+2.01\%$
test_cql_speed[False-None]	29.5414ms	25.5251ms	39.1771 Ops/s	38.3693 Ops/s	$\color{#35bf28}+2.11\%$
test_cql_speed[False-backward]	38.8990ms	35.3790ms	28.2654 Ops/s	28.4343 Ops/s	$\color{#d91a1a}-0.59\%$
test_cql_speed[True-None]	12.5006ms	11.9817ms	83.4609 Ops/s	80.6359 Ops/s	$\color{#35bf28}+3.50\%$
test_cql_speed[True-backward]	17.8104ms	17.4728ms	57.2318 Ops/s	55.8816 Ops/s	$\color{#35bf28}+2.42\%$
test_cql_speed[reduce-overhead-None]	12.8091ms	12.1056ms	82.6064 Ops/s	81.6303 Ops/s	$\color{#35bf28}+1.20\%$
test_cql_speed[reduce-overhead-backward]	17.8950ms	17.5660ms	56.9283 Ops/s	56.3506 Ops/s	$\color{#35bf28}+1.03\%$
test_a2c_speed[False-None]	5.7758ms	5.3797ms	185.8831 Ops/s	185.1784 Ops/s	$\color{#35bf28}+0.38\%$
test_a2c_speed[False-backward]	12.0269ms	11.7805ms	84.8863 Ops/s	84.1868 Ops/s	$\color{#35bf28}+0.83\%$
test_a2c_speed[True-None]	3.9929ms	3.6123ms	276.8349 Ops/s	272.1732 Ops/s	$\color{#35bf28}+1.71\%$
test_a2c_speed[True-backward]	8.7813ms	8.5462ms	117.0108 Ops/s	107.2891 Ops/s	$\textbf{\color{#35bf28}+9.06\%}$
test_a2c_speed[reduce-overhead-None]	4.0996ms	3.6772ms	271.9461 Ops/s	272.1013 Ops/s	$\color{#d91a1a}-0.06\%$
test_a2c_speed[reduce-overhead-backward]	9.2829ms	8.7394ms	114.4245 Ops/s	115.4241 Ops/s	$\color{#d91a1a}-0.87\%$
test_ppo_speed[False-None]	6.2314ms	5.8699ms	170.3593 Ops/s	169.1986 Ops/s	$\color{#35bf28}+0.69\%$
test_ppo_speed[False-backward]	12.9193ms	12.5608ms	79.6127 Ops/s	79.9532 Ops/s	$\color{#d91a1a}-0.43\%$
test_ppo_speed[True-None]	4.0013ms	3.6089ms	277.0917 Ops/s	275.0427 Ops/s	$\color{#35bf28}+0.74\%$
test_ppo_speed[True-backward]	8.5666ms	8.3122ms	120.3049 Ops/s	109.2328 Ops/s	$\textbf{\color{#35bf28}+10.14\%}$
test_ppo_speed[reduce-overhead-None]	3.7268ms	3.5869ms	278.7950 Ops/s	276.1709 Ops/s	$\color{#35bf28}+0.95\%$
test_ppo_speed[reduce-overhead-backward]	8.9005ms	8.6726ms	115.3052 Ops/s	109.6676 Ops/s	$\textbf{\color{#35bf28}+5.14\%}$
test_reinforce_speed[False-None]	5.0218ms	4.5391ms	220.3067 Ops/s	215.0657 Ops/s	$\color{#35bf28}+2.44\%$
test_reinforce_speed[False-backward]	7.5314ms	7.3317ms	136.3934 Ops/s	133.7449 Ops/s	$\color{#35bf28}+1.98\%$
test_reinforce_speed[True-None]	2.9640ms	2.8082ms	356.0984 Ops/s	349.7049 Ops/s	$\color{#35bf28}+1.83\%$
test_reinforce_speed[True-backward]	7.8616ms	7.6251ms	131.1466 Ops/s	130.9991 Ops/s	$\color{#35bf28}+0.11\%$
test_reinforce_speed[reduce-overhead-None]	3.1999ms	2.8281ms	353.5928 Ops/s	354.5456 Ops/s	$\color{#d91a1a}-0.27\%$
test_reinforce_speed[reduce-overhead-backward]	8.0159ms	7.8070ms	128.0900 Ops/s	123.4890 Ops/s	$\color{#35bf28}+3.73\%$
test_iql_speed[False-None]	25.3954ms	20.1807ms	49.5522 Ops/s	47.8322 Ops/s	$\color{#35bf28}+3.60\%$
test_iql_speed[False-backward]	36.1396ms	30.4535ms	32.8370 Ops/s	32.6900 Ops/s	$\color{#35bf28}+0.45\%$
test_iql_speed[True-None]	8.7562ms	8.3990ms	119.0613 Ops/s	116.0022 Ops/s	$\color{#35bf28}+2.64\%$
test_iql_speed[True-backward]	16.8986ms	16.4587ms	60.7583 Ops/s	60.7580 Ops/s	$+0.00\%$
test_iql_speed[reduce-overhead-None]	8.7543ms	8.4087ms	118.9245 Ops/s	113.2652 Ops/s	$\color{#35bf28}+5.00\%$
test_iql_speed[reduce-overhead-backward]	17.2561ms	16.8846ms	59.2254 Ops/s	57.5483 Ops/s	$\color{#35bf28}+2.91\%$
test_rb_sample[TensorDictReplayBuffer-ListStorage-RandomSampler-4000]	7.5625ms	6.0282ms	165.8881 Ops/s	166.3523 Ops/s	$\color{#d91a1a}-0.28\%$
test_rb_sample[TensorDictReplayBuffer-LazyMemmapStorage-RandomSampler-10000]	0.5585ms	0.2803ms	3.5676 KOps/s	3.5678 KOps/s	$-0.01\%$
test_rb_sample[TensorDictReplayBuffer-LazyTensorStorage-RandomSampler-10000]	0.6812ms	0.2588ms	3.8641 KOps/s	3.8372 KOps/s	$\color{#35bf28}+0.70\%$
test_rb_sample[TensorDictReplayBuffer-ListStorage-SamplerWithoutReplacement-4000]	6.0008ms	5.7535ms	173.8083 Ops/s	175.4509 Ops/s	$\color{#d91a1a}-0.94\%$
test_rb_sample[TensorDictReplayBuffer-LazyMemmapStorage-SamplerWithoutReplacement-10000]	0.5801s	0.7092ms	1.4100 KOps/s	3.6551 KOps/s	$\textbf{\color{#d91a1a}-61.42\%}$
test_rb_sample[TensorDictReplayBuffer-LazyTensorStorage-SamplerWithoutReplacement-10000]	0.5642ms	0.3104ms	3.2212 KOps/s	3.9566 KOps/s	$\textbf{\color{#d91a1a}-18.59\%}$
test_rb_sample[TensorDictReplayBuffer-LazyMemmapStorage-sampler6-10000]	1.6451ms	1.4169ms	705.7678 Ops/s	804.9533 Ops/s	$\textbf{\color{#d91a1a}-12.32\%}$
test_rb_sample[TensorDictReplayBuffer-LazyTensorStorage-sampler7-10000]	1.6311ms	1.3193ms	757.9565 Ops/s	859.2684 Ops/s	$\textbf{\color{#d91a1a}-11.79\%}$
test_rb_sample[TensorDictPrioritizedReplayBuffer-ListStorage-None-4000]	6.0723ms	5.9434ms	168.2546 Ops/s	170.6699 Ops/s	$\color{#d91a1a}-1.42\%$
test_rb_sample[TensorDictPrioritizedReplayBuffer-LazyMemmapStorage-None-10000]	0.8107ms	0.4454ms	2.2451 KOps/s	2.3637 KOps/s	$\textbf{\color{#d91a1a}-5.02\%}$
test_rb_sample[TensorDictPrioritizedReplayBuffer-LazyTensorStorage-None-10000]	0.7299ms	0.4365ms	2.2910 KOps/s	2.4837 KOps/s	$\textbf{\color{#d91a1a}-7.76\%}$
test_rb_iterate[TensorDictReplayBuffer-ListStorage-RandomSampler-4000]	5.8816ms	5.8018ms	172.3597 Ops/s	175.3358 Ops/s	$\color{#d91a1a}-1.70\%$
test_rb_iterate[TensorDictReplayBuffer-LazyMemmapStorage-RandomSampler-10000]	2.0141ms	0.2840ms	3.5208 KOps/s	817.4006 Ops/s	$\textbf{\color{#35bf28}+330.73\%}$
test_rb_iterate[TensorDictReplayBuffer-LazyTensorStorage-RandomSampler-10000]	0.4688ms	0.2910ms	3.4361 KOps/s	3.9068 KOps/s	$\textbf{\color{#d91a1a}-12.05\%}$
test_rb_iterate[TensorDictReplayBuffer-ListStorage-SamplerWithoutReplacement-4000]	5.9017ms	5.7165ms	174.9320 Ops/s	173.7219 Ops/s	$\color{#35bf28}+0.70\%$
test_rb_iterate[TensorDictReplayBuffer-LazyMemmapStorage-SamplerWithoutReplacement-10000]	0.9112ms	0.2724ms	3.6715 KOps/s	3.6481 KOps/s	$\color{#35bf28}+0.64\%$
test_rb_iterate[TensorDictReplayBuffer-LazyTensorStorage-SamplerWithoutReplacement-10000]	0.4920ms	0.2641ms	3.7868 KOps/s	3.9205 KOps/s	$\color{#d91a1a}-3.41\%$
test_rb_iterate[TensorDictPrioritizedReplayBuffer-ListStorage-None-4000]	9.9555ms	6.0964ms	164.0301 Ops/s	167.8244 Ops/s	$\color{#d91a1a}-2.26\%$
test_rb_iterate[TensorDictPrioritizedReplayBuffer-LazyMemmapStorage-None-10000]	1.7575ms	0.4294ms	2.3286 KOps/s	2.3424 KOps/s	$\color{#d91a1a}-0.59\%$
test_rb_iterate[TensorDictPrioritizedReplayBuffer-LazyTensorStorage-None-10000]	0.6992ms	0.4513ms	2.2156 KOps/s	2.4591 KOps/s	$\textbf{\color{#d91a1a}-9.90\%}$
test_rb_populate[TensorDictReplayBuffer-ListStorage-RandomSampler-400]	0.4927s	14.8160ms	67.4947 Ops/s	194.3828 Ops/s	$\textbf{\color{#d91a1a}-65.28\%}$
test_rb_populate[TensorDictReplayBuffer-LazyMemmapStorage-RandomSampler-400]	3.8177ms	1.7029ms	587.2280 Ops/s	410.5582 Ops/s	$\textbf{\color{#35bf28}+43.03\%}$
test_rb_populate[TensorDictReplayBuffer-LazyTensorStorage-RandomSampler-400]	2.0154ms	0.9994ms	1.0006 KOps/s	937.1570 Ops/s	$\textbf{\color{#35bf28}+6.77\%}$
test_rb_populate[TensorDictReplayBuffer-ListStorage-SamplerWithoutReplacement-400]	7.7904ms	5.0537ms	197.8754 Ops/s	55.5335 Ops/s	$\textbf{\color{#35bf28}+256.32\%}$
test_rb_populate[TensorDictReplayBuffer-LazyMemmapStorage-SamplerWithoutReplacement-400]	7.7034ms	2.0132ms	496.7158 Ops/s	593.4979 Ops/s	$\textbf{\color{#d91a1a}-16.31\%}$
test_rb_populate[TensorDictReplayBuffer-LazyTensorStorage-SamplerWithoutReplacement-400]	7.9992ms	1.2033ms	831.0333 Ops/s	814.7136 Ops/s	$\color{#35bf28}+2.00\%$
test_rb_populate[TensorDictPrioritizedReplayBuffer-ListStorage-None-400]	0.4683s	14.5328ms	68.8099 Ops/s	187.2255 Ops/s	$\textbf{\color{#d91a1a}-63.25\%}$
test_rb_populate[TensorDictPrioritizedReplayBuffer-LazyMemmapStorage-None-400]	10.3913ms	2.0938ms	477.6053 Ops/s	484.5926 Ops/s	$\color{#d91a1a}-1.44\%$
test_rb_populate[TensorDictPrioritizedReplayBuffer-LazyTensorStorage-None-400]	1.1674ms	0.9789ms	1.0215 KOps/s	751.8691 Ops/s	$\textbf{\color{#35bf28}+35.86\%}$
test_rb_extend_sample[ReplayBuffer-LazyTensorStorage-RandomSampler-10000-10000-100-True]	37.4765ms	32.9664ms	30.3339 Ops/s	29.5803 Ops/s	$\color{#35bf28}+2.55\%$
test_rb_extend_sample[ReplayBuffer-LazyTensorStorage-RandomSampler-10000-10000-100-False]	19.1332ms	17.5668ms	56.9257 Ops/s	56.8484 Ops/s	$\color{#35bf28}+0.14\%$
test_rb_extend_sample[ReplayBuffer-LazyTensorStorage-RandomSampler-100000-10000-100-True]	36.6901ms	33.7176ms	29.6581 Ops/s	28.9581 Ops/s	$\color{#35bf28}+2.42\%$
test_rb_extend_sample[ReplayBuffer-LazyTensorStorage-RandomSampler-100000-10000-100-False]	19.0957ms	17.6291ms	56.7243 Ops/s	55.6070 Ops/s	$\color{#35bf28}+2.01\%$
test_rb_extend_sample[ReplayBuffer-LazyTensorStorage-RandomSampler-1000000-10000-100-True]	37.9927ms	35.5118ms	28.1597 Ops/s	27.1326 Ops/s	$\color{#35bf28}+3.79\%$
test_rb_extend_sample[ReplayBuffer-LazyTensorStorage-RandomSampler-1000000-10000-100-False]	21.0167ms	19.0803ms	52.4102 Ops/s	51.0876 Ops/s	$\color{#35bf28}+2.59\%$