about test performance

[wtestererer]

Hi I tried the "polladiff" model with defult settings on METR_LA dataset and I got test performance with mae around 9 by "exp.test(setting)", but I found the performance listed in the paper is around 3, which is closer to the validation loss during training. So could you please tell where the results listed in the paper come from?

below is what it looks like by running "exp.test(setting)" with batch size of 16

thanks!!

Use GPU: cuda:0
adjinit shape: torch.Size([207, 207])

testing : polladiff_metr_sl12_pl12_dm64_nh8_nl3_df256_lsmae_Exp_0<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<
test 6831
test shape: (426, 16, 12, 207) (426, 16, 12, 207)
Model parameter counts: 292573
step:0, mae:9.049021904703098, mape:0.2984916958266438, rmse:13.764303424148503
step:1, mae:8.904563929555515, mape:0.2983197104215376, rmse:13.832018254023689
step:2, mae:9.601394630364727, mape:0.30338513052295757, rmse:13.827111555226491
step:3, mae:8.83912548933311, mape:0.3062654549609094, rmse:14.42612567849841
step:4, mae:9.752340579728696, mape:0.30295781441557657, rmse:13.71602397459143
step:5, mae:9.824919110191052, mape:0.3066034655247891, rmse:13.939541994921774
step:6, mae:9.316452934045559, mape:0.301449763240199, rmse:13.84500009665374
step:7, mae:9.3732364115618, mape:0.29976794056985306, rmse:13.688384951189027
step:8, mae:9.441006810903394, mape:0.3001081018550719, rmse:13.678050994038548
step:9, mae:9.119859469275136, mape:0.31108005306400777, rmse:14.599795200604573
step:10, mae:8.888910758368331, mape:0.30814929166684857, rmse:14.53876717217241
step:11, mae:9.868891483508238, mape:0.3071122009587492, rmse:13.954774759453528
total mae:9.331643625961554, mape:0.303640885252262, rmse:13.984158171293508
mae:9.331642226078097, mape:0.3036408451288611, rmse:13.987890062252562