Choose proper moving average kernel for short input

[PeihanDou]

Hello! Thank you for your well-commented code! I'm currently using Autoformer to deal with some data which have very short input length, such as only 8 timestamps. I noticed that the default moving average kernel size in series decomposition part is 25, which maybe too long for the input in this case. I tried some smaller kernel such as 3, 5, 7. But the model turned out to be worse on validation dataset. Do you have any suggestions about adjusting hyper parameters for short input? Any suggestion would be appreciated. Thank you!

[wuhaixu2016]

Hi, thanks for your usage of this repo.
(1) input length
I think the input length should be re-considered. We have discussed the input length in Appendix C of the paper. Generally speaking, longer inputs will provide more information, which can benefit the forecasting. Also, the input length can be affected by the sampling rate. Thus, I suggest to re-check the data pattern for the determination of input length.

(2) Only use the decoder (If your prediction horizon is long)
if you only have a limitation on the input length, while the forecasting horizon is long, I think you can only adopt the Autoformer decoder. And use the input length as the 'label_len' in this repo.

(3) input is short and output is also short.
In this condition, you can remove the moving average, and only use the Auto-Correlation. Because the shorter time series will contain a simpler temporal pattern, maybe you don't need decomposition.

[PeihanDou]

Thank you for your respond! It is very insightful.

For (2), could you give some more clarification? If we only use the decoder, then how to handle the encoder's output? Or you are saying that use only decoder to generate all Q, K, V and let decoder be the whole model? Thank you!

[wuhaixu2016]

I mean the latter case: "only decoder to generate all Q, K, V and let decoder be the whole model?"
In your case, the encoder seems to be meaningless if it only captures the information of 8 time points. You can adopt the Autoformer decoder to aggregate past information and generate the future. Note that, in this case, the decoder does not have the cross information, thus, it only contains one Auto-Correlation block, which is more like an encoder.

[PeihanDou]

Thank you very much! That make sense!

[Med-Rokaimi]

condition

for (3), how to remove the moving average please?
I've tried to remove it from the settings but coming with error.