You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
I noticed that this implementation of Transformer here used residual attention, which does not appear in some of the other baselines mentioned in the paper. So I wonder if you have performed additional ablation studies to see the effect of residual attention for forecasting?
The text was updated successfully, but these errors were encountered:
Hi @jerrywn121 , thanks for your questions! We use residual attention because we want to be as close as possible to the original Transformer model in NLP, and only find the key factors that make it work well when applying it to time series datasets (patch; channel-independence). Also, residual attention was used in some previous papers like TST (https://arxiv.org/abs/2010.02803). We haven't studied the effect of it yet.
For RevIN, we do the additional ablation in table 11 in the paper. It improves the model marginally in general. Hope this helps.
I noticed that this implementation of Transformer here used residual attention, which does not appear in some of the other baselines mentioned in the paper. So I wonder if you have performed additional ablation studies to see the effect of residual attention for forecasting?
The text was updated successfully, but these errors were encountered: