BUG：Wrong att-mask in decoder

[slczgwh]

A BUG when creating model:

Informer2020/models/model.py

Lines 50 to 53 in a87092b

	AttentionLayer(FullAttention(True, factor, attention_dropout=dropout, output_attention=False),
	d_model, n_heads),
	AttentionLayer(FullAttention(False, factor, attention_dropout=dropout, output_attention=False),
	d_model, n_heads),

The BUG lead the cross-att in decoder using NO casual mask, while self-att using casual mask. Fortunately there is no information leak in informer, but still totally different with what you wrote in the paper.

[cookieminions]

Please refer to Figure 1. In decoder, we use masked multi-head prob-sparse self-attention and multi-head cross attention, so we use casual mask in self-attention and no casual mask in cross-attention. Since we utilize generative inference to generate prediction results, the causal mask in decoder is actually not critical. But you can also use mask in cross-attention ,and choose to use prob attn or full attn freely.

[zhouhaoyi]

Suppose there is no more discussion and I will close this issue in 12h.