CLS Token

[IoSonoMarco]

Hi all, thanks for the great work. I've been looking at the example codes and modules and I noticed you have an arg for choosing the class strategy as projection or cls_token or average. I guess it doesn't apply to iTransformer since it only works with projection. But I was wondering whether you had feedback about using a learnable class token as a final representation for tasks like timeseries classification. Have you tried it already?

[WenWeiTHU]

We tested itransformer on the classification task with tslib, but it seems alternative models can be a good choice.

We implement it by averaging the variate tokens. However, introducing learnable class tokens like ViT has not been trialed. Did you get some results?