Confusing cls_token handling in ViTTransfomer::forward

[syboomsy]

I am quite confused when reading the code of forwarding function.
At line 421, why remove the class token given self.with_cls_token is True?
If it is correct, then at line 412, why remove the class token, given self.with_cls_token is False?
For the same flag var with_cls_token , decoder logic contradicts with encoder logic, could anyone explain this?