About mean_pooling on text sequence

[celestialxevermore]

Dear author, I hope you enjoy your new year.

I have a quetion about, the reason why you do not use the function, _mean_pooling_for_similarity_sequence.

Is there any special reason for that?

I also looked out your previous model, UniVL, but I cannot find any reason about that.

I hope you reply soon.

Thx.

Sincerly,

[ArrowLuo]

Hi @celestialxevermore, good question., but to be honest, I forget the last reason. (Maybe) When I tested the architecture, the results did not have a big gap, so I kept the same process as CLIP did finally. The _mean_pooling_for_similarity_sequence just filters the first `[CLS]' and finishes mean pooling on real tokens. Best~

[celestialxevermore]

Thx!!