Model outputs tuples

[yjiang18]

Hi, could you explain how you generate the tweet sentence embedding please? I check the shape of the output based on the example, features = bertweet(input_ids) seems to have embeddings of each token in feature[0] (e.g., [1,20,768]) and tweet sentence embedding in feature[1] (e.g., [1, 768])? If so, please could you let me know how you generate feature[1]? Is it based on [CLS] token or simply average the whole word token embeddings? Thanks!

[datquocnguyen]

As far as I understand it is based on the [CLS] token. However, I am not 100% sure.
You might ask the HuggingFace transformers team for the final confirmation.