Negative STS22 scores

[Muennighoff]

STS22 scores should not be negative https://competitions.codalab.org/competitions/33835#results

[NouamaneTazi]

Can you provide a sample where the score is negative please?

[Muennighoff]

!pip install -q mteb
from mteb import MTEB
from sentence_transformers import SentenceTransformer
# Define the sentence-transformers model name
model_name = "average_word_embeddings_komninos"
model = SentenceTransformer(model_name)
evaluation = MTEB(tasks=["STS22"])
evaluation.run(model, output_folder=f"results/{model_name}")