1.12.75

1.12.75 (2024-07-09)

Fix

• fix: standardize PairClassification results (#1063)

• make pairclassification as one dict

• lint

• add max

• return precision, recall

---

Co-authored-by: Roman Solomatin <36135455+Samoed@users.noreply.github.com> (e244777)

Unknown

• Update points table (4e29dce)

• Update points table (4a5c0f4)

1.12.74

1.12.74 (2024-07-09)

Fix

• fix: Fix Jina model loading (#1062)

match ST model card load script (4ce6989)

1.12.73

1.12.73 (2024-07-09)

Fix

• fix: Simplify (#1017)

• Simplify

• Fix (a28c722)

1.12.72

1.12.72 (2024-07-09)

Fix

• fix: Fixes for encode_conversation and better default for CLI device (#1048)

• fix: ensure that argument are passed correctly for fatihdial

• fix: reformat encode_corpus to always check for encode_conversation functionality. Additionally remove encode_corpus from DresModel

• Added better default for device in CLI (2f1dc38)

1.12.71

1.12.71 (2024-07-08)

Fix

• fix: reuse corpus utils for LLM2Vec (#1040)

• reuse function by modifying little

• make lint

---

Co-authored-by: ryan.agile <ryan.agile@kakaobrain.com> (0cb5807)

1.12.70

1.12.70 (2024-07-08)

Fix

• fix: Add STS22.v2 and LivedoorNewsClustering.v2 (#1055)

• Added STS22.v2

• Added LivedoorNewsClustering.v2

• Updated description of v2

• Update mteb/tasks/Clustering/jpn/LivedoorNewsClustering.py

Co-authored-by: Niklas Muennighoff <n.muennighoff@gmail.com>

---

Co-authored-by: Niklas Muennighoff <n.muennighoff@gmail.com> (01c551a)

• fix: ensure reranking kwargs are properly passed (#1058) (9f922ae)

Unknown

• Update tasks table (f41a963)

1.12.69

1.12.69 (2024-07-08)

Documentation

• docs: points (#1057)

• points

• Update 980.jsonl (30f12ca)

Fix

• fix: Update salesforce_models (#1053)

Update salesforce_models.py (061c3e3)

Unknown

• Update points table (38552d5)

• Fix query format (#1042)

• Fix query format

• format (8adcaa0)

1.12.68

1.12.68 (2024-07-05)

Fix

• fix: update init in the classfication task for the swalihiNewsClassfication model to be seen (#1044)

• swahili_news_classfication

• update_two

• Update Swahili addin source and refeence

• Update SwahiliNewsClassification.py

• Update SwahiliNewsClassification.py

• addition info about group

• remove unnecessary line

• Create init.py

• update init.py

This update init in the classification tasks is to enable it to be seen by the tasks calling.

from .swa.SwahiliNewsClassification import *

• Upate with new test and lint (0822cd6)

Unknown

• Update tasks table (265e861)

1.12.67

1.12.67 (2024-07-04)

Fix

• fix: Added Missing Models (#1014)

• Added utility function for downloading models results

• Added result tables

• Added missing models

• fix: Fixed subset warning for later versions of polars

• Update scripts/mmteb/running_model/check_results.py

Co-authored-by: Niklas Muennighoff <n.muennighoff@gmail.com>

• removed used code segment

• format

---

Co-authored-by: Niklas Muennighoff <n.muennighoff@gmail.com> (602f4c0)

1.12.66

1.12.66 (2024-07-03)

Fix

• fix: Added encode_kwargs as the input for encode arguments and added batch_size to CLI (#1030)

• ci: fix outdated CI for pandas

• fix: Added encode_kwargs as the input for encode arguments and added batch_size to CLI

This PR adds encode_kwargs as an argument to MTEB.run(..., encode_kwargs) which allows passing in arguments to model.encode calls. This has the benefit of removing/distinguishing between multiple model arguments present in the code.

Where default were already in place I have replaced kept them, future PRs might standardize or remove these.

I also add a test to ensure that the encode_kwargs in all cases are passed to the model.encode.

I additionally also add the batch_size argument to the CLI.

• add batch size in for backward compat.

• fixed misspecified defaults

• format

• Added in example of encode_kwargs

• Update mteb/abstasks/AbsTask.py

Co-authored-by: Isaac Chung <chungisaac1217@gmail.com>

• Update README.md

Co-authored-by: Isaac Chung <chungisaac1217@gmail.com>

---

Co-authored-by: Isaac Chung <chungisaac1217@gmail.com> (9032f02)