Skip to content

1.39.0

Choose a tag to compare

View all tags
@KennethEnevoldsen KennethEnevoldsen released this 01 Oct 10:46
· 1417 commits to main since this release
1.39.0
cf26684

1.39.0 (2025-10-01)

Feature

  • feat: Officially include RTEB in the leaderboard (#3222)

  • feat - adjust Rteb's Benchmark

  • feat - add blank

  • fix menu names

  • Update mteb/leaderboard/benchmark_selector.py

Co-authored-by: Roman Solomatin <samoed.roman@gmail.com>

  • moving around tasks

  • fix: Update RTEB summary columns (#3226)

  • fix(models): ensure prompt_type is passed to format_instruction (#3216)

  • 1.38.58

Automatically generated by python-semantic-release

  • Adding Cohere's output_dimension and embedding_type parameter (#3204)

  • Adding Cohere's output_dimension and embedding_type parameter
    Cohere's embed-v4 binary and int8

  • Correcting due to comments

  • dataset: add swedish cpc patent classifications to mteb (#3072)

  • feat: add swedish cpc patent classifications to mteb

  • fix: formatting and init imports

  • fix: update mteb task according to feedback

  • fix: perform citation and code formatting

  • fix: add train and test split for both datasets

  • fix: AttributeError in ColPaliEngineWrapper similarity method (#3177)

  • fix: delete kwargs for similarity score in ColPaliEngineWrapper for method behavior

  • chore: fix colpali_models similarity handle device

  • Update tasks & benchmarks tables

  • 1.38.59

Automatically generated by python-semantic-release

  • fix: prevent EOS token truncation (#3218)

  • fix(models): prevent EOS token truncation for BMRetriever

  • refactor(models): refactor tokenizer setup in InstructSentenceTransformerWrapper

  • fix(models): correct eos token handling in BMRetrieverWrapper

  • 1.38.60

Automatically generated by python-semantic-release

  • Update giga embeddings (#3210)

  • update giga embeddings

  • update giga embeddings

  • 3b-september-2025

  • fixed

  • lint

  • Update mteb/models/ru_sentence_models.py

Co-authored-by: Roman Solomatin <samoed.roman@gmail.com>

  • change revision due to flash-attn dependency

  • change apply_instruction_to_passages

Co-authored-by: Kolodin Egor <eikolodin@sberbank.ru>
Co-authored-by: Roman Solomatin <36135455+Samoed@users.noreply.github.com>
Co-authored-by: Roman Solomatin <samoed.roman@gmail.com>
Co-authored-by: Неизвестный Пользователь722497 <dolegosmirnov@sberbank.ru>

  • fix: Refactor split create_tables into static Benchmark methods (#3126)

  • feat - Split create_tables into static Benchmark methods

  • feat - format

  • Update mteb/leaderboard/table.py

Co-authored-by: Kenneth Enevoldsen <kenevoldsen@pm.me>

  • feat - remove search query;take benchmark result as input;addressing the circular import,

  • feat - format

  • Update mteb/benchmarks/benchmark.py

Co-authored-by: Kenneth Enevoldsen <kenevoldsen@pm.me>

  • Update mteb/benchmarks/benchmark.py

Co-authored-by: Kenneth Enevoldsen <kenevoldsen@pm.me>

  • feat - use to_dataframe;clean table.py;move creat_table

  • feat - fix circular import

  • feat - clean-up

  • feat - format

Co-authored-by: Kenneth Enevoldsen <kenevoldsen@pm.me>

  • 1.38.61

Automatically generated by python-semantic-release

  • Extending the RTEB benchmark (#3223)

Adding another voyageai model

  • Update tasks & benchmarks tables

  • feat - filter_by_privacy

  • feat - add new fields for rteb part

  • feat - getattr

  • feat - adjust privacy filter logic

  • feat - enhance summary table column renaming and add 'is_public' field mapping

  • fix: remove unused 'is_public' attribute from TaskResult

Co-authored-by: Yongbin Choi <whybe.choi@gmail.com>
Co-authored-by: semantic-release <semantic-release>
Co-authored-by: fzoll <5575946+fzoll@users.noreply.github.com>
Co-authored-by: Atheer <atheer2104@protonmail.com>
Co-authored-by: Yong woo Song <ywsong.dev@kakao.com>
Co-authored-by: github-actions[bot] <github-actions[bot]@users.noreply.github.com>
Co-authored-by: Egor <31567312+ekolodin@users.noreply.github.com>
Co-authored-by: Kolodin Egor <eikolodin@sberbank.ru>
Co-authored-by: Roman Solomatin <36135455+Samoed@users.noreply.github.com>
Co-authored-by: Roman Solomatin <samoed.roman@gmail.com>
Co-authored-by: Неизвестный Пользователь722497 <dolegosmirnov@sberbank.ru>
Co-authored-by: Kenneth Enevoldsen <kenevoldsen@pm.me>
Co-authored-by: smile <smile@pinai.io>
Co-authored-by: ethan <smiletoye@gmail.com>

  • removed show_rteb args

  • avoid defining function where we can just use the metadata

  • minor fixes

  • minor fixes

  • fix: Correct logic for filtering public tasks in ModelResult class (#3230)

Co-authored-by: ethan <smiletoye@gmail.com>

Co-authored-by: q275343119 <275343119@qq.com>
Co-authored-by: Roman Solomatin <samoed.roman@gmail.com>
Co-authored-by: 笑尿伊人 <44760272+q275343119@users.noreply.github.com>
Co-authored-by: Yongbin Choi <whybe.choi@gmail.com>
Co-authored-by: fzoll <5575946+fzoll@users.noreply.github.com>
Co-authored-by: Atheer <atheer2104@protonmail.com>
Co-authored-by: Yong woo Song <ywsong.dev@kakao.com>
Co-authored-by: github-actions[bot] <github-actions[bot]@users.noreply.github.com>
Co-authored-by: Egor <31567312+ekolodin@users.noreply.github.com>
Co-authored-by: Kolodin Egor <eikolodin@sberbank.ru>
Co-authored-by: Roman Solomatin <36135455+Samoed@users.noreply.github.com>
Co-authored-by: Неизвестный Пользователь722497 <dolegosmirnov@sberbank.ru>
Co-authored-by: smile <smile@pinai.io>
Co-authored-by: ethan <smiletoye@gmail.com> (11f9c1d)

Unknown

  • Update tasks & benchmarks tables (867105f)

  • Update tasks & benchmarks tables (65f29e6)

  • dataset: Add Software Issue Localization Datasets (#3178)

  • add software issue localization datasets

  • add software issue localization datasets

  • update and add multilingual datasets

  • fix citation format issues

  • Update mteb/tasks/Reranking/eng/SWEbenchVerifiedReranking.py

  • fix linting issues

Co-authored-by: Roman Solomatin <samoed.roman@gmail.com> (e56e7c4)

  • model: Update Youtu embedding model (#3227)

  • add youtu models

  • add a blank line

  • fix the optional dependencies and lint the code

  • remove unused dependencies and reformat

  • revise prompt_type

  • update youtu_models

Co-authored-by: springxchen <springxchen@tencent.com> (0000ae2)

  • model: New qzmodel (#3211)

  • Update qzhou_models.py

  • Update qzhou_models.py

  • reformat script code

  • Update configuration

  • According to our new decision, the model name has been changed to "QZhou-Embedding-Zh".

  • Fix variable naming issues. (e299345)

  • Update tasks & benchmarks tables (7f5990a)

  • Extending the RTEB benchmark (#3223)

Adding another voyageai model (4f58684)

Assets 6
Loading