Skip to content

2.12.24

Choose a tag to compare

View all tags
@KennethEnevoldsen KennethEnevoldsen released this 20 Apr 08:53
· 755 commits to main since this release
2.12.24
30bc62c

2.12.24 (2026-04-20)

Fix

  • fix: remove columns with none (#4446)

  • remove columns with none

  • Apply suggestions from code review

Co-authored-by: Kenneth Enevoldsen <kenevoldsen@pm.me>

Co-authored-by: Kenneth Enevoldsen <kennethcenevoldsen@gmail.com>
Co-authored-by: Kenneth Enevoldsen <kenevoldsen@pm.me> (63c0af6)

Unknown

  • [Mveb] vggsound classification (#4444)

[MVEB] Add VGGSound audio-visual classification tasks

Add VGGSoundVAClassification (video+audio, va2c) and VGGSoundVClassification (video-only, v2c) for the VGGSound audio-visual dataset (Chen et al., ICASSP 2020).
Dataset contains 9,888 test clips across 308 sound classes from YouTube videos. Audio is the primary signal in the original task; the v2c variant serves as a
video-only baseline. Uses 5-fold cross-validation since the released split only contains test. Follows the standard MVEB classification task structure. Addresses
part of #4130 (MVEB Overview - Classification).

Co-authored-by: Yashwanth Devavarapu <yashwanthdevavarapu@Yashwanths-MacBook-Pro.local> (bf113b4)

  • add mteb/Shot2Story20K dataset (#4449)

add mteb/Shot2Story20K_test dataset (b597f37)

  • Add YouCook2_val retrieval tasks (#4432)

  • Add YouCook2_val retrieval tasks (V2T, T2V, A2T, T2A)

Made-with: Cursor

  • Update mteb/tasks/retrieval/eng/youcook2_retrieval.py

Co-authored-by: Roman Solomatin <samoed.roman@gmail.com>

  • add stats

  • update

Co-authored-by: Roman Solomatin <samoed.roman@gmail.com> (f0e5ab6)

  • Add VATEX retrieval tasks (#4433)

  • Add VATEX_test_1k retrieval tasks (V2T, T2V, A2T, T2A)

Made-with: Cursor

  • Update mteb/tasks/retrieval/eng/vatex_retrieval.py

Co-authored-by: Roman Solomatin <samoed.roman@gmail.com>

  • Update mteb/tasks/retrieval/eng/vatex_retrieval.py

Co-authored-by: Roman Solomatin <samoed.roman@gmail.com>

  • update

Co-authored-by: Roman Solomatin <samoed.roman@gmail.com> (f93dc75)

  • model: add BidirLM/BidirLM-Omni-2.5B-Embedding (#4370)

  • feat: add BidirLM/BidirLM-Omni-2.5B-Embedding model implementation

  • fix: address reviewer comments on BidirLM-Omni-2.5B-Embedding:

  • Load model via SentenceTransformer with trust_remote_code=True
  • Remove the _get_instruction() description fallback. Add explicit
    prompts for 22 MIEB/MAEB tasks

  • refactor: update to sentence transformers 5.4 and rely on encode function to get embedding

  • Apply suggestions from code review

Co-authored-by: Roman Solomatin <samoed.roman@gmail.com>

  • Feat: instruct with prompt args chat template

  • Fix: rely on EncodeKwargs for encoder function

  • Feat: improve readability

  • Refactor: Change how modality are passed to encode

  • Fix: lint error

  • Refactor: args encode

  • Refactor: Import from Bidir

  • comments update

  • Simplify get instruction (ne need for _lookup_prompt stripped)

Co-authored-by: Roman Solomatin <samoed.roman@gmail.com> (9e5c592)

Assets 6
Loading