1.12.18

1.12.18 (2024-06-05)

Fix

• fix: Ensure result are consistently stored in the same way (#876)

• Ensure result are consistently stored in the same way

• (due to failing test): updated missing dataset references
• (to test with more than one model) Added e5 models base and large
• updated mteb.get_model to now include metadata in the model object
• ensure that model name is always included when saving (with a default when it is not available)
• use the ModelMeta for the model_meta.json

• format

• minor test fixes

• docs: Minor updated to repro. workflow docs

• fixed failing test

• format

• Apply suggestions from code review

Co-authored-by: Isaac Chung <chungisaac1217@gmail.com>

• docs: update PR template

• fix: Added benchmark object (#878)

• removed duplicate task

• Added benchmark object

• removed import for duplicate task

• fix dataset references

• added seb

• Added test for running benchmarks

• changed tasks to be an iterable

• format

• Apply suggestions from code review

Co-authored-by: Niklas Muennighoff <n.muennighoff@gmail.com>
Co-authored-by: Isaac Chung <chungisaac1217@gmail.com>

---

Co-authored-by: Isaac Chung <chungisaac1217@gmail.com>
Co-authored-by: Niklas Muennighoff <n.muennighoff@gmail.com> (fb843d0)

Unknown

• Update tasks table (dfbdfdc)