2.11.8

2.11.8 (2026-03-24)

Fix

• fix: skip single-label cluster sets in legacy clustering evaluation (#4254)

fix: skip encode for single-label cluster sets in legacy clustering

For cluster sets with only one unique label (e.g. cluster set #29 in
ArxivClusteringP2P with only 'quant-ph'), skip the expensive encode and
clustering steps. The v_measure is always 1.0 for these sets, so we add
it directly to preserve backward-compatible scores while saving compute.

Fixes #1866 (47caa30)

• fix: resolve circular imports and add CI enforcement (#4277)

• fix: resolve circular imports and add CI enforcement

Move top-level import mteb statements that created circular dependency
chains through mteb/__init__.py into function bodies or replace with
direct submodule imports. This eliminates 38 circular import chains.

Files fixed:

• task_metadata.py: defer import mteb to method body
• cache.py: defer import mteb to function bodies
• _create_table.py: replace import mteb with direct import from
  mteb.models.get_model_meta
• deprecated_evaluator.py: defer import mteb to method body
• abs_encoder.py: defer import mteb to method bodies
• task_result.py: import TaskMetadata from submodule directly,
  defer get_task import to method body

Add scripts/check_circular_imports.py that detects runtime circular
imports via AST analysis and integrate it into both make lint and
make lint-check targets so CI prevents future regressions.

Closes #3937

• fix: use top-level direct imports where possible, inline only where circular

Move imports from mteb.get_tasks, mteb.models.get_model_meta, and
mteb.benchmarks.get_benchmark to top-level in files where it doesn't
cause circular dependencies (abs_encoder, cache, deprecated_evaluator).
Keep inline imports only in task_metadata.py and task_result.py where
circular dependency chains make top-level imports impossible.
Replace import mteb with direct submodule imports throughout.

• fix: remove CI script and fix circular import in abs_encoder

Remove scripts/check_circular_imports.py and its Makefile references
per reviewer consensus. Move get_task import in abs_encoder.py to
function-level to break the circular import chain:
mteb.abstasks → abstask → mteb.models → abs_encoder → get_tasks → mteb.abstasks

• fix: update mock path for get_model_metas in cache tests

The import was changed from mteb.get_model_metas to a direct import
in mteb.cache, so the mock needs to patch mteb.cache.get_model_metas.

• Update tests/test_result_cache_load_from_cache.py

Co-authored-by: Kenneth Enevoldsen <kenevoldsen@pm.me>

• Apply suggestion from @KennethEnevoldsen

• fix patch

• fix: patch get_model_metas where it's looked up, not where it's defined

unittest.mock.patch must target the namespace where the name is used.
Since cache.py does from mteb.models.get_model_meta import get_model_metas,
the correct patch target is mteb.cache.get_model_metas, not
mteb.models.get_model_meta.get_model_metas.

---

Co-authored-by: Kenneth Enevoldsen <kenevoldsen@pm.me>
Co-authored-by: Roman Solomatin <36135455+Samoed@users.noreply.github.com> (7795c9b)

• fix: resolve API models in performance-over-time plot (#4276)

• fix: resolve API models in performance-over-time plot

API models (voyage, openai, cohere, etc.) were being filtered out of the
performance-over-time plot because their reference URLs are blog posts or
docs pages, not HuggingFace URLs. The existing lookup only worked for
HuggingFace URLs where org/model could be extracted from the URL path.

Add fallback model meta resolution: first try HuggingFace URL extraction
(existing behavior), then match by (reference_url, display_name), then
by display_name alone. Also build lookup dicts once per plot instead of
per-row for better performance.

Fixes #4141

• fix: pass release date through summary table instead of parsing markdown

Instead of reverse-engineering model metadata from markdown cells in the
plot function, add Release Date as a column in all summary table variants.
This makes the performance-over-time plot work for all models (including
API models) without needing URL-based fallback resolution.

• fix: hide Release Date column from summary table display

Pass raw summary data through gr.State for plot functions while
dropping Release Date from the displayed gr.DataFrame table. (cca4a52)

• fix: Deprecate sentence_transformers_loader in favor of SentenceTransformerEncoderWrapper (#4279)

• deprecate: sentence_transformers_loader in favor of SentenceTransformerEncoderWrapper

sentence_transformers_loader was just a thin wrapper around
SentenceTransformerEncoderWrapper. This deprecates the function
(with a DeprecationWarning) and updates all internal usages to use
SentenceTransformerEncoderWrapper directly as the model loader.

The function remains exported for backward compatibility.

Closes #3739

• fix: use @deprecated decorator instead of warnings.warn (f9abce8)