Skip to content

v2.1.0

Latest

Choose a tag to compare

@ftosoni ftosoni released this 11 Oct 14:47
· 1 commit to main since this release

MediaWiki Code2Code Search v2.1.0

This is the version described in the SoftwareX article
(https://doi.org/10.1016/j.softx.2026.103127).

The index, embeddings and metadata database are unchanged from v2.0.0:
the pre-computed artefacts on Zenodo (https://doi.org/10.5281/zenodo.20586256)
and Hugging Face (https://huggingface.co/datasets/ftosoni/mediawiki-code2code-search)
remain valid for this release.

Evaluation

  • Replaced the single-judge evaluation with a blinded, pooled, multi-judge protocol:
    both systems' top-10 results are pooled, deduplicated by content hash, shuffled
    and anonymised before grading.
  • Three LLM judges from different model families; inter-judge agreement
    Krippendorff's α = 0.74.
  • On the 27-query benchmark, jury-mean nDCG@10 is 0.81 for Code2Code vs 0.63 for BM25.
  • New scripts: blind_pool.py, run_judge_api.py, assemble_judge_output.py,
    analyze_labels.py. Blinded prompts, per-judge labels and the analysis report are
    released under scripts/evaluation/judging/.
  • Removed the previous single-judge scripts and annotations.
  • README now includes an evaluation section with per-category and per-query results.

Localisation

  • The interface is now translated through translatewiki.net; translations are merged
    automatically into main.
  • Available languages grew from 17 to 30.
  • The language selector is no longer hard-coded: the new GET /locales endpoint lists
    the locales found in frontend/i18n, with autonyms taken from CLDR.
  • Fixed i18n issues reported in T426522; untranslated messages fall back to English.

Deployment and API

  • Continuous deployment: each push to main redeploys the Toolforge service.
  • New dependency: langcodes[data].
  • Added API tests for the new endpoint.

Frontend and documentation

  • Footer links to the Zenodo and Hugging Face datasets and to the translatewiki.net
    project page.
  • Added the architecture diagram (PDF, SVG and TikZ source) and updated branding notes.

Full changelog: v2.0.0...v2.1.0