Repository navigation
MediaWiki Code2Code Search v2.1.0
This is the version described in the SoftwareX article
(https://doi.org/10.1016/j.softx.2026.103127).
The index, embeddings and metadata database are unchanged from v2.0.0:
the pre-computed artefacts on Zenodo (https://doi.org/10.5281/zenodo.20586256)
and Hugging Face (https://huggingface.co/datasets/ftosoni/mediawiki-code2code-search)
remain valid for this release.
Evaluation
- Replaced the single-judge evaluation with a blinded, pooled, multi-judge protocol:
both systems' top-10 results are pooled, deduplicated by content hash, shuffled
and anonymised before grading. - Three LLM judges from different model families; inter-judge agreement
Krippendorff's α = 0.74. - On the 27-query benchmark, jury-mean nDCG@10 is 0.81 for Code2Code vs 0.63 for BM25.
- New scripts:
blind_pool.py,run_judge_api.py,assemble_judge_output.py,
analyze_labels.py. Blinded prompts, per-judge labels and the analysis report are
released underscripts/evaluation/judging/. - Removed the previous single-judge scripts and annotations.
- README now includes an evaluation section with per-category and per-query results.
Localisation
- The interface is now translated through translatewiki.net; translations are merged
automatically intomain. - Available languages grew from 17 to 30.
- The language selector is no longer hard-coded: the new
GET /localesendpoint lists
the locales found infrontend/i18n, with autonyms taken from CLDR. - Fixed i18n issues reported in T426522; untranslated messages fall back to English.
Deployment and API
- Continuous deployment: each push to
mainredeploys the Toolforge service. - New dependency:
langcodes[data]. - Added API tests for the new endpoint.
Frontend and documentation
- Footer links to the Zenodo and Hugging Face datasets and to the translatewiki.net
project page. - Added the architecture diagram (PDF, SVG and TikZ source) and updated branding notes.
Full changelog: v2.0.0...v2.1.0