Skip to content

LTB v0.1.7 — certified-translator refs for all 16 LTB pairs

Choose a tag to compare

View all tags
@Lawrenzho-bit Lawrenzho-bit released this 20 May 16:46
· 14 commits to main since this release
v0.1.7
ae1b3bd
This commit was created on GitHub.com and signed with GitHub’s verified signature.
GPG key ID: B5690EEEBB952194
Verified
Learn about vigilant mode.

Major dataset milestone: every LTB language pair now has at least 10 certified-translator reference documents. Closes the en-ru gap from v0.1.6.4 (was N=0 after script-validation cleanup) and upgrades all 7 surviving extension pairs from ml-curated to certified-translator quality.

What changed

Released scripts/add_flores_extension_refs.py — a one-shot, idempotent enrichment script that matches each region's English text in FLORES eng_Latn.devtest and pulls the parallel translation from each v0.1.6 extension-pair FLORES file. 408 region-pair refs added across 10 FLORES-derived docs, 0 unmatched.

Per-pair coverage at v0.1.7

Pair v0.1.6.4 v0.1.7 Δ
en-ru 0 10 +10 (FLORES-only)
en-ko, en-vi, en-id, en-ur, en-uz, en-kk, en-zh-tw 3 each 13 each +10 each (10 FLORES + 3 rileykim)
Core 8 pairs 20–28 unchanged —

Reference provenance now

Version Doc range Provenance Grade
v0.1 / v0.1.3 001–010 Author-curated Native-speaker non-professional
v0.1.4 011–025 rileykim/multilingual-document ml-curated
v0.1.5 026–035 (core 8) FLORES-200 certified-translator
v0.1.7 026–035 (extension 8) FLORES-200 certified-translator
v0.1.6 036–059 rileykim/multilingual-document ml-curated

Methodology roadmap updates

  • Critique #1 (sample size) → ✅ Largely fixed at v0.1.7 (all pairs N≥10 certified)
  • Critique #8 (single-author-curated ref) → ⚠️ Partially addressed across v0.1.5 + v0.1.7

See docs/methodology-roadmap.md.

Out of scope for this release

Leaderboard scores remain at v0.1.6 numbers. Re-running NLLB / opus-mt / identity / qwen-vl against the new references is a follow-on scoring milestone.

Citation

@misc{ltbench2026,
  title  = {LayoutTranslateBench: A Benchmark for Document Translation with Layout Preservation},
  year   = {2026},
  url    = {https://github.com/Lawrenzho-bit/LayoutTranslateBench},
  note   = {Version 0.1.7}
}
Assets 2
Loading