Repository navigation
Home
A translation memory is a record of decisions already made, and retrieval is the discipline of finding them again.
tm-rag-copilot is a small, fully offline demo of retrieval-augmented translation that finds the closest segments in a synthetic translation memory, translates with a local model or a labeled mock, checks every output with deterministic rules, and measures itself on a gold set.
简体中文： 这是一个完全离线的检索增强翻译演示：在虚构的翻译记忆和术语库中按语义检索最相似的句段，用本地模型或确定性 mock 生成译文，以确定性规则做 QA，并在标准查询集上测量结果。
繁體中文： 這是一個完全離線的檢索增強翻譯示範：在虛構的翻譯記憶與術語庫中依語義檢索最相似的句段，以本地模型或確定性 mock 產生譯文，以確定性規則做 QA，並在標準查詢集上測量結果。
| What it is | A measured retrieval and translation pipeline over a translation memory (TMX) and term base (TBX) |
| Language | Python 3.12, with fastembed, Qdrant in local mode, and pydantic |
| Tests | 57 passed, run on 2026-11-06 on Windows with Python 3.12.10 |
| Status | Demo on synthetic data. This repository makes no claim of deployment or production use |
| License | Apache-2.0 |
Translation teams keep memories of approved sentences and lists of approved terms, because consistency across thousands of strings matters as much as any single translation. A language model on its own knows neither, so this project retrieves the relevant approved work, places it in the prompt, and then checks the output with rules that need no model. Every configuration carries a measured result, including the one that made retrieval worse.
- Retrieval: multilingual embeddings, Qdrant payload filters, and BM25 with reciprocal rank fusion written from scratch
- Evaluation: recall@k, MRR, and QA pass rate on a gold set, with results stored as JSON
- Localization formats and rules: TMX, TBX, placeholders, tags, and approved and forbidden terms
- LLM integration: versioned prompts, JSON extraction, one retry, and bad replies counted as failures
- Experimental method: one change at a time, with results recorded and kept or dropped
All numbers come from the committed results/*.json, on 41 gold queries and 133 synthetic units. The full page is at dboles99.github.io/tm-rag-copilot.
| Configuration | recall@1 | recall@10 | MRR | QA pass (provider) |
|---|---|---|---|---|
| Baseline | 0.5122 | 0.8049 | 0.6146 | 33 of 41, 80.5% (mock) |
| zh-CN filter, the best retrieval | 0.5854 | 0.8293 | 0.6755 | 34 of 41, 82.9% (mock) |
| Hybrid BM25 and RRF, kept as an option only | 0.5366 | 0.6585 | 0.5632 | 36 of 41, 87.8% (mock) |
| Baseline retrieval with local Qwen | 0.5122 | 0.8049 | 0.6146 | 40 of 41, 97.6% (Qwen) |
QA pass measures six mechanical checks and does not measure translation quality. The mock copies the nearest match and does not translate, and the data is invented for this repository. Evidence explains what the 97.6% can and cannot support, including the fact that it is the highest score these rules allow on this gold set.
- Getting Started: install it and run a search
- How It Works: the pipeline, with one query followed through three configurations
- Evidence: tests, every measured result, and known failures
- Design Decisions: what was tried, kept, and dropped
- Limits and Next Steps: what a real system would still need
Designed and authored by Daniel Boles; implemented with coding agents. Apache-2.0.