translation

Translation

Relevant source files

• src/repodocs/translate.py
• tests/test_fix_translate_cite.py

Overview

repodocs translate (implemented by cmd_translate) takes an already-generated wiki (a directory of *.md pages plus plan.json) and produces a per-language copy under <out>/<lang>/. It is a post-processing step over generate/html output, not a separate generation pipeline: it re-runs the LLM only to translate prose, then reuses the same citation-linting and index-writing machinery as the rest of repodocs.

Sources: src/repodocs/translate.py:L117-L171

Command flow

Sources: src/repodocs/translate.py:L117-L171

Validation and page selection

cmd_translate first validates lang against LANG_RE (^[A-Za-z0-9_-]+$), rejecting path-like values such as .. or ../x before any directory is created — this is what prevents --lang from being used to write outside out/. It then globs out/*.md (excluding index.md) and, if --pages/only is given, narrows to matching stems, dying if none match.

Sources: src/repodocs/translate.py:L117-L126, tests/test_fix_translate_cite.py:L48-L57

Two-phase translation

Phase 1 runs on the main thread and is presence-skip only: a page already present in dest is skipped with a [skip] ... (exists; --force to retranslate) message unless --force is passed. Translations are not tracked by source hash, so re-running without --force never re-translates existing output.

Phase 2 builds (name, prompt) items via translate_prompt and runs them through parallel_llm, a bounded worker pool; only the main thread writes files as results stream back. Timeouts (subprocess.TimeoutExpired), generic exceptions, and nonzero LLM exit codes are each counted toward failed and reported to stderr per page rather than aborting the whole run.

Sources: src/repodocs/translate.py:L129-L158

Translation prompt

translate_prompt(md, lang) looks up a human-readable language name from LANG_NAMES (imported from render) and instructs the model to preserve markdown structure, code/inline code, file paths, URLs, mermaid blocks, and every Sources: line verbatim, translating only prose, headings, and natural-language table cells.

Sources: src/repodocs/translate.py:L1-L27

Heading localization

After translation, localize_headings(md, lang) swaps the ## Relevant source files H2 for a localized heading looked up in SOURCES_HEADING (currently only "pt": "## Arquivos-fonte relevantes"). It walks lines tracking fenced-code state and only replaces the first non-fenced line that starts with ## Relevant source files, leaving occurrences inside ``` code fences untouched. Tests confirm both that a real H2 outside a fence is translated and that an occurrence solely inside a fence is left byte-for-byte unchanged.

Sources: src/repodocs/translate.py:L36-L54, tests/test_fix_translate_cite.py:L14-L33

plan.json translation

translate_plan_file sends only the slug/title/purpose triples (not files) to the LLM via translate_plan_prompt, which asks for a JSON array of {"slug","title","purpose"} with slug unchanged. The response is parsed with parse_pages into a map keyed by slug, then merged back into the original entries in their original order, preserving each entry's files list verbatim and falling back to the original title/purpose if a slug is missing from the translated map.

On any of FileNotFoundError, subprocess.TimeoutExpired, ValueError (including a non-list root or empty translated map), json.JSONDecodeError, or OSError, it prints a warning and copies src to dest untranslated, returning False so cmd_translate can count it as a failure.

Sources: src/repodocs/translate.py:L57-L95, tests/test_fix_translate_cite.py:L36-L45, tests/test_fix_translate_cite.py:L60-L80

Citation preservation

Preserving citations across translation is enforced in two layers:

1. Prompt-level: translate_prompt explicitly tells the model not to translate or alter Sources: lines.
2. Post-hoc verification: _citation_drift(out, dest, pages) re-extracts all full [path:La-Lb](href) citations from each source page and each translated page using FULL_CITATION_RE (from citations.py), and computes src_cites - dst_cites. Comparing the full link — not just the visible label — catches both outright removal and href tampering (e.g., a translated label pointing at an altered href). If any page has missing citations, cmd_translate calls die(...) with a message listing up to 5 affected slugs and a "+N more" suffix, so the whole translate run fails rather than silently shipping a page with lost provenance.

lint_citations also runs on the translated output before the drift check, warning (not failing) on citations whose line ranges are invalid.

Sources: src/repodocs/translate.py:L98-L114, src/repodocs/translate.py:L159-L171

Tests exercise both outcomes directly: a fake parallel_llm that returns a translated page missing the Sources: line causes cmd_translate to raise SystemExit(1), while a fake response that keeps the citation line lets cmd_translate return 0 and the citation survive in the written file.

Sources: tests/test_fix_translate_cite.py:L83-L121

Return value and index

After translation, plan translation, linting, and drift-checking, cmd_translate reconstructs the set of present translated page stems, resolves pages via _pages_for(dest / "plan.json", present), and calls write_index(repo, dest, pages) to emit an index for the translated tree. The function returns 1 if any page or the plan.json translation failed, 0 otherwise; a propagated plan-translation failure is covered by a dedicated test that stubs a failing run_llm and asserts rc == 1 while the untranslated fallback copy is still written to dest/plan.json.

Sources: src/repodocs/translate.py:L159-L171, tests/test_fix_translate_cite.py:L60-L80