architecture
Now I have enough to write the page.
- src/repodocs/cli.py
- src/repodocs/_util.py
- src/repodocs/scan.py
- src/repodocs/plan.py
- src/repodocs/generate.py
- src/repodocs/render.py
- src/repodocs/publish.py
- src/repodocs/translate.py
- src/repodocs/citations.py
- src/repodocs/backend.py
- src/repodocs/diagrams.py
- src/repodocs/gitlinks.py
repodocs is organized as a linear pipeline of modules, each corresponding to one repodocs subcommand: scan → plan → generate → (translate) → html (render) → (render-diagrams) → publish / publish-wiki. cli.py is the only module that wires argument parsing to these stages; every other module is a pure-ish library callable directly by tests or by cli.py. The module docstring in cli.py states the pipeline order and the repodocs all shortcut that chains scan, plan, generate, and html in one call.
Sources: src/repodocs/cli.py:L1-L44
Sources: src/repodocs/cli.py:L54-L63
scan.scan() walks the repository once with os.walk, skipping .git, node_modules, and similar directories (SKIP_DIRS in _util.py), the repo's own repo-docs/graphify-out output trees, and any configured --out directory at any depth, so a rerun never ingests its own generated Markdown/JSON. It also drops symlinks that escape the repo root. The function returns a dict of src_files, per-file line_counts, top_dirs (first path component grouping), and presence flags for README/CONTRIBUTING/CHANGELOG/SECURITY/manifests/CI/tests. scan_inventory() wraps scan() and adds readme_headings (extracted by readme_headings(), which skips fenced code blocks) for the planner-facing JSON form used by repodocs scan --json.
Sources: src/repodocs/scan.py:L1-L99, src/repodocs/_util.py:L11-L28
plan.py turns the scan inventory into a page list, either via an LLM call (llm_plan, invoked through backend.run_llm) or via the deterministic fallback plan_pages(). plan_pages() always emits an overview page (README + manifests + top files by line count), conditionally installation (when a README or manifest exists) and architecture (when there are ≥2 source files, built from top_candidates() — the highest-line-count files up to CANDIDATES_PER_PAGE), then one component-<slug> page per top-level directory with ≥2 files (or per large standalone module when there are no multi-file directories), capped at MAX_COMPONENTS, followed by conditional development and changelog pages. Slug collisions are resolved by alloc_slug(), which suffixes -2, -3, etc. graph_digest() optionally reads graphify-out/graph.json (a NetworkX node-link graph) to compute in-degree "god objects" and most-imported files, sharpening the LLM planner prompt when a graphify graph is present; it returns "" silently when the file is absent or malformed, since graphify is optional.
Sources: src/repodocs/plan.py:L1-L120
generate.cmd_generate() loads plan.json (via plan.load_plan, auto-running plan if missing), filters out pages with unsafe slugs (_safe_page() rejects any slug that doesn't match SLUG_RE or whose destination path would resolve outside out, since a hand-edited plan.json is treated as untrusted input), and then decides per page whether to regenerate.
decide_page()/generate_decision() implement the incremental rebuild: each page's files are SHA-256 hashed (compute_file_hash(), memoized in a shared hcache across pages so a file shared by multiple pages is hashed once per run) and compared against <out>/.hashes.json written by the previous run. A page regenerates when its .md is missing, --force is passed, or any of its files' hashes changed (including files added/removed from the plan entry); otherwise it is skipped.
Pages needing regeneration are dispatched through backend.parallel_llm() (bounded by REPODOCS_JOBS, clamped 1–16) using the prompt built by page_prompt() — the same prompt format documented in this project's own AGENTS.md contract, including the graphify-out/graph.json hint when that file exists. Each successful response is passed through citations.repair_citations() before being written to <out>/<slug>.md; .hashes.json is saved after every completion so a crash mid-run can resume. After all pages are processed, citations.lint_citations() runs (warnings only, not blocking) and write_index() regenerates index.md from the full plan (not just a --pages subset). cmd_generate returns nonzero if any backend call failed.
Sources: src/repodocs/generate.py:L16-L37, src/repodocs/generate.py:L63-L185
backend.py abstracts the three LLM backends (REPODOCS_BACKEND=omp|claude|codex, default claude). backend_name()/require_backend() validate the env var; effective_model() resolves REPODOCS_MODEL, defaulting to claude-sonnet-5 for the claude backend and letting omp/codex use their own CLI defaults. backend_contract() loads the vendored AGENTS.md + planner/writer prompt files from PROFILE_SOURCE (repo-docs-profile/ next to the package) and caches them in _CONTRACT_CACHE, keyed by whether the prompt starts with "Plan the wiki pages" or "Write the wiki page" — this is how non-OMP backends (claude, codex) receive the same page-format/citation contract without depending on the target repo's own agent files. parallel_llm() (used by generate.py and translate.py) runs a ThreadPoolExecutor bounded by jobs_count().
Sources: src/repodocs/backend.py:L1-L94
translate.cmd_translate() (driven by translate_prompt()/translate_plan_prompt()) sends one backend call per generated page to translate prose while preserving markdown structure, code, mermaid blocks, and Sources: lines verbatim — the prompt explicitly instructs the model not to alter citation links. translate_plan_file() separately translates plan.json's title/purpose fields in a single call, copying the plan untranslated (with a warning) on failure. localize_headings() deterministically swaps the ## Relevant source files heading for its localized form (e.g. Portuguese ## Arquivos-fonte relevantes) outside of fenced code blocks. Output lands at <out>/<lang>/<slug>.md; existing translated pages are skipped unless --force. Citation lint and index.md generation are re-run against the translated tree afterward.
Sources: src/repodocs/translate.py:L1-L65
render.py builds <out>/wiki.html, a single self-contained dark-themed viewer that embeds the generated Markdown inline and renders it client-side with marked, mermaid, and highlight.js. Assets are loaded from a pinned CDN by default (CDN_ASSETS), each guarded by a Subresource Integrity hash (SRI) so a compromised CDN response is rejected by the browser; --vendor downloads the same files into <out>/assets/ for fully offline use (VENDOR_FILES/VENDOR_ASSETS). group_pages() buckets slugs into Overview/Features/Reference/Development nav groups deterministically, based on known slug names (OVERVIEW, DEV sets), preserving plan order and omitting empty groups. lang_labels() supplies localized UI strings (English/Portuguese) inferred from the output directory name, so a translated build at <out>/pt renders a Portuguese-labeled UI.
Sources: src/repodocs/render.py:L1-L80
citations.py implements the "source-cited" guarantee shared by generate, translate, and publish. CITATION_RE/FULL_CITATION_RE match [path:La-Lb] labels and full [path:La-Lb](path#La-Lb) links. lint_citations() is a non-blocking warning pass run after generate/translate, checking that cited paths exist and line ranges are in bounds. repair_citations() is applied to every raw LLM response before it's written to disk: it canonicalizes a citation whose label and href agree on path/range but the href resolves to a file only by unique filename suffix match (the model dropping a src/<pkg>/ prefix), or whose end line overshoots the file's real length — clamping it rather than leaving a bogus range; anything genuinely ambiguous is left untouched. requires_evidence() determines whether a page's prose (excluding the "Relevant source files" bullet list and Sources: lines) needs at least one citation. citation_problems() — after stripping HTML comments and code so a citation can't hide there — collects every citation/evidence violation across a set of pages, and enforce_citations() calls die() to hard-block publish/publish-wiki when any staged content page has missing or invalid citations, listing up to 12 problems.
Sources: src/repodocs/citations.py:L1-L36, src/repodocs/citations.py:L81-L194
gitlinks.py provides the shared git/GitHub URL utilities used by render, publish, and citations: _github_slug() parses an owner/repo slug from an ssh or https GitHub remote URL, validating each segment against _SLUG_PART so injected HTML/attribute metacharacters can't flow into a generated citation href; wiki_remote_url() derives the <owner>/<repo>.wiki.git clone URL in the same scheme as a given origin; github_base() combines the origin slug with the current HEAD SHA to build a https://github.com/<o>/<r>/blob/<sha> base used to rewrite citation links to absolute GitHub URLs at publish time.
Sources: src/repodocs/gitlinks.py:L1-L60
diagrams.cmd_render_diagrams (via diagrams.py) is an optional, clone-only step that finds every ```mermaid block (_MERMAID_RE) in the generated pages and renders it to a pastel PNG using tools/diagram_poster.ts, a Bun + Playwright tool located relative to the source checkout (tool_path() returns None when running from an installed wheel that didn't ship the tool). _poster_yaml() JSON-encodes title/kicker text into a YAML wrapper so special characters can't break parsing, and _render_png() shells out to bun per diagram, removing the intermediate YAML/HTML regardless of success. This step exists because GitHub's in-wiki mermaid renderer intermittently fails to load, while a committed PNG always displays; it is not part of the zero-dependency Python core and is meant to run after generate/all and before publish-wiki.
Sources: src/repodocs/diagrams.py:L1-L60
publish.py covers two targets: a GitHub Pages branch (cmd_publish, not fully read in this pass) and a GitHub Wiki (cmd_publish_wiki). Both start from _pages_for(), which reconstructs page order/titles from plan.json (falling back to sorted slugs for any .md present but absent from the plan), and stage_publish(), which copies the publishable tree (wiki.html → index.html, assets/, *.md, plan.json, plus per-language subdirectories) into a staging directory purely via the filesystem, rejecting any symlinked file or directory before it is read. publish_push() pushes the staged tree as a single orphan commit through a throwaway detached git worktree with a uuid-scoped local branch name, so the user's working tree and current branch are never touched, and force-pushes only the disposable docs branch. Before any real push, both publish paths call citations.enforce_citations() to block on citation problems and scan the staged tree for secret-like patterns.
Sources: src/repodocs/publish.py:L1-L100
cli.py defines one cmd_*/*_HELP pair per subcommand and a main() dispatcher. parse_repo_and_flags() extracts the positional repo path (defaulting to .) while skipping flags that take a value (--out, --pages, --lang, --branch, --remote). cmd_all() implements repodocs all: it optionally shells out to graphify update (unless --no-graph), then calls cmd_scan, plan.llm_plan, generate.cmd_generate, and render.build_html(..., vendor=True) in sequence, returning the first nonzero exit code encountered. cmd_setup() installs the vendored repo-docs-profile/ into ~/.omp/profiles/repo-docs/agent/ for the OMP backend only, via the pure setup_install() helper (missing → installed, identical → up to date, differing → kept local unless --force).
Sources: src/repodocs/cli.py:L193-L336, src/repodocs/cli.py:L417-L457