planning
- src/repodocs/plan.py
- tests/test_fix_plan.py
plan.py produces plan.json, the artifact describing which wiki pages a run of repodocs will generate and which source files each page should be written from. Planning is either delegated to an LLM backend or, if that fails, falls back to a deterministic heuristic so the tool always produces a usable plan.
Sources: src/repodocs/plan.py:L1-L16
llm_plan(repo, out, dry_run=False, force=False) is the primary driver. It scans the repository for inventory facts via scan_inventory, builds a planner prompt, and either prints the prompt (dry_run) or fingerprints it and checks for a cached plan.json/.plan.hash pair. If the cached fingerprint matches the current prompt and the cached JSON validates, the cached plan is reused (--force bypasses this). Otherwise it calls run_llm(repo, prompt), parses and validates the response, and checks that all mandatory_slugs are present. On any of FileNotFoundError, subprocess.TimeoutExpired, ValueError, or json.JSONDecodeError (including a raised ValueError for missing mandatory slugs or empty results), it prints a warning to stderr and falls back to plan_pages(repo, scan(repo, out)), the heuristic planner. The resulting pages are written to out/plan.json; the fingerprint file is only written when the plan came from the LLM, and is deleted if stale, so a heuristic fallback is never mistaken for a validated cached plan on the next run.
Sources: src/repodocs/plan.py:L240-L282
load_plan(repo, out, allow_omp=True) is the read path used by later pipeline stages: if out/plan.json exists it is loaded and revalidated with validate_pages; if it's missing or corrupt, and allow_omp is true, it invokes llm_plan to (re)plan; if allow_omp is false (dry runs), it falls back directly to the heuristic plan_pages.
Sources: src/repodocs/plan.py:L285-L296
planner_prompt(repo, inv) assembles the text sent to the LLM backend. It lists up to 200 source files and up to 40 README headings from the inventory (inv), computes mandatory_slugs(inv), and states the granularity rule: one page per feature (README heading, state machine, subcommand, tool, config surface, integration, or storage format), targeting 15-30 pages for a substantial repo, plus optional security, limitations, per-integration, and migration pages. It appends an optional graph_digest(repo) block and instructs the model to output only a JSON array matching the page schema (slug, title, purpose, files), with slugs matching ^[a-z0-9-]+$.
Sources: src/repodocs/plan.py:L162-L190
mandatory_slugs(inv) always includes "overview", then conditionally adds "installation" (README or manifests present), "architecture" (2+ source files), "changelog", "security", "contributing", and "development" (tests, CI, or CONTRIBUTING present) based on inventory facts.
Sources: src/repodocs/plan.py:L144-L159
graph_digest(repo, max_nodes=25, max_files=15) reads an optional precomputed graphify graph at <repo>/graphify-out/graph.json (NetworkX node-link format). It returns "" on any of OSError, UnicodeDecodeError, json.JSONDecodeError, KeyError, or TypeError, or if nodes/links are missing or empty — repodocs never requires graphify. When present, it computes node degree across edges to rank "god nodes" (most-connected concepts) and counts imports/imports_from edges per target file to rank "most-imported files." Both lists are formatted into a text block prepended to the planner prompt, instructing the LLM to prefer this digest over exploring files directly.
Sources: src/repodocs/plan.py:L92-L141
test_graph_digest_survives_invalid_utf8 confirms that a graph.json containing invalid UTF-8 bytes causes graph_digest to return "" rather than raising.
Sources: tests/test_fix_plan.py:L12-L16
parse_pages(text) strips a wrapping code fence if present, then extracts the substring between the first [ and last ] and parses it as JSON, raising ValueError if no array delimiters are found.
Sources: src/repodocs/plan.py:L193-L201
validate_pages(repo, raw) is the safety boundary between LLM/cached output and the rest of the pipeline. For each entry it: rejects non-dict entries; rejects slugs failing SLUG_RE; drops duplicate slugs; filters files to strings that pass safe_repo_file(repo, f) (rejecting path traversal such as "../evil"); and if every candidate file was invalid, anchors the page on README.md/readme.md if present. Dropped entries and files are logged to stderr. The output is capped at CANDIDATES_PER_PAGE files per page.
Sources: src/repodocs/plan.py:L204-L232
test_validate_pages_drops_null_files_instead_of_crashing confirms files: null is coerced to an empty list rather than raising. test_load_plan_validates_cached_plan_rejecting_traversal_slug confirms that even a previously-written cached plan.json is re-validated on load, dropping an entry with slug "../evil". test_load_plan_recovers_from_corrupt_cache confirms a plan.json containing invalid JSON causes load_plan to fall back to the heuristic plan (first page "overview") rather than raising.
Sources: tests/test_fix_plan.py:L19-L53
plan_fingerprint(prompt) returns the SHA-256 hex digest of the prompt text — the plan is treated as a pure function of the prompt. llm_plan compares this fingerprint (stored in out/.plan.hash) against the current prompt before deciding whether to reuse plan.json. test_llm_plan_falls_back_when_missing_mandatory_slug verifies that when the mocked LLM output omits the mandatory "installation" slug (repo has a README), llm_plan falls back to the heuristic plan and does not write .plan.hash, so a subsequent run will not mistake the fallback for a validated LLM plan.
Sources: src/repodocs/plan.py:L235-L282, tests/test_fix_plan.py:L56-L72
plan_pages(repo, facts) builds the same page schema (slug, title, purpose, files) without an LLM, using precomputed facts (from scan/scan_inventory). It always emits "overview" (README + manifests + top candidate source files by line count via top_candidates). It conditionally emits "installation" (README or manifests present) and "architecture" (2+ source files, using top_candidates over all source files).
Sources: src/repodocs/plan.py:L22-L44
For per-component pages, if facts["top_dirs"] has directories with 2+ files (excluding .), one component-<slug> page is emitted per directory, sorted by file count descending, each scoped to that directory's top candidate files. Otherwise, one component-<slug> page is emitted per individual source file with 100+ lines (excluding test files via is_test), scoped to that single file. Component pages are capped at MAX_COMPONENTS.
Sources: src/repodocs/plan.py:L59-L74
Slug collisions are avoided via alloc_slug, which appends -2, -3, etc. when two distinct directory or module names slugify to the same string. test_plan_pages_collision_safe_slugs_for_distinct_dirs verifies this: directories named "Foo Bar" and "foo-bar" both slugify to "foo-bar", and the test asserts both "component-foo-bar" and "component-foo-bar-2" appear with no duplicate slugs.
Sources: src/repodocs/plan.py:L46-L57, tests/test_fix_plan.py:L75-L93
Finally, plan_pages conditionally appends "development" (CONTRIBUTING.md, CI configs, or test files present) and "changelog" (CHANGELOG.md present).
Sources: src/repodocs/plan.py:L76-L88
| Field | Type | Description |
|---|---|---|
slug |
string | Kebab-case identifier matching ^[a-z0-9-]+$, unique per plan |
title |
string | Human-readable page title |
purpose |
string | One-sentence description of the page's scope |
files |
list of strings | Repo-relative candidate source file paths for the page |
Sources: src/repodocs/plan.py:L186-L190, src/repodocs/plan.py:L228-L231