scan

Repository Scanning

Relevant source files

• src/repodocs/scan.py
• tests/test_fix_scan_util.py

Overview

repodocs.scan performs a single, deterministic filesystem walk of the target repository and produces the inventory artifact that the planning stage consumes. It exposes three functions: scan(), which walks the tree and collects raw facts; readme_headings(), which extracts heading text from the repo's README; and scan_inventory(), which combines both into the final structured record.

Sources: src/repodocs/scan.py:L1-L11

scan(): the filesystem walk

scan(repo, out=None) walks repo with os.walk, skipping repodocs' own generated trees (repo-docs/, graphify-out/, and whatever directory out points to) so that a rerun never ingests its own vendored output. Directories are also pruned by SKIP_DIRS, dotfile directories, and a MAX_DEPTH cutoff. For each candidate directory it resolves the --out directory (if any) and compares it against each subdirectory's resolved path so the pruning works regardless of how deeply nested out is under repo.

Sources: src/repodocs/scan.py:L11-L34

For files, scan() checks fp.is_symlink() first and drops any symlink whose resolved target falls outside the repository root (repo_real) — this prevents a planted symlink from smuggling an out-of-tree file into the wiki. Remaining files are tested with is_source(); matches are added to src_files, bucketed by top-level directory in top_dirs, and line-counted via count_lines() into line_counts.

Sources: src/repodocs/scan.py:L35-L47

CI workflow files are gathered separately by globbing .github/workflows/* and filtering each entry through safe_repo_file(), which drops symlinks that escape the repo or loop back on themselves. The list is sorted before being returned.

Sources: src/repodocs/scan.py:L49-L56

Returned fact dictionary

scan() returns a dict with the following keys:

Key	Description
src_files	Sorted list of repo-relative source file paths
line_counts	Dict mapping each source file to its line count
top_dirs	Dict mapping top-level directory name (or ".") to its list of source files
has_readme	Whether README.md or readme.md exists and isn't an escaping symlink
manifests	Subset of MANIFESTS present in the repo
has_contributing / has_changelog / has_security	Presence of CONTRIBUTING.md, CHANGELOG.md, SECURITY.md
ci	Sorted list of .github/workflows/* file paths
tests	Subset of src_files identified as test files by is_test()

All presence checks use the has() helper, which calls safe_repo_file(repo, name) so that escaping or looping symlinks never register as present.

Sources: src/repodocs/scan.py:L49-L68

readme_headings()

readme_headings(repo) reads README.md (or readme.md) line by line, toggling a fenced flag on lines beginning with triple backticks, and collects lines matching ^#{1,6}\s+\S outside of fenced code blocks. This avoids picking up #-prefixed text inside code samples (e.g. shell comments) as headings. Returns an empty list if no README exists.

Sources: src/repodocs/scan.py:L71-L82

scan_inventory(): the final artifact

scan_inventory(repo, out=None) calls scan() and readme_headings() and assembles the combined inventory dict that downstream planning consumes:

{
    "name": <repo directory name>,
    "source_file_count": <int>,
    "source_files": [...],
    "manifests": [...],
    "readme_headings": [...],
    "has_readme": bool,
    "has_contributing": bool,
    "has_changelog": bool,
    "has_security": bool,
    "ci": [...],
    "tests": [...],
}

Note that line_counts and top_dirs from the raw scan() facts are not carried into scan_inventory()'s output; only source_files, source_file_count, and the presence/list fields above are exposed.

Sources: src/repodocs/scan.py:L85-L99

Symlink and path safety

Scanning treats three distinct symlink hazards, each covered by a regression test:

• Escaping source symlinks: a symlink resolving outside repo_real is excluded from src_files before it's ever considered a source file.
• Escaping metadata symlinks: README.md, manifest files, and CI workflow files that are symlinks pointing outside the repo are excluded via safe_repo_file(), so has_readme, manifests, and ci don't report them as present.
• Symlink loops: a self-referencing symlink (e.g. loop -> loop) causes Path.resolve() to raise OSError (ELOOP); safe_repo_file() catches this and returns None rather than propagating the exception.

Sources: tests/test_fix_scan_util.py:L13-L23, tests/test_fix_scan_util.py:L51-L72, tests/test_fix_scan_util.py:L106-L112

Non-regular files and nested --out

Dangling symlinks (pointing to a nonexistent target) and non-regular files such as FIFOs are excluded from src_files and line_counts, since scan() only processes entries where fp.is_file() is true. A --out directory nested arbitrarily deep under the repo (e.g. src/generated/) is pruned during the walk while sibling directories (src/actual/) are scanned normally, since the out-directory comparison is resolved against each candidate subdirectory rather than only at the repo root.

Sources: tests/test_fix_scan_util.py:L13-L48

CI workflow ordering

The ci list returned by scan() is always sorted lexicographically, regardless of the order files are created or returned by the filesystem, and excludes any workflow file that is a symlink escaping the repo.

Sources: tests/test_fix_scan_util.py:L75-L103