utilities

Shared Utilities

Relevant source files

• src/repodocs/_util.py

_util.py is the internal module holding constants and small helper functions shared across the pipeline stages — scan, plan, generate, render, translate, diagrams, citations, backend, publish, and cli — so that filesystem safety checks, error handling, and naming conventions stay consistent everywhere they're used.

Sources: src/repodocs/_util.py:L1-L9

Constants

Name	Value	Purpose
SRC_EXTS	set of source file extensions (.py, .js, .rs, .go, ...)	recognizing source files by suffix
MANIFESTS	["package.json", "pyproject.toml", "Cargo.toml", "go.mod", "Gemfile", "composer.json"]	manifest files scanned for repo metadata
SKIP_DIRS	{".git", "node_modules", "dist", "build", "__pycache__", "vendor", ".venv", "venv"}	directories excluded from scanning
MAX_DEPTH	5	scan recursion depth cap
MAX_COMPONENTS	6	cap used during planning
CANDIDATES_PER_PAGE	12	cap on candidate files per wiki page
SLUG_RE	^[a-z0-9-]+$	validates page slugs

Sources: src/repodocs/_util.py:L11-L33

die() — uniform fatal error exit

def die(msg: str, code: int = 1):
    print(msg, file=sys.stderr)
    sys.exit(code)

die() prints msg to stderr and terminates the process with code (default 1). It is the single exit point used by cli.py, backend.py, render.py, translate.py, diagrams.py, citations.py, generate.py, and publish.py whenever a stage encounters an unrecoverable error, giving the CLI a consistent error-reporting and exit-code convention across all pipeline stages.

Sources: src/repodocs/_util.py:L36-L38

log() — timestamped progress output

def log(msg: str):
    """Timestamped progress line to stderr."""
    print(f"[{time.strftime('%H:%M:%S')}] {msg}", file=sys.stderr)

log() writes an HH:MM:SS-prefixed progress line to stderr. It is used by plan.py, generate.py, and diagrams.py to report progress without polluting stdout, which is reserved for generated content.

Sources: src/repodocs/_util.py:L41-L43

safe_repo_file() — path traversal guard

def safe_repo_file(repo: Path, rel: str) -> Path | None:
    ...

This is the core filesystem-safety helper. Given a repo root and a repo-relative path rel, it resolves rel under repo (following symlinks) and returns the resolved Path only if it is a regular file that stays inside the repo root. It returns None for:

• absolute paths (os.path.isabs(rel))
• paths containing NUL bytes ("\x00" in rel)
• .. traversal or symlink escapes outside repo (checked via target.relative_to(root), catching ValueError)
• symlink loops or other resolution failures (OSError, RuntimeError)
• targets that are not regular files (target.is_file())

safe_repo_file() is called from scan.py, plan.py, generate.py, and citations.py (and covered directly by tests/test_fix_scan_util.py and tests/test_repodocs.py) any time a repo-relative path supplied by a candidate list, citation, or plan entry needs to be dereferenced against the repository on disk — guarding against path traversal or symlink-escape attempts in untrusted repository content.

Sources: src/repodocs/_util.py:L77-L90

Other small helpers

is_source(p) classifies a Path as source code: it checks the suffix against SRC_EXTS, and for extension-less files, peeks at the first two bytes for a #! shebang (covering scripts like hat or repodocs). is_test(rel) flags a relative path as a test file by checking for tests/test path components, a test_ filename prefix, or a _test./.test./.spec. suffix pattern. count_lines(p) counts newlines in a file by reading it in binary mode, returning 0 on OSError. slugify(name) lowercases and collapses non-alphanumeric runs into single hyphens, falling back to "component" for empty results, used to derive page slugs. dedup(items) removes duplicates from a list while preserving order.

Sources: src/repodocs/_util.py:L46-L74, src/repodocs/_util.py:L93-L103

These helpers are consumed by scan.py (is_source, is_test, count_lines, safe_repo_file, plus the MANIFESTS/MAX_DEPTH/SKIP_DIRS constants) and plan.py (is_test, dedup, slugify, safe_repo_file, SLUG_RE, MAX_COMPONENTS, CANDIDATES_PER_PAGE), keeping filename classification and slug generation consistent between the scanning and planning stages.

Sources: src/repodocs/_util.py:L11-L94