publishing
I have enough context to write the page now.
- src/repodocs/publish.py
- tests/test_fix_publish.py
repodocs.publish implements the two public-facing publish commands, repodocs publish (cmd_publish) and repodocs publish-wiki (cmd_publish_wiki), plus the shared safety machinery both rely on: a pure-filesystem staging step, a temporary git worktree/clone used to perform the actual push, a protected-branch guard, and a regex-based scan for leaked secrets in the staged tree. Neither command touches the caller's working tree or current branch — all git mutation happens inside disposable worktrees or clones.
Sources: src/repodocs/publish.py:L1-L18
cmd_publish requires out/wiki.html to already exist (built by repodocs html --vendor) and dies otherwise. It then runs, in order: the protected-branch check via publish_branch_safe, remote URL resolution (git remote get-url <remote>), a CDN-usage warning if wiki.html references cdn.jsdelivr.net, a citation-problems check via citation_problems over _publishable_subdir_mds(out), staging via stage_publish, a secret scan via staged_secret_findings, and finally either a dry-run listing or a real push via publish_push gated on --allow-public.
Sources: src/repodocs/publish.py:L160-L210
_publishable_subdir_mds enumerates the .md files that stage_publish will actually stage: the wiki's own top-level pages (excluding index.md) plus, for each subdirectory (translated variant) that already has a rendered wiki.html, that subdirectory's pages. Subdirectories without a rendered wiki.html are skipped so an in-progress translation cannot block publishing an otherwise-finished base wiki.
Sources: src/repodocs/publish.py:L146-L157
test_publishable_subdir_mds_skips_dir_without_rendered_html exercises exactly this gate.
Sources: tests/test_fix_publish.py:L15-L27
cmd_publish_wiki requires at least one .md page in out, resolves the remote URL, and derives the GitHub Wiki clone URL via wiki_remote_url(url) — dying if the remote isn't a github.com remote. It then computes a citation base (https://github.com/<slug>/blob/<head_sha>) only when the resolved slug and HEAD sha look valid, and additionally only when citations_safe(...) — checked against the selected remote's pushed branches, not hardcoded to origin — confirms the current HEAD has actually been pushed and the working tree is clean relative to out; otherwise citations are left relative and a warning is printed. Staging happens via stage_wiki before enforce_citations runs, specifically so a symlinked page is rejected by stage_wiki rather than being read first by the citation scanner. After a secret scan, the command either prints a dry-run listing or pushes via publish_wiki_push gated on --allow-public.
Sources: src/repodocs/publish.py:L351-L408
test_cmd_publish_wiki_stages_before_scanning_citations covers the staging-before-scanning ordering, and test_cmd_publish_wiki_index_only_reaches_home_fallback covers the case where only index.md exists (no overview.md) and must still reach the documented Home.md fallback.
Sources: tests/test_fix_publish.py:L193-L221
Both staging functions are pure-filesystem operations (no git, no network) that copy content from the out generation directory into a fresh temporary directory, and both explicitly reject symlinks before any read or copy — put_file/put_dir in stage_publish and the read helper plus a plan.json symlink check in stage_wiki.
stage_publish mirrors the GitHub Pages layout: it copies wiki.html to index.html, the assets/ directory, all top-level .md pages, and plan.json, for the base out directory and for each first-level subdirectory (translations) that itself has a wiki.html. It writes a .nojekyll marker and returns the sorted list of staged relative paths.
Sources: src/repodocs/publish.py:L36-L78
stage_wiki rewrites content into GitHub-Wiki layout: overview.md becomes Home.md (or index.md as a fallback source, never itself staged as a page), other .md pages are copied by stem name, PNG diagrams are copied verbatim, and a _Sidebar.md is generated from plan.json's page order/titles via _pages_for, falling back to each page's own # heading (via md_title) when the plan has no title. Citation links in every staged page are rewritten to absolute base blob URLs via rewrite_citation_links when a base is supplied.
Sources: src/repodocs/publish.py:L20-L33, src/repodocs/publish.py:L213-L266
test_stage_wiki_rejects_symlinked_plan_json and test_reject_symlink_dest_rejects_symlinked_file/..._parent_dir cover the symlink-refusal behavior on both the staging read path and the wiki-clone write path.
Sources: tests/test_fix_publish.py:L99-L157
publish_push pushes the staged tree as a single orphan commit to <remote>/<branch>. It creates a throwaway detached git worktree in a fresh temp directory, checks out a fresh orphan branch named repodocs-publish-<uuid4 hex> (independent of the target branch name, so a second publish to the same target branch never collides with a leftover local ref from a prior run), clears the orphan index/tree with git rm -rf --quiet ., copies every staged file in, commits, and force-pushes (git push -f) to HEAD:<branch>. The finally block always removes the worktree, deletes the local temp branch, prunes worktrees, and removes the temp directory — regardless of push success.
Sources: src/repodocs/publish.py:L81-L113
test_publish_push_repeat_same_branch_succeeds publishes twice to the same bare remote/branch and asserts no leftover repodocs-publish-* local branches remain in the source repo.
Sources: tests/test_fix_publish.py:L73-L94
Unlike publish_push, publish_wiki_push clones the wiki repo (git clone --depth 1 <wiki_url>), overwrites only the staged filenames in the clone (leaving any other, manually-added wiki pages untouched), and if the tree changed, commits and pushes normally — it never force-pushes. Each destination path is resolved through _reject_symlink_dest before being written, so a wiki clone that itself contains a symlink cannot redirect a write to an arbitrary local path. It returns the new commit sha, or None if nothing changed.
A clone failure is classified by matching stderr/stdout against _WIKI_UNINITIALIZED_RE ("repository.*(?:not found|does not exist)", case-insensitive/dotall), which narrowly matches GitHub's "no wiki pages yet" and a nonexistent local/bare remote path; a match raises WikiNotInitialized with a message directing the user to enable Wikis and create the first Home page, while anything else (auth failures, network errors) raises WikiPublishError with the real failure detail. All wiki-clone git commands go through _wiki_git, which raises WikiPublishError with bounded stderr/stdout (via failure_detail) rather than the raw argv, so a credential-bearing remote URL is never leaked in an error message.
Sources: src/repodocs/publish.py:L269-L348
test_clone_auth_failure_is_wikipublisherror_not_uninitialized and test_clone_missing_local_path_is_still_wikinotinitialized cover the two branches of this classification, and test_publish_wiki_push_rejects_symlinked_clone_destination_before_copy covers the _reject_symlink_dest gate.
Sources: tests/test_fix_publish.py:L119-L192
Resolves a relative path under a root directory component-by-component, raising ValueError as soon as any existing path component is itself a symlink — used to prevent a malicious wiki clone (or plan.json) from redirecting a staged write outside the intended destination tree.
Sources: src/repodocs/publish.py:L299-L308
PROTECTED_PUBLISH_BRANCHES = {"main", "master", "trunk"}
def publish_branch_safe(branch: str) -> bool:
return branch.removeprefix("refs/heads/").lower() not in PROTECTED_PUBLISH_BRANCHEScmd_publish calls this before any staging or push work and dies with refusing to force-push protected branch ... when it returns False, since publish_push always force-pushes. The check normalizes a refs/heads/ prefix and case before comparing, so refs/heads/Main and MAIN are both correctly recognized as unsafe.
Sources: src/repodocs/publish.py:L116-L143
test_publish_branch_safe_normalizes_refs_heads_prefix asserts main, Main, MAIN, and refs/heads/main are all rejected while docs and refs/heads/docs are accepted.
Sources: tests/test_fix_publish.py:L48-L55
publish-wiki has no equivalent branch guard since it pushes to the wiki's own history with a normal (non-force) push rather than a force-pushed docs branch.
PUBLISH_SECRET_PATTERNS = (
("private key", re.compile(r"-----BEGIN (?:RSA |EC |OPENSSH |DSA |ENCRYPTED )?PRIVATE KEY-----")),
("GitHub token", re.compile(r"\b(?:gh[pousr]_[A-Za-z0-9]{20,}|github_pat_[A-Za-z0-9_]{20,})\b")),
("cloud/API key", re.compile(r"\b(?:AKIA[0-9A-Z]{16}|sk-[A-Za-z0-9_-]{20,})\b")),
)| Label | Pattern matches |
|---|---|
private key |
PEM key headers, including ENCRYPTED PRIVATE KEY
|
GitHub token |
ghp_/gho_/ghu_/ghs_/ghr_ tokens and github_pat_ fine-grained tokens |
cloud/API key |
AWS access key IDs (AKIA...) and sk- style secret keys |
staged_secret_findings walks every file under the staging directory, reads it with errors="ignore", and for each matching line records (relative_path, line_number, label) — it never echoes the matched value itself. Both cmd_publish and cmd_publish_wiki run this scan after staging and before any push, dying with a bounded, value-free findings summary (first 10) if anything is found.
Sources: src/repodocs/publish.py:L119-L139, src/repodocs/publish.py:L182-L185, src/repodocs/publish.py:L386-L389
test_secret_scan_catches_encrypted_private_key and test_encrypted_private_key_pattern_matches_directly regression-test the ENCRYPTED PRIVATE KEY case specifically, since the pattern's optional key-type prefix must include ENCRYPTED .
Sources: tests/test_fix_publish.py:L58-L70
Both commands support a dry-run mode that lists staged files (and, for cmd_publish, prints the target remote/branch and a would-be GitHub Pages URL) without pushing, and require --allow-public to perform the actual push — cmd_publish dies with refusing public push without --allow-public; run --dry-run and review first and cmd_publish_wiki dies with the equivalent message when --allow-public is absent and dry_run is False.
Sources: src/repodocs/publish.py:L186-L198, src/repodocs/publish.py:L390-L399