docs: PDF output + docs CI validation (Phase 4+5 of N) - #124
Merged
Conversation
Implement build_pdf(): render each top-level book (2 API refs + 9 guides + articles' 2 sub-books = 13 PDFs) to docs-build/pdf/<book>.pdf. Chapters are concatenated in _meta.toml order (API refs: index first, then alphabetical); a CSS title page carries the live version (dist/RELEASE, 5.3.33) + copyright, with a running header/footer via CSS paged-media (_templates/pdf-print.css). Engine: pandoc(html) -> weasyprint. Chosen over LaTeX because it needs no TeX toolchain, is deterministic, and reuses the existing HTML path. All 13 books render (api_c 655pp, programmer_reference 370pp, ...) in ~3.5 min total. Also fixes two real bugs the link-check surfaced: pandoc_md_to_html now rewrites cross-tree .md links (../../api/c/foo.md) to .html, and build_html copies tree images (docs-src/<tree>/img/*) into the built site so <img> links resolve. Adds real stub pages for the 2 genuinely-undocumented public APIs (db_env_set_func_assert, db_env_set_win_security) written from their db.h prototypes + source comments, so the completeness gate can be a hard 100%. flake.nix devShell gains the docs validation toolchain (weasyprint, poppler-utils, mandoc, codespell, lychee, write-good) so CI matches local.
Add .github/workflows/docs.yml (modeled on ci.yml/fuzz.yml; nix devShell for
tool parity). Triggers: push to master, PRs touching docs-src/**, dispatch,
and a weekly schedule.
HARD gates (block PRs):
- build: build.py --no-pdf (HTML + man, 0 errors) + self-check
- no-loss: verify_all.py runs verify.py over all 13 migrated trees; fails on
any content drop (retention < threshold or a code/section hard drop)
- completeness: man_coverage.py --ci; 28/28 public functions documented, and
every uncovered method must be on the frozen allowlist (a NEW undocumented
API fails). Matcher also taught the dbsite/set_-drop DocBook stem shapes.
- spelling: spellcheck.py runs codespell keyed on (path, word), baselined to
the ~153 legacy typos so only newly-introduced typos fail
- internal link-check: lychee --offline + lychee.toml (deferred-tree and
un-migrated-asset links excluded); every migrated internal link must resolve
- man-lint: mandoc -Tlint over all .3, 0 ERRORS
ADVISORY (continue-on-error): prose (write-good passive/wordiness counts).
BEST-EFFORT/scheduled: pdf build + validate_pdf.py (slow), external link-check.
Every command was dry-run inside 'nix develop' and passes.
am_second.md points at 'second.javas' (a malformed link to a Java API example in the deferred java tree); the file is absent in the upstream DocBook too, so it is a pre-existing dead link, not a migration regression. The flake-pinned lychee 0.24.1 (what CI's nix develop uses) extracts it where an older lychee did not, so add it to the exclude set. Full gate suite re-verified inside 'nix develop' (the exact CI env): 0 link errors, all gates pass.
Coccinelle convention checksNo new violations. ✅ Resolved since baseline (2) -- update dist/cocci/baseline.txt to lock these in. |
ABI diff vs
|
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Phase 4 — PDF (13 per-book PDFs)
Engine: pandoc(HTML) → weasyprint (rejected TeX: tectonic needs network, texlive heavy). Deterministic, reuses the HTML path, renders all books in ~3.5 min. Title page (project + live version from dist/RELEASE + copyright) + running header/footer via CSS paged-media. Books: api_c (655p), programmer_reference (370p), api_stl (257p), installation, upgrading, gsg_txn, gsg, collections, gsg_db_rep, bdb-sql, articles×2, porting — all title pages carry Berkeley DB 5.3.33.
validate_pdf.py: 13/13 PASS.Phase 5 — CI (
.github/workflows/docs.yml)Triggers: push→master, PRs touching docs-src/**, dispatch, weekly. Hard gates: build (HTML+man, 0 err) · no-loss (all 13 trees, 0 drops) · completeness (
man_coverage.py --ci) · spelling · internal link-check (lychee, 14908 links / 0 errors) · man-lint (mandoc, 0 ERRORS / 787+ pages). Advisory: prose (write-good), external links (225 rotted — expected for decades-old docs). Best-effort/scheduled: PDF build+validate.Completeness resolution: wrote real stub pages for the 2 genuinely-undocumented APIs (
db_env_set_func_assert,db_env_set_win_security) → functions gate = 28/28 (100%); the 40 getter/vtable methods are a frozen allowlist (a NEW undocumented method fails CI).Independently verified
1465 HTML + 789 man + 13 PDF built (all 5.3.33); no-loss gate PASS (13 trees); completeness gate PASS (28/28). Touched only docs-src/, docs.yml, flake.nix (devShell), 2 stub .md. Phase 6 (publish gh-pages) is next.
Honest tradeoffs: ~153 legacy prose typos baselined (fixing churns no-loss-protected content); spelling gate blocks only NEW typos. weasyprint over TeX (documented in PLAN.md).