Set up a docs site (mkdocs) for in-depth documentation - #93
Conversation
The depth was there but scattered: five package READMEs, ten files under docs/fleet, the inference-server doc, the design notes and the tuning runs, each reachable only by knowing it existed. The README refresh deliberately kept the README to sell-and-orient, which left nothing that could absorb it. `pixi run docs` serves a mkdocs Material site and `pixi run docs-build` builds it under --strict; a docs workflow gates every PR on that build and deploys main to GitHub Pages. Both live in their own pixi environment with no default feature, so the robot env is untouched and a contributor building the site solves mkdocs rather than ROS -- the lock diff adds the docs environment and removes nothing. The site does not move any documentation. A package README is read in a checkout and on GitHub at least as often as on a site, so it stays with its package and docs/hooks/repo_links.py mounts it at build time -- 24 files, from the package READMEs to design/BOM.md to the hardware research notes -- and the edit pencil on a mounted page opens the file it came from. Every tracked markdown file except the README and CLAUDE.md is now reachable from the nav. Mounting is what makes the links the interesting part. A copied file's relative links point into the repo, so each one is resolved against the file's real home and rewritten: to another site page where the target is one (a mounted file or anything under docs/), and to a GitHub blob URL where it is source the site does not serve. A figure from outside docs/ is copied in rather than linked, so the site never reaches back to GitHub to render its own pages. Anything the rewriter cannot resolve is left exactly as written, which makes it a mkdocs warning and fails --strict rather than shipping as a 404. Two things fell out of turning that on. Headings are slugged GitHub's way (pymdownx.slugs, set in the hook because spelling a Python object in YAML needs a tag check-yaml refuses to load): every heading link in the repo was written against a file rendered on GitHub, and the two sluggers disagree wherever a heading contains an em dash, which is most of them here. And mote_perception/README.md's two links to the inference-server doc pointed at ../../docs/, one level above the repo -- broken on GitHub too, and found by the first strict build. Three pages are new, because no single file covered what the nav needed to point at: a landing page, getting-started, and simulation and sites/maps/zones guides that tie the existing material together.
![claude[bot]](https://avatars.githubusercontent.com/in/1236702?s=60&v=4){kind=small}
|
|
||
| concurrency: | ||
| group: pages-${{ github.ref }} | ||
| cancel-in-progress: true |
There was a problem hiding this comment.
cancel-in-progress: true applies to the whole workflow-level concurrency group (pages-${{ github.ref }}), which also covers the deploy job below it (.github/workflows/docs.yml#L44-L56). Since every push to main lands in the same group, a second push to main while a previous run's deploy (the actual actions/deploy-pages publish) is still in flight will cancel it mid-deploy. GitHub's own Pages "starter workflow" deliberately keeps cancel-in-progress: false for exactly this reason — cancellation is only safe on the PR-only build path, not a path that includes a live production deploy.
Suggest scoping cancellation to non-main runs, so PR builds still cancel superseded runs but a live deploy on main is never killed:
| return config | ||
|
|
||
|
|
||
| def on_files(files, config): |
There was a problem hiding this comment.
_figures is a module-level dict (docs/hooks/repo_links.py#L75) populated by insertion inside _resolve (docs/hooks/repo_links.py#L103), which runs from on_files on every rebuild. MkDocs caches hook modules for the life of the mkdocs serve process, so this dict persists across live-reload rebuilds rather than resetting. If a mounted file's figure reference is edited out, or a figure is renamed/deleted, while pixi run docs is running, the stale entry survives and figure.read_bytes() (docs/hooks/repo_links.py#L174-L175) raises FileNotFoundError out of the hook, crashing the dev server.
Reset it at the top of on_files so each rebuild starts clean:
|
|
||
| concurrency: | ||
| group: pages-${{ github.ref }} | ||
| cancel-in-progress: true |
There was a problem hiding this comment.
(Correcting my comment above — the suggestion block was dropped. Here's the concrete fix, scoping cancellation to non-main runs so PR builds still cancel superseded runs but a live deploy on main is never killed:)
| cancel-in-progress: true | |
| cancel-in-progress: ${{ github.ref != 'refs/heads/main' }} |
| return config | ||
|
|
||
|
|
||
| def on_files(files, config): |
There was a problem hiding this comment.
(Correcting my comment above — the suggestion block was dropped. Here's the concrete fix, resetting _figures at the top of on_files so each rebuild starts clean:)
| def on_files(files, config): | |
| def on_files(files, config): | |
| _figures.clear() |
…ilds Two defects from review, both in the paths that only run outside a single `mkdocs build`. The workflow-level concurrency group covers the deploy job as well as the build, so `cancel-in-progress: true` meant a second push to main could cancel `actions/deploy-pages` half-way through publishing. Cancellation is now scoped to non-main refs, so a superseded PR build still dies and a deploy never does -- which is why GitHub's own Pages workflow declines to cancel at all. `_figures` is module state, and mkdocs loads a hook module once per process (`Hooks._load_hook` is lru_cached), so under `mkdocs serve` it accumulated across every rebuild instead of describing the current one. Rename or delete a figure mid-session and the stale entry outlives it, so the copy-in pass reads a path that is gone and takes the dev server down with FileNotFoundError. Reproduced by building twice in one process with the figure deleted between: build 2 raises before the clear, and succeeds after it.
1 participant
Set up a mkdocs (Material) documentation site so the in-depth docs have a
single navigable home, without moving them away from the code.
Branch: worktree-docs-site (committed, not pushed), one commit ed9af75.
What landed
mkdocs.yml+ adocspixi feature/environment (no-default-feature, likelint):pixi run docslive-serves,pixi run docs-buildbuilds with--strict. The lock diff is purely additive -- 1 removed line, the diffheader -- so the robot/default env is untouched.
.github/workflows/docs.yml:--strictbuild on every PR, deploy to GitHubPages on
main(upload-pages-artifact + deploy-pages).docs/hooks/repo_links.py: mounts 24 markdown files that live with the code(the five package READMEs, mote_arm/BENCH.md, the sub-package READMEs,
design/{README,BOM,ASSEMBLY,WIRING}.md and the five design research notes)
as site pages at build time, and rewrites their relative links -- to another
site page where the target is one, to a GitHub blob/tree URL where it is
source the site does not serve. Figures from outside
docs/are copied inunder
figures/so the site renders its own pages without reaching back toGitHub. Anything unresolvable is left as written, so it becomes a mkdocs
warning and fails
--strict. A mounted page's edit pencil opens the file itwas copied from.
zones, missions, map cleanup, Foxglove, chaos), perception + inference
server, the SO-101 arm, fleet (runbook, architecture, both contracts, server
pipelines, all six verification ledgers), simulation + its three harnesses,
and the tuning notes.
docs/index.md,docs/getting-started.md,docs/simulation.md, plusdocs/robot/sites.mdfor sites/maps/zones.fleet and contributing sections.
Nothing was deleted or moved. Every tracked
.mdexcept the root README andCLAUDE.md is reachable from the nav (verified programmatically against
git ls-files '*.md').Two real bugs the strict build found
mote_perception/README.mdlinked the inference-server doc twice as../../docs/inference-server.md, one level above the repo root -- broken onGitHub too. Fixed.
pymdownx.slugs, set inon_configrather than YAML because the tag
check-yamlrefuses to load). Every headinglink in the repo was written against GitHub's renderer, and the two sluggers
disagree wherever a heading has an em dash -- which is most of them here.
Without this, ~10 cross-references landed at the top of the page instead of
the section.
validation.anchors: warnnow keeps that true.Verified
pixi run docs-build: 49 pages, zero warnings under--strict(withomitted_files,absolute_links,unrecognized_linksandanchorsall setto warn).
pixi run docs: serves on http://127.0.0.1:8000/Mote/ (thesite_urlpath);home,
robot/sites/,fleet/andarm/bench/all 200.copied-in figures, tables, code blocks and rewritten source links.
pixi run lint: 12/12 hooks pass.Needs a human
GitHub Pages has to be enabled in repo settings with Source: GitHub Actions
before the deploy job can publish; until then the PR build check still runs.
The site URL assumed throughout is https://clachdev.github.io/Mote/.