Skip to content

Auto-build vite assets in docs/ justfile#26

Merged
tony merged 6 commits into
mainfrom
vite-build-docs
May 2, 2026
Merged

Auto-build vite assets in docs/ justfile#26
tony merged 6 commits into
mainfrom
vite-build-docs

Conversation

@tony
Copy link
Copy Markdown
Member

@tony tony commented May 2, 2026

Summary

  • Add _assets-build private recipe in docs/justfile that runs pnpm exec vite build before any HTML-output sphinx build, with auto-install of node_modules/ if missing.
  • Wire every HTML-output builder (html, dirhtml, singlehtml, epub, htmlhelp, qthelp, devhelp) as a dependent of _assets-build. Non-HTML builders skip the dependency.
  • Document the same pattern (justfile + Makefile snippets, plus an explanation of the sphinx-autobuild-vs-sphinx-build divergence) on docs/packages/gp-sphinx-vite.md for downstream users wiring up their own docs orchestration.

Closes the gap that took down https://gp-sphinx.git-pull.com/ during the v0.0.1a15 attempt: cold checkouts and just clean runs left furo-tw.css / furo.js absent from the theme's static directory, sphinx-build silently skipped the missing static files (no -W warning), and the deployed HTML referenced 403'd assets.

Why orchestration-layer, not extension-layer

sphinx-autobuild runs gp-sphinx-vite's builder-inited hook, which detects the autobuild via argv0 / SPHINX_AUTOBUILD env and spawns pnpm exec vite build --watch itself — so just start already Just Works (per commit 013884d's auto-pnpm-install). Plain sphinx-build runs in prod mode where the extension is intentionally a no-op (production wheel installs must not require a Node runtime). So the only correct place to plug the gap for just build is the orchestration layer that calls sphinx-build.

Changes

docs/justfile

Adds:

[private]
_assets-build:
    #!/usr/bin/env bash
    set -euo pipefail
    web_root=$(uv run python -c 'import gp_furo_theme; r = gp_furo_theme.get_vite_root(); print(r or "")' 2>/dev/null || true)
    if [ -z "$web_root" ]; then
        echo "[assets] gp_furo_theme.get_vite_root() returned None — assuming wheel install"
        exit 0
    fi
    if ! command -v pnpm >/dev/null 2>&1; then
        echo "[assets] pnpm not on PATH; skipping vite build"
        exit 0
    fi
    if [ ! -d "$web_root/node_modules" ]; then
        (cd "$web_root" && pnpm install --frozen-lockfile)
    fi
    (cd "$web_root" && pnpm exec vite build)

Each HTML-output recipe gains the : _assets-build prerequisite. clean is extended to wipe the vite output directory so the next build is genuinely cold.

docs/packages/gp-sphinx-vite.md

New ## Wiring just build / make build for downstream users section with the canonical recipe in both justfile and Makefile shapes, plus a pointer to gp_furo_theme.get_vite_root() for users on alternate theme parents.

Design decisions

Resolve the vite root via gp_furo_theme.get_vite_root(), not hard-coded path. The helper returns None for wheel installs (no web/ source tree exists, pre-built assets are already in the wheel) and the absolute path for workspace checkouts. The recipe degrades gracefully in both cases.

Skip silently when pnpm isn't on PATH. Some users may not have pnpm installed (e.g. installing gp-sphinx-vite from a wheel and never planning to author theme assets). The recipe skips with a one-line note rather than aborting the parent build — any pre-existing output remains in place.

Mirror gp_sphinx_vite.hooks._ensure_node_modules semantics in shell. Auto-install of node_modules/ on first run — same code path the autobuild flow already takes — so just clean; just html works without a separate pnpm install step.

Verification

Cold start works end-to-end:

$ rm -rf packages/gp-furo-theme/src/gp_furo_theme/theme/gp-furo/static/
$ just build-docs
[assets] running pnpm exec vite build in .../web
... 57.94 kB furo-tw.css, 4.51 kB furo.js
... build succeeded.

$ ls docs/_build/html/_static/styles/furo-tw.css docs/_build/html/_static/scripts/furo.js
   both present

Test plan

  • uv run ruff check . --fix --show-fixes — clean
  • uv run ruff format . — unchanged
  • uv run mypy — clean (197 source files)
  • uv run py.test --reruns 0 -vvv — 1317 passed, 159 skipped
  • just build-docs from a wiped static dir — produces styled output with both furo-tw.css and furo.js in _build/html/_static/
  • Manual verification that downstream-facing snippets in docs/packages/gp-sphinx-vite.md render correctly under just build-docs

Out of scope (follow-up)

The .github/workflows/{docs,tests,release}.yml workflows still call uv run sphinx-build / uv build / smoke runners directly without any pnpm/vite step — same root cause as the live-site outage, but on the CI side. Will land as a separate commit on this PR (and the PR's [DO NOT MERGE] branch trigger lets us validate the deploy actually styles correctly before merging to main).

The gp-furo-theme smoke runner in scripts/ci/package_tools.py is also separately broken (registers gp_furo_theme as an extension instead of via html_theme = "gp-furo", tripping the package's non-HTML-builder guard) — not in this PR's scope.

…L output

why: `just build-docs` (→ `just -f docs/justfile html`) called
plain `sphinx-build` directly, leaving the vite-managed theme
assets (`furo-tw.css`, `furo.js`) untouched. Those files are
gitignored on disk — they're build artifacts produced by
`packages/gp-furo-theme/web/`'s `pnpm exec vite build`. Cold
checkouts, post-`just clean` runs, and CI environments without a
prior vite build all hit the same gap: sphinx-build runs without
the assets, the deployed HTML still references
`_static/styles/furo-tw.css` and `_static/scripts/furo.js`, and
the live site renders unstyled. This is exactly what took down
https://gp-sphinx.git-pull.com/ when v0.0.1a15 was about to ship.

`just start` doesn't have the problem because it runs
`sphinx-autobuild`, and `gp-sphinx-vite`'s `_builder_inited` hook
detects autobuild via argv0 / `SPHINX_AUTOBUILD` env, then auto-
installs `node_modules/` (per
`gp_sphinx_vite.hooks._ensure_node_modules`) and spawns
`pnpm exec vite build --watch` itself. Plain `sphinx-build` runs
in `prod` mode where the extension is intentionally a no-op
(production wheel installs must not require Node), so the only
place the gap can be closed without breaking that contract is the
orchestration layer that calls `sphinx-build`.

what:
- docs/justfile: add a private `_assets-build` recipe that
  resolves the vite root via `gp_furo_theme.get_vite_root()`,
  auto-installs `node_modules/` if missing
  (`pnpm install --frozen-lockfile`), and runs
  `pnpm exec vite build`. Skips cleanly when running from a wheel
  install (no `web/` source tree) or when `pnpm` isn't on PATH.
  Mirrors `gp_sphinx_vite.hooks._ensure_node_modules` semantics
  in shell so both code paths agree on what "fresh enough" means.
- docs/justfile: declare `_assets-build` as a prerequisite of
  every HTML-output builder — `html`, `dirhtml`, `singlehtml`,
  `epub`, `htmlhelp`, `qthelp`, `devhelp`. Non-HTML builders
  (`latex`, `latexpdf`, `text`, `man`, `gettext`, `linkcheck`,
  `doctest`, `json`, `texinfo`, `info`, `redirects`, `changes`,
  `checkbuild`) skip the prerequisite — they don't render the
  theme assets.
- docs/justfile: `clean` now also wipes the vite-built static
  directory (`packages/gp-furo-theme/.../static/`) so the next
  `just build` is a true cold start. The assets are gitignored
  anyway and `_assets-build` reproduces them.

- docs/packages/gp-sphinx-vite.md: append a `## Wiring `just build`
  / `make build` for downstream users` section. Explains the
  sphinx-autobuild-vs-sphinx-build divergence (extension handles
  the former; the latter needs orchestration-layer help), then
  shows the canonical justfile + Makefile snippets and points
  users on alternate theme parents at `gp_furo_theme.get_vite_root()`
  as the helper to mirror in their own packages.

verified:
    $ rm -rf packages/gp-furo-theme/src/gp_furo_theme/theme/gp-furo/static/
    $ just build-docs
    [assets] running pnpm exec vite build in .../web
    ../src/.../static/styles/furo-tw.css   57.94 kB │ gzip: 11.77 kB
    ../src/.../static/scripts/furo.js       4.51 kB │ gzip:  1.71 kB
    ✓ built in ...
    ...
    Build finished. The HTML pages are in _build/html.
    $ ls docs/_build/html/_static/styles/furo-tw.css \
         docs/_build/html/_static/scripts/furo.js
    -rw-r--r-- 1 d d 4.5K ... furo.js
    -rw-r--r-- 1 d d  57K ... furo-tw.css

Pre-commit gate: ruff/format/mypy/pytest all green; the change is
justfile + docs only, no behavioural code edits — test counts
match the prior 1317 passed / 159 skipped baseline.

CI follow-up (not in this commit): `.github/workflows/{docs,tests,
release}.yml` still call `uv run sphinx-build` / `uv build` /
smoke runners directly without any pnpm/vite step. Switching the
workflow's build step from raw `sphinx-build` to
`(cd docs && just dirhtml)` (with `extractions/setup-just` +
`pnpm/action-setup` + `actions/setup-node` added before it) would
make CI pick up this fix transparently. Separate, focused commit.
@codecov-commenter
Copy link
Copy Markdown

codecov-commenter commented May 2, 2026

Codecov Report

✅ All modified and coverable lines are covered by tests.
✅ Project coverage is 89.05%. Comparing base (a3b7009) to head (ed1874e).

Additional details and impacted files
@@           Coverage Diff           @@
##             main      #26   +/-   ##
=======================================
  Coverage   89.04%   89.05%           
=======================================
  Files         183      183           
  Lines       15138    15150   +12     
=======================================
+ Hits        13480    13492   +12     
  Misses       1658     1658           

☔ View full report in Codecov by Sentry.
📢 Have feedback on the report? Share it here.

🚀 New features to boost your workflow:
  • ❄️ Test Analytics: Detect flaky tests, report on failures, and find test suite problems.

tony added a commit that referenced this pull request May 2, 2026
…e-build-docs

This commit is a SQUASH-OUT-BEFORE-MERGE marker for two distinct
changes that must land *together* on the validation branch but
should be cleaned up at merge time.

PERMANENT — keep on merge:

The pnpm/Node setup + `Install JS workspace` + `Build vite-managed
theme assets` steps are the actual fix for the live-site outage
that hit https://gp-sphinx.git-pull.com/ on the rolled-back v0.0.1a15
attempt. The workflow previously called `uv run sphinx-build -W`
directly without ever running `pnpm install` or `pnpm exec vite
build`, so the gitignored `furo-tw.css` / `furo.js` artifacts were
never produced — sphinx-build silently skipped the missing static
files (no `-W` warning), the deployed HTML referenced 403'd assets,
and the live site loaded unstyled. The new steps mirror what
``docs/justfile``'s `_assets-build` recipe does locally (PR #26):
install the JS workspace, then build the vite output before
sphinx-build runs.

TEMPORARY — drop on merge:

The `vite-build-docs` branch entry on the `push:` trigger fires
the live S3 + CloudFront deploy from this PR's branch instead of
waiting for merge to `main`. That lets us validate end-to-end —
"does the rendered site at gp-sphinx.git-pull.com actually load
furo-tw.css and furo.js this time?" — before any merge that would
move main forward without proof. The trigger entry MUST be
deleted (or this entire commit reverted and replaced with a
clean permanent-only commit) before squash-merge to main; leaving
it would let any future push to a branch named `vite-build-docs`
deploy production docs.

Recommended cleanup at merge time: revert this commit on the
branch with `git revert HEAD`, then add a clean follow-up commit
that re-introduces ONLY the permanent vite-build steps (without
the trigger expansion). Or delete this commit via interactive
rebase + squash, then create the cleaned-up version as a single
final commit before clicking merge.

what:
- .github/workflows/docs.yml `on.push.branches:` — add
  `vite-build-docs` entry (temporary).
- .github/workflows/docs.yml — add four steps between
  `Install just` and `Print Python versions`:
    * `Set up pnpm` via `pnpm/action-setup@v4` pinned to v10.
    * `Set up Node` via `actions/setup-node@v4` pinned to v22,
      with `cache: pnpm` for faster reruns.
    * `Install JS workspace` running
      `pnpm install --frozen-lockfile` from the workspace root.
    * `Build vite-managed theme assets` running
      `pnpm --filter @gp-sphinx/furo-theme-web exec vite build`.
  Each step gates on the same `steps.changes.outputs.publishable
  == 'true'` filter as the existing build / publish steps so
  no-op pushes (e.g. unrelated branches, README-only edits)
  still skip cleanly.

verification target (after this lands and the docs workflow
fires from this branch):
- https://gp-sphinx.git-pull.com/_static/styles/furo-tw.css → 200
- https://gp-sphinx.git-pull.com/_static/scripts/furo.js → 200
- The rendered home page loads with full styling.
- `gh run view` for the docs workflow shows a `Build vite-managed
  theme assets` step that produced ~58 kB of CSS + ~4.5 kB of JS.

Tracks: PR #26
why: The docs deploy workflow called `uv run sphinx-build -W` directly
without ever running `pnpm install` or `pnpm exec vite build`. The
gitignored `furo-tw.css` / `furo.js` artifacts were therefore never
produced in the runner workspace; sphinx-build silently skipped the
missing static files (no `-W` warning fires for stylesheets declared
in `theme.conf` that aren't on disk); the deployed HTML referenced
404'd `_static/styles/furo-tw.css` and `_static/scripts/furo.js`;
and the live site at https://gp-sphinx.git-pull.com/ rendered
unstyled after the v0.0.1a14 release.

This is the CI half of the same fix the docs/justfile gained in PR #26's
first commit: locally `_assets-build` runs vite before any HTML-output
sphinx build, gated by `gp_furo_theme.get_vite_root()` and shell-mirroring
`gp_sphinx_vite.hooks._ensure_node_modules`. The CI workflow needs the
same step inserted between dependency install and sphinx-build.

what:
- .github/workflows/docs.yml: add four steps between
  `Install just` and `Print Python versions`:
  - `Set up pnpm` via `pnpm/action-setup@v6` (current latest;
    https://github.com/pnpm/action-setup/releases) pinned to
    pnpm major version 10 to match the workspace lockfile
    (`pnpm-lock.yaml` lockfileVersion 9.0, compatible with
    pnpm 9-10).
  - `Set up Node` via `actions/setup-node@v6` (current latest;
    https://github.com/actions/setup-node/releases) on Node
    22 LTS with `cache: pnpm` for faster reruns.
  - `Install JS workspace` running `pnpm install --frozen-lockfile`
    from the workspace root (matches the local `_assets-build`
    recipe's `pnpm install` invocation when `node_modules/` is
    missing).
  - `Build vite-managed theme assets` running `pnpm --filter
    @gp-sphinx/furo-theme-web exec vite build`. The `--filter`
    targets the workspace package by its `name` in `package.json`
    so the recipe is path-independent: it works whether the
    workflow's CWD is repo root, `docs/`, or anywhere else.
  Each step gates on the same
  `steps.changes.outputs.publishable == 'true'` filter as the
  existing build/publish steps so unrelated branches and
  README-only edits skip cleanly.

verification:
- After this lands the docs workflow produces
  `furo-tw.css` (~58 kB) and `furo.js` (~4.5 kB) in the static
  output before sphinx-build copies them into `_build/html/_static/`.
- A clean run of `gh run view <run-id>` shows
  `Build vite-managed theme assets` between dependency install
  and sphinx-build.
- Once deployed, https://gp-sphinx.git-pull.com/_static/styles/furo-tw.css
  and /_static/scripts/furo.js return 200 (currently 403).
tony added a commit that referenced this pull request May 2, 2026
… branch

This commit is purely temporary scaffolding for PR #26. It lets us
validate the previous commit's vite-build-on-CI fix end-to-end —
"does the rendered site at gp-sphinx.git-pull.com actually load
furo-tw.css and furo.js?" — by triggering the live S3 + CloudFront
deploy from this PR's branch instead of waiting for merge to main.

The trigger entry MUST be deleted before squash-merge to main.
Leaving it would let any future push to a branch named
``vite-build-docs`` deploy production docs without a merge —
useful only as a once-off validation harness for this specific PR,
not as a permanent affordance.

Recommended cleanup at merge time: revert this commit on the
branch with ``git revert HEAD``, force-push, then squash-merge.
The previous commit (the actual permanent vite-build CI fix) lands
unaffected.

what:
- .github/workflows/docs.yml: add ``vite-build-docs`` to the
  ``on.push.branches:`` filter, with an inline ``[DO NOT MERGE]``
  comment marking the entry for removal at merge time.

Tracks: PR #26
@tony tony force-pushed the vite-build-docs branch from baac15e to 3643b49 Compare May 2, 2026 21:30
tony added a commit that referenced this pull request May 2, 2026
why: The justfile fenced block at gp-sphinx-vite.md:62 declared
`just` as the language, but Pygments has no `just` lexer (it's a
relatively new build tool with no built-in Pygments support in
stable releases). Sphinx falls back to the `none` lexer at parse
time and emits ``WARNING: Pygments lexer name 'just' is not
known [misc.highlighting_failure]``, which under the docs
workflow's ``sphinx-build -W`` flag becomes a hard error and
fails the build (caught on PR #26's first CI run, run #25262357575).

Tried `make` first — closest semantic match since just borrows
recipe-style ``target: deps`` / tab-indented bodies / variable
expansion from Make. But the Pygments make lexer can't tokenise
the just-specific ``[private]`` recipe annotation and emits
``Lexing literal_block ... as "make" resulted in an error at
token: '['. Retrying in relaxed mode.`` — still a warning, still
trips ``-W``.

Plain ``text`` is the right call: no syntax highlighting, no
lexer warnings. The Makefile snippet directly below uses
``makefile`` (which works fine, no just-specific syntax in that
block), so the highlight asymmetry between the two snippets is a
genuine reflection of which one Pygments can handle.

what:
- docs/packages/gp-sphinx-vite.md:62: replace ```` ```just ````
  with ```` ```text ```` on the downstream-facing justfile
  snippet.

verified: ``rm -rf docs/_build && uv run sphinx-build -W -b
dirhtml docs docs/_build/html`` returns ``build succeeded.`` with
zero warnings.
@tony tony force-pushed the vite-build-docs branch from bce8183 to 8f72e40 Compare May 2, 2026 22:49
@tony tony force-pushed the vite-build-docs branch from 8f72e40 to 4b2262c Compare May 2, 2026 22:52
tony added 3 commits May 2, 2026 18:15
why: The justfile fenced block at gp-sphinx-vite.md:62 declared
`just` as the language, but Pygments has no `just` lexer (it's a
relatively new build tool with no built-in Pygments support in
stable releases). Sphinx falls back to the `none` lexer at parse
time and emits ``WARNING: Pygments lexer name 'just' is not
known [misc.highlighting_failure]``, which under the docs
workflow's ``sphinx-build -W`` flag becomes a hard error and
fails the build (caught on PR #26's first CI run, run #25262357575).

Tried `make` first — closest semantic match since just borrows
recipe-style ``target: deps`` / tab-indented bodies / variable
expansion from Make. But the Pygments make lexer can't tokenise
the just-specific ``[private]`` recipe annotation and emits
``Lexing literal_block ... as "make" resulted in an error at
token: '['. Retrying in relaxed mode.`` — still a warning, still
trips ``-W``.

Plain ``text`` is the right call: no syntax highlighting, no
lexer warnings. The Makefile snippet directly below uses
``makefile`` (which works fine, no just-specific syntax in that
block), so the highlight asymmetry between the two snippets is a
genuine reflection of which one Pygments can handle.

what:
- docs/packages/gp-sphinx-vite.md:62: replace ```` ```just ````
  with ```` ```text ```` on the downstream-facing justfile
  snippet.

verified: ``rm -rf docs/_build && uv run sphinx-build -W -b
dirhtml docs docs/_build/html`` returns ``build succeeded.`` with
zero warnings.
why: Pygments has no built-in ``just`` lexer at the time of
writing, so docs that include justfile snippets either fall back
to ``\`\`\`text`` (no highlighting) or pick a near-relative like
``make`` that mistokenises just-specific syntax (``[private]``
recipe attributes, ``{{ … }}`` interpolations, ``set`` / ``import``
keywords). Under ``sphinx-build -W`` either path produces a hard
error or a useless render; gp-sphinx's docs are about to gain a
justfile snippet for the downstream ``just build`` recipe pattern,
so we need real highlighting.

Mirrors the pattern used by ``~/work/cihai/unihan-etl/docs/conf.py``
which registers ``CsvLexer`` / ``TsvLexer`` ``RegexLexer``
subclasses locally and assigns them via
``from sphinx.highlighting import lexers; lexers["csv"] = ...``.
Sphinx exposes the ``lexers`` module-level mapping for exactly
this purpose — extension code that wants a custom lexer plugs in
without needing a published Pygments entry-point package.

what:
- docs/conf.py: import ``RegexLexer``, ``bygroups``, ``include``
  from ``pygments.lexer``; the standard token types
  (``Comment``, ``Keyword``, ``Name``, ``Number``, ``Operator``,
  ``Punctuation``, ``String``, ``Text``); and ``lexers`` from
  ``sphinx.highlighting``.
- docs/conf.py: add ``JustfileLexer`` ``RegexLexer`` subclass
  covering the syntax we actually use:
  - ``# ...`` comments (``Comment.Single``)
  - ``[private]`` / ``[no-exit-message]`` recipe attributes
    (``Punctuation`` + ``Name.Decorator``)
  - ``set shell := [...]`` / ``import`` / ``mod`` / ``alias`` /
    ``export`` / ``unexport`` keywords (``Keyword.Reserved``)
  - ``var := "value"`` variable assignments
  - ``recipe-name dep1 dep2: prereq`` recipe headers
    (``Name.Function`` + ``Punctuation``)
  - ``{{ expression }}`` interpolations (``String.Interpol``)
  - ``"..."``, ``'...'``, ``\`...\`` string literals
  Recipe bodies are not parsed deeply — they're treated as Text
  until the next non-indented line. Most just bodies are shell,
  and accurate sub-shell highlighting is more work than the docs
  need. The lexer's ``aliases`` registers ``just`` and
  ``justfile``; ``filenames`` matches ``justfile`` / ``Justfile``
  / ``*.just``.
- docs/conf.py: ``lexers["just"] = JustfileLexer()`` registration.

scope note: the lexer is local to ``docs/conf.py``, not exposed
as a workspace package. Downstream consumers who want the same
highlighting in their own docs can copy the ``JustfileLexer``
block into their ``conf.py`` (or their own theme's bootstrap)
verbatim.
…le snippet

why: The local ``JustfileLexer`` registered in the prior commit
makes ``\`\`\`just`` resolvable for Pygments highlighting. Flip
the downstream-facing recipe block back from ``\`\`\`text`` (the
defensive landing after the ``\`\`\`make`` lexer choked on
``[private]``) to ``\`\`\`just`` so it renders with proper
syntax highlighting in the rendered docs. The rendered HTML now
wraps the block in ``<div class="highlight-just notranslate">``
and emits Pygments token classes for comments, the recipe
attribute, recipe headers, and string literals.

what:
- docs/packages/gp-sphinx-vite.md:62: replace ``\`\`\`text``
  with ``\`\`\`just`` on the downstream recipe-pattern snippet.

verified:
- ``rm -rf docs/_build && uv run sphinx-build -W -b dirhtml docs
  docs/_build/html`` returns ``build succeeded.`` with zero
  warnings (no ``Pygments lexer name 'just' is not known``
  warning fires now that ``lexers["just"]`` is populated).
- Rendered HTML at
  ``docs/_build/html/packages/gp-sphinx-vite/index.html`` shows
  the block highlighted as expected: ``<span class="c1">``
  comments, ``<span class="nd">private</span>`` decorator,
  ``<span class="nf">_assets-build</span>`` recipe name.
@tony tony force-pushed the vite-build-docs branch from 4b2262c to c1a22d3 Compare May 2, 2026 23:15
@tony tony merged commit 36c843a into main May 2, 2026
40 checks passed
@tony tony deleted the vite-build-docs branch May 2, 2026 23:27
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants