Build `sphinx-vite-builder`: PEP 517 backend + Sphinx extension for vite-aware projects

[tony]

Product vision

Build sphinx-vite-builder — a single Python package that any Sphinx theme or docs project using vite can consume. It exposes two orthogonal entry points:

1. PEP 517 build backend — [build-system].build-backend = "sphinx_vite_builder.build" runs pnpm exec vite build before delegating wheel/sdist construction to hatchling. End users pip install from PyPI/sdist without needing pnpm; contributors uv sync and the editable install Just Works.
2. Sphinx extension — extensions = ["sphinx_vite_builder"] in conf.py auto-orchestrates vite at docs-build time: one-shot vite build for sphinx-build, vite build --watch child process for sphinx-autobuild. No justfile, no Makefile, no manual pnpm exec vite build step.

Both heads share a smart subprocess core built on asyncio + rich, absorbing the production-grade design already shipping in gp-sphinx-vite (this workspace's existing extension that we'll consolidate into sphinx-vite-builder).

The package is generic — designed for any vite-aware Sphinx project, not just gp-furo-theme. It joins the family of tools that own a native toolchain end-to-end: maturin (Rust), sphinx-theme-builder (webpack), now sphinx-vite-builder (vite).

Study these first

• PR Auto-build vite assets in docs/ justfile #26 (Auto-build vite assets in docs/ justfile) — the just/Makefile orchestration that sphinx-vite-builder's extension head replaces. The recipes and rationale there inform the extension's auto-hookup behavior.
• PR Fail loud when vite assets are missing or pnpm bootstrap fails #27 (Fail loud when vite assets are missing or pnpm bootstrap fails) — the runtime fail-loud diagnostics that ship the consumer-side error reporting. sphinx-vite-builder extends this with build-time equivalents in both heads.
• The four research dossiers below collect everything we found in the upstream references (maturin, uv, flit, hatchling, sphinx-theme-builder, sphinx, myst-parser, furo, vite, vitest, pnpm) plus a handful of in-house async/subprocess + rich orchestration references (an async dev-server, a batch-orchestration script, and a build/dev pair from a docs pipeline).

Action

Open a new PR after #27 merges that introduces packages/sphinx-vite-builder/ and consolidates the existing gp-sphinx-vite orchestration into it. Migration path detailed below in Implementation phases.

What changed from the original framing

This issue's first version proposed a 580-LOC backend that wraps hatchling. The expanded scope is:

Aspect	Original (was)	Now
Scope	PEP 517 backend only	Backend + Sphinx extension (two entry points, one package)
Audience	Workspace-internal	Generic, publishable from day one
Subprocess management	Bare subprocess.run	asyncio + rich, modeled on gp-sphinx-vite
Auto-hookup	None	sphinx-build and sphinx-autobuild run vite automatically when extension is loaded
LOC budget	~580	~1200 (backend ~150, extension ~250 incl. hooks, subprocess core ~400 absorbed from gp-sphinx-vite, tests ~400)

Architecture

sphinx_vite_builder/                       # The Python package
├── __init__.py                            # Sphinx extension entry: setup(app)
├── build.py                               # PEP 517 backend entry (pure functions)
├── _internal/
│   ├── process.py                         # SmartAsyncProcess (asyncio + rich)
│   ├── bus.py                             # AsyncioBus (sync↔async bridge for Sphinx hooks)
│   ├── vite.py                            # pnpm/vite invocation, pnpm/PATH detection
│   ├── config.py                          # detect_mode() — autobuild/dev/prod heuristic
│   ├── hooks.py                           # builder-inited / build-finished handlers
│   └── errors.py                          # PnpmMissingError, ViteFailedError, etc.
└── _testing/
    └── fixtures.py                        # pytest fixtures for fake-pnpm / fake-vite

Both __init__.py:setup (Sphinx extension) and build.py:build_wheel (PEP 517 backend) consume the same _internal/ core. The two heads never call each other; they share substrate.

Layer 1 — Smart subprocess core (_internal/process.py, _internal/bus.py)

Absorbed from gp-sphinx-vite/process.py + bus.py. Production patterns we keep:

• Asyncio-native subprocess — asyncio.create_subprocess_exec() with PIPE'd stdout/stderr; concurrent line-by-line drainers via asyncio.gather(*drainers, return_exceptions=True) to prevent deadlock (existing pattern at gp-sphinx-vite/process.py:74-187)
• Graceful SIGTERM → SIGKILL escalation with configurable timeout (process.py:terminate()), idempotent across repeated calls
• POSIX process group isolation — os.setsid() so SIGTERM kills the whole vite tree (improvement to add over current code)
• PYTHONUNBUFFERED=1 injected into subprocess env so chained Python tools don't buffer
• Rich-aware logging — drainers prefix each line with [label] and route to a rich Console
• AsyncioBus — single asyncio loop in a daemon thread; call_sync() uses asyncio.run_coroutine_threadsafe to let sync Sphinx hooks await async coroutines (bus.py:93-121)
• Idempotent teardown — atexit.register() + signal.signal(SIGINT/SIGTERM/SIGHUP) chains with weak refs so long-lived processes don't leak (hooks.py:231-273)

Layer 2 — Vite orchestration (_internal/vite.py)

• Detect package opt-in: web/ dir alongside pyproject.toml ⇒ vite-managed; absent ⇒ unpacked sdist or non-vite package, no-op
• shutil.which("pnpm") check up-front; raise PnpmMissingError with corepack enable + pnpm.io/installation hints
• node_modules/ check; run pnpm install --frozen-lockfile if missing
• One-shot mode: pnpm --filter <package> exec vite build (returns when complete)
• Watch mode: pnpm --filter <package> exec vite build --watch (long-lived SmartAsyncProcess)
• Capture all stderr with build context; surface via ViteFailedError on non-zero exit

Layer 3a — PEP 517 backend (build.py)

Mirrors flit_core's buildapi.py (85 LOC) shape. Delegates wheel construction to hatchling (verified pure-function in hatchling/src/hatchling/build.py:17-84):

# packages/sphinx-vite-builder/sphinx_vite_builder/build.py (sketch, ~50 LOC)
from __future__ import annotations
import typing as t
import hatchling.build as _hatchling
from ._internal.vite import run_vite_build, VitePhase

def build_wheel(wheel_directory, config_settings=None, metadata_directory=None):
    run_vite_build(VitePhase.WHEEL)
    return _hatchling.build_wheel(wheel_directory, config_settings, metadata_directory)

def build_editable(wheel_directory, config_settings=None, metadata_directory=None):
    run_vite_build(VitePhase.EDITABLE)
    return _hatchling.build_editable(wheel_directory, config_settings, metadata_directory)

def build_sdist(sdist_directory, config_settings=None):
    run_vite_build(VitePhase.SDIST)  # Pre-bake static/ so sdist→wheel works without pnpm
    return _hatchling.build_sdist(sdist_directory, config_settings)

# Optional hooks alias verbatim — no extra logic
get_requires_for_build_wheel = _hatchling.get_requires_for_build_wheel
get_requires_for_build_sdist = _hatchling.get_requires_for_build_sdist
get_requires_for_build_editable = _hatchling.get_requires_for_build_editable
prepare_metadata_for_build_wheel = _hatchling.prepare_metadata_for_build_wheel
prepare_metadata_for_build_editable = _hatchling.prepare_metadata_for_build_editable

run_vite_build() short-circuits when web/ is absent (the unpacked-sdist case). The sdist contains the pre-baked static/ tree (not gitignored at sdist-build time because vite ran), so the wheel-from-sdist build skips vite cleanly.

Layer 3b — Sphinx extension (__init__.py)

Consumed via extensions = ["sphinx_vite_builder"] in conf.py. Hooks two events:

• builder-inited (defined at sphinx/sphinx/events.py:115) — fires after the builder is constructed but before any docs are read. Mode resolution:
  • auto (default) — detect autobuild via env/argv/parent-process; dev if autobuild, else prod
  • dev — spawn vite build --watch (long-lived; survives autobuild rebuilds)
  • prod — run vite build once, block until done
  • disabled — no-op (useful when an external orchestration handles vite)
• build-finished (same file:242) — no-op for watch mode (process must survive the rebuild loop); teardown is via atexit + signal handlers

sphinx-autobuild has no extension-facing hook protocol — it spawns sphinx-build as a fresh subprocess per rebuild, so each invocation gets a new Sphinx() app and builder-inited re-fires. The extension's builder-inited handler is idempotent: if a vite watch is already running, return early. Detection of autobuild mode is heuristic (config.py:detect_mode() checks env, argv, parent-process; matches the existing gp-sphinx-vite pattern).

raise sphinx.errors.ConfigError(msg) (defined at sphinx/sphinx/errors.py:81-84) for user-fixable failures (missing pnpm, install error, vite spawn failure). ConfigError halts the build with a multi-line actionable message.

Workspace bootstrap (chicken-and-egg)

• packages/sphinx-vite-builder/pyproject.toml declares build-backend = "hatchling.build" (or flit_core.buildapi) — sphinx-vite-builder is itself a pure-Python package, no vite needed for its own build
• packages/gp-furo-theme/pyproject.toml declares build-backend = "sphinx_vite_builder.build" + backend-path = ["../sphinx-vite-builder"] — pip/uv adds the in-tree directory to sys.path so the backend resolves without PyPI
• After publish to PyPI, consumers can drop backend-path and use requires = ["sphinx-vite-builder>=..."]

Implementation phases

Phase 1 — Backend head + smart subprocess core (PR scope)

1. Scaffold packages/sphinx-vite-builder/ with pyproject.toml (hatchling-built), LICENSE, README.md
2. Port process.py + bus.py from gp-sphinx-vite to sphinx_vite_builder/_internal/. Add os.setsid() for POSIX process group isolation (improvement over current).
3. Implement _internal/vite.py — orchestration module, fast-fail on missing pnpm/vite
4. Implement build.py — PEP 517 backend, ~50 LOC delegating to hatchling
5. Implement _internal/errors.py — PnpmMissingError, ViteFailedError, NodeModulesInstallError
6. Wire gp-furo-theme/pyproject.toml to use the new backend; drop force-include blocks (they're rejected by force-include-requires-disk-existence; the backend handles it)
7. Drop the explicit pnpm exec vite build step from tests.yml docs job and release.yml (the backend handles it transparently when uv sync / uv build runs)
8. Tests: tests/test_sphinx_vite_builder_build.py covers each PEP 517 hook + delegation correctness + missing-pnpm/vite paths
9. Verify: uv sync from clean checkout works; uv build --package gp-furo-theme produces wheel with assets; CI's deferred smoke (gp-sphinx) and smoke (sphinx-gp-theme) failures resolve naturally; no manual vite step needed anywhere

Phase 2 — Extension head (consolidate gp-sphinx-vite)

1. Move gp-sphinx-vite's extension code (hooks.py, config.py, setup() in __init__.py) into sphinx_vite_builder/
2. Add sphinx_vite_builder to consumer conf.py extensions lists; remove gp_sphinx_vite
3. Deprecate or delete packages/gp-sphinx-vite/
4. Update docs/ justfile to drop start-docs / build-docs vite steps if Sphinx extension auto-orchestrates them
5. Tests: extension-side via sphinx.testing.util.SphinxTestApp + @pytest.mark.sphinx('html') (existing pattern in sphinx/tests/test_application.py)

Phase 2 may land as a follow-up PR after Phase 1 ships.

Phase 3 — Publish to PyPI (optional)

When sphinx-vite-builder is stable, publish from the workspace's release pipeline. External Sphinx-theme projects using vite (outside this workspace) can then adopt it via standard [build-system].requires.

Required tests (fast-fail coverage)

For both heads. Each requirement has at least one corresponding test:

Backend head

• ✅ build_wheel delegates to hatchling.build_wheel after running vite (mock hatchling, assert called)
• ✅ build_editable delegates to hatchling.build_editable
• ✅ build_sdist runs vite + delegates
• ✅ run_vite_build() short-circuits when web/ is absent (sdist-from-temp scenario)
• ✅ PnpmMissingError raised with actionable message when shutil.which("pnpm") is None
• ✅ NodeModulesInstallError raised when pnpm install exits non-zero
• ✅ ViteFailedError raised when pnpm exec vite build exits non-zero
• ✅ Wheel built from a clean tree contains the vite-built static/ files
• ✅ Sdist contains pre-baked static/; wheel-from-sdist works without pnpm

Extension head

• ✅ setup(app) registers the two events + config values + returns the metadata dict
• ✅ on_builder_inited is idempotent (re-firing under autobuild rebuild → no second spawn)
• ✅ auto mode: detects autobuild via env / argv / parent-process; falls back to prod otherwise
• ✅ dev mode: spawns vite build --watch; survives build-finished
• ✅ prod mode: runs vite build once; blocks until completion
• ✅ disabled mode: no-op
• ✅ Missing pnpm: raises ConfigError from builder-inited with the workspace+pnpm hint
• ✅ Vite spawn failure: raises ConfigError with the build-context error
• ✅ Teardown: SIGTERM/SIGINT/SIGHUP triggers process cleanup
• ✅ Teardown: atexit triggers process cleanup
• ✅ POSIX: child inherits process group so SIGTERM kills the tree

Smart subprocess core

• ✅ Concurrent stdout/stderr drainers complete without deadlock
• ✅ SIGTERM → SIGKILL escalation fires after configured timeout
• ✅ PYTHONUNBUFFERED=1 injected into subprocess env
• ✅ Rich console handles color-detection fallback (no TTY → plain text)

Research summary (what we read)

The full code-level research lives in agent dossiers; this section indexes the upstream sources cited in the architecture above.

Backend patterns

• maturin (maturin/maturin/__init__.py:158-165) — the get_requires_for_build_wheel auto-install trick for Rust via puccinialin. We don't replicate (pnpm isn't pip-installable), but the env-var bypass + shutil.which() pattern transfers.
• uv (uv/crates/uv-build/python/uv_build/__init__.py:72-141) — uv_build is a thin Python wrapper around a Rust binary, no hook interface, can't inject custom build steps. Verdict: not suitable as a delegation target. Stick with hatchling. Tracked at astral-sh/uv#11502.
• flit_core (flit/flit_core/flit_core/buildapi.py, 85 LOC) — the canonical PEP 517 backend module shape. Pure functions, CWD-relative pyproject.toml, hooks aliased where the implementation is identical.
• hatchling (hatchling/src/hatchling/build.py:17-84) — verified pure-function delegation surface, no import-time state, reads pyproject.toml from CWD with no caller intervention. Wrappable in ~30 lines.
• sphinx-theme-builder (sphinx-theme-builder/, ~2140 LOC) — the closest analog (webpack-aware backend used by Furo, sphinx-book-theme, pydata-sphinx-theme). Rolls own ZIP packing rather than delegating; uses nodeenv for Node isolation. We delegate to hatchling and use system pnpm via corepack — significantly thinner. Bootstrap pattern (its own pyproject.toml uses flit_core.buildapi) directly informs our workspace bootstrap.

Sphinx extension patterns

• Sphinx events (sphinx/sphinx/events.py) — builder-inited (line 115) and build-finished (line 242) are the two events we hook. Lifecycle: config-inited → builder-inited → env-before-read-docs → source-read → doctree-read → env-updated → write-started → doctree-resolved → build-finished.
• sphinx-autobuild (readme + main module) — no extension-facing hook protocol; spawns sphinx-build as a fresh subprocess per rebuild. Extensions detect autobuild heuristically. The existing gp-sphinx-vite/config.py:detect_mode() covers env / argv / parent-process detection — port verbatim.
• Furo (furo/src/furo/__init__.py) — example of theme + extension dual-purpose package; the single-setup() pattern in __init__.py is the cleanest. Furo refuses to load via extensions to avoid double-init; we don't (we ARE an extension).
• myst-parser (myst-parser/myst_parser/sphinx_ext/__init__.py) — minimal entry-point file; full extension logic in a sphinx_ext/ submodule. We follow this if __init__.py grows past ~100 LOC.
• sphinx.errors.ConfigError (sphinx/sphinx/errors.py:81-84) — the right exception class for user-fixable failures. Halts build with category-prefixed multi-line message.
• Test fixtures (sphinx/tests/test_application.py) — sphinx.testing.util.SphinxTestApp + @pytest.mark.sphinx('html', testroot=...) is the canonical pattern.

Subprocess + UI patterns

• gp-sphinx-vite (packages/gp-sphinx-vite/src/gp_sphinx_vite/) — full asyncio bus + ViteProcess + atexit/signal teardown design. Production-grade; absorb verbatim into sphinx_vite_builder/_internal/. Files: process.py (244 LOC), bus.py (194 LOC), hooks.py (274 LOC), config.py (192 LOC).
• In-house async dev-server reference — aiohttp + asyncio.create_subprocess_exec + line-by-line stream draining via async iterator over process.stdout. Reference for async stream consumption without deadlock.
• In-house batch-orchestration reference — rich.live.Live + rich.table.Table for live status updates (refresh-per-second config, status table that updates from a manifest). Pattern for the build pipeline status panel.
• In-house build-orchestration reference — CommandOutput helper class wrapping rich's Console, Panel, Progress, TaskID. Reference for structured CLI output (start/finish messages, multi-step progress bars, panel-styled error summaries).
• In-house event-driven build reference — EventBus pub/sub + async subprocess in a builder subsystem; same CommandOutput reuse. Reference for event-driven build orchestration where BuildRequestedEvent / BuildCompletedEvent / FileChangedEvent decouple the builder from the watcher and server subsystems.
• vite programmatic API (vite/packages/vite/src/node/index.ts:29-39) — exports build(), createServer(), preview(), defineConfig(), resolveConfig(). We shell out via pnpm exec vite for now (Python→Node bridge is fragile); these exports are documented for if/when we ship a Node helper script.
• pnpm --filter — selects a workspace package by name pattern; pnpm-workspace.yaml defines the package set. pnpm --filter @scope/pkg exec <cmd> runs <cmd> in that package's context.
• rich (per rich.readthedocs.io) — minimum-viable surface: Console, Live, Status, Progress, Panel, styled console.print(f"[green]✓[/green] ...").

Specs

• PEP 517 (build system contract) — defines the hook surface (build_wheel, build_sdist, get_requires_for_build_*, prepare_metadata_for_build_*); also defines backend-path for in-tree backends
• PEP 660 (editable installs) — adds build_editable, get_requires_for_build_editable, prepare_metadata_for_build_editable
• PEP 621 (project metadata) — [project] table that hatchling reads
• PEP 508 (dependency specifiers) — what get_requires_for_build_* returns

Out of scope

• Publishing sphinx-vite-builder to PyPI — Phase 1 uses backend-path for workspace consumption only. Phase 3 publishes to PyPI if external interest materializes.
• Asset hashing / source-map handling — current vite.config.ts produces stable filenames (furo-tw.css, furo.js); if hashing is ever introduced the backend will need to emit a manifest.
• Replacing sphinx-theme-builder for upstream Furo / sphinx-book-theme / pydata-sphinx-theme — sphinx-vite-builder complements rather than replaces.
• Auto-installing pnpm — pnpm isn't pip-installable; the failure mode is "user runs corepack enable" not "backend bootstraps a Node env" (that's what sphinx-theme-builder's nodeenv does, deliberately not adopted because corepack is the modern Node convention).
• Calling vite via Node's programmatic API — Phase 1 shells out via pnpm exec vite build. A future Phase could ship a Node helper script that imports 'vite' directly; not needed for the first iteration.

Status

Phase 1 = the new PR this issue tracks. Opens after PR #27 merges. Two-PR sequence:

1. PR Fail loud when vite assets are missing or pnpm bootstrap fails #27 (open, ready for review) — runtime fail-loud diagnostics standalone
2. New PR after Fail loud when vite assets are missing or pnpm bootstrap fails #27 merges — sphinx-vite-builder Phase 1 (backend head + subprocess core); Phase 2 (extension consolidation) and Phase 3 (PyPI publish) follow as separate PRs