Migrate docs to Antora + publish Dokka as sibling /api/ (#494) by michalharakal · Pull Request #495 · SKaiNET-developers/SKaiNET

michalharakal · 2026-04-13T13:49:59Z

Closes #494.

Summary

Migrates mainline SKaiNET's docs to Antora following Divio documentation principles (Tutorials / How-to / Reference / Explanation), adds a GitHub Actions workflow to build and publish the site, integrates the existing Gradle-generated operator fragments plus a new cross-backend coverage matrix into the Antora Reference section, and bundles the Dokka-generated API reference as a sibling `/api/` path of the Antora site. Copies the Docker image verbatim from `SKaiNET-transformers` (node:20-alpine + chromium + mermaid-cli + @antora/cli@3.1 + asciidoctor-kroki@0.18) — same rendering stack on both repos' docs sites.

The six commits

Scaffold (`aef78ef9`) — `antora.yml`, `antora-playbook.yml`, `docs/modules/ROOT/nav.adoc` (minimal), landing page, `docs/.docker/Dockerfile` verbatim from transformers. Local build: exit 0, zero warnings.
Content migration (`18f9c80d`) — 11 `.md` files converted to AsciiDoc via `pandoc/core:latest` in Docker, moved under Divio categories. Pre-existing `.adoc` files git-mv'd to preserve history. Images moved to `docs/modules/ROOT/images/`. Bulk-rewrote 5 `[source,mermaid]` fences to `[mermaid]` for kroki recognition. Escaped 4 literal `xref::` / `include::` prose mentions in `operator-design.adoc` that were tripping the parser. Net 32 files changed.
Repoint ops-docs generator (`b98ad962`) — `documentation.outputDirectory` now points at `docs/modules/ROOT/pages/reference/operators/generated/`. Renamed from `generated` to avoid AsciiDoc's italic-formatting parser mangling the path in xrefs. New `deriveAntoraXrefPrefix()` helper in `GenerateDocumentationTask.kt` auto-derives the module-relative xref prefix from the configured output directory so bare-filename xrefs in the generated `index.adoc` resolve correctly under Antora.
Ops status matrix emitter (`4e75aec8`) — new `emitOpsStatusMatrix()` method in `GenerateDocumentationTask` writes a single-page operator × backend matrix to `reference/ops-status-matrix.adoc` with rows per `Operator.function` pair, columns per backend, emoji cells (✅ ⚠️ ❌ ⏳ 🚧 —), and a "Done N/total" footer per backend. Widened `shortStatus()` to cover the real KSP vocabulary (`implemented`, `in_progress`, `missing`) alongside the planning vocabulary.
`docs.yml` workflow (`4754ef93`) — new `.github/workflows/docs.yml` with JDK 25 pre-step, Gradle cache, `generateDocs` + `dokkaGenerate` pre-steps, `docker/setup-buildx-action@v3` + `docker/build-push-action@v5` with `cache-from/to: type=gha` (transformers' workflow skipped caching; this one improves on that), Antora run in the container, `actions/upload-pages-artifact@v3` + `actions/deploy-pages@v4` gated on `refs/heads/develop`. `dokka-pages.yml` deleted — one workflow, one Pages source.
Dokka bundle wiring (`30488cd2`) — registers a `bundleDokkaIntoSite` Copy task in the root `build.gradle.kts` that copies `build/dokka/html/` into `docs/build/site/api/`. Intentionally NOT wired into the `build` lifecycle so local `./gradlew build` never silently creates a half-populated site directory. The workflow invokes it only AFTER the Antora container finishes, avoiding the ordering trap where Gradle pre-creates the target directory and a later Antora run wipes it. The `reference/api.adoc` stub's `link:../api/index.html[]` now resolves.

End-to-end local verification

```bash
./gradlew --no-daemon generateDocs dokkaGenerate
docker build -t skainet-antora:local docs/.docker
rm -rf docs/build/site
docker run --rm -v "$PWD:/antora" -w /antora \
skainet-antora:local docs/antora-playbook.yml
./gradlew --no-daemon bundleDokkaIntoSite
python3 -m http.server --directory docs/build/site 8080
```

Click-check passed at http://localhost:8080/:

Landing page renders with SKaiNET logo, four Divio section intros, Diátaxis explainer
Nav shows Tutorials / How-to / Reference / Explanation with all 22 pages
Operator reference (auto-generated from KSP) renders with backend status tables
Operator coverage matrix renders with live emoji status and "Done 1/3" totals per backend
`reference/api.adoc` link opens the Dokka aggregate at `/api/index.html`
`/api/` subtree has full Dokka navigation for every module (`skainet-lang/`, `skainet-compile/`, `skainet-backends/`, `skainet-io/`, `skainet-data/`)

Known cleanup debt (non-blocking warnings)

The Antora build exits clean (zero errors) but produces 13 warnings that are all pre-existing content issues uncovered by the migration:

Missing attribute references in `operator-design.adoc` (`{framework_name}`, `{ksp_version}`, `{dokka_version}`, `{asciidoctorj_version}`) — the original author declared them in an attributes block that got lost in a prior edit. Fix: define the four attributes in `docs/antora.yml` or at the top of the page.
Section level out-of-sequence in `skainet-for-ai.adoc` (5 occurrences) and `arduino-c-codegen.adoc` (1) — pandoc's markdown → asciidoc conversion skipped heading levels where the source did. Fix: re-level each offending section header.
One kroki mermaid 400 on a large diagram in `hlo-getting-started.adoc` — the encoded diagram payload hits kroki.io and comes back with a 400 (URL length limit, likely). Transformers' docs render mermaid via the same stack and work; this one specific diagram is too big. Fix: simplify the diagram or switch to the local mermaid-cli path via asciidoctor-kroki's `kroki-fetch-diagram: false` mode.

All of these should land as a follow-up cleanup PR ("Antora migration polish") — none of them block the site from building and deploying.

Post-merge actions

`workflow_dispatch` the new `docs.yml` workflow immediately after merge to avoid a ~1 minute GitHub Pages 404 window between `dokka-pages.yml` disappearing and `docs.yml`'s first run publishing.
Confirm the first deploy artifact contains both `index.html` and `api/index.html` at the root of the `github-pages` tarball.
Open a follow-up issue for the 13 cleanup warnings.

🤖 Generated with Claude Code

Adds the minimum Antora config needed to produce a one-page static site: - docs/antora-playbook.yml — site playbook, asciidoctor-kroki extension, default Antora UI bundle, output at docs/build/site - docs/antora.yml — component `skainet`, version ~ (HEAD) - docs/modules/ROOT/nav.adoc — minimal Divio navigation scaffold (sections empty; pages land in commit 2) - docs/modules/ROOT/pages/index.adoc — landing page with Diataxis section intros and an admonition pointing LLM readers at the SKaiNET-transformers sibling repo - docs/.docker/Dockerfile — verbatim copy from SKaiNET-transformers: node:20-alpine + Chromium + font-noto + @antora/cli@3.1 + @antora/site-generator@3.1 + asciidoctor-kroki@0.18 + @mermaid-js/mermaid-cli@11. Built locally as skainet-antora:local, never pushed to any registry. Verified locally: docker build -t skainet-antora:local docs/.docker docker run --rm -v "$PWD:/antora" -w /antora \ skainet-antora:local --stacktrace docs/antora-playbook.yml Exit 0, zero warnings, produces docs/build/site/skainet/index.html. The navigation sections are intentionally stubbed for this commit — content migration to the Divio layout (Tutorials / How-to / Reference / Explanation) is commit 2. Dokka bundling into a sibling /api/ path is commit 6. First step of the six-commit docs-to-Antora migration plan. See issue #494. Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>

Moves every existing doc file under `docs/` into the Antora `modules/ROOT/pages/` tree, organized by the four Diataxis categories: Tutorials/ — getting-started and DSL learning paths How-to/ — task-oriented recipes (build, cli, I/O, training, Arduino) Reference/ — architecture, api stub pointing at Dokka Explanation/ — design articles, theory, examples, perf, known issues Markdown sources were converted to AsciiDoc via pandoc 3.9 (run through the `pandoc/core` Docker image for determinism). Pre-existing `.adoc` files were git-mv'd to preserve history. Image assets (SKaiNET-compiler.svg, SKaiNET-logo.png) moved to `docs/modules/ROOT/images/` and `image::` references adjusted to Antora-relative paths. Two structural cleanups handled in the same commit: - `[source,mermaid]` fences bulk-rewritten to `[mermaid]` so asciidoctor-kroki recognizes them. 5 diagrams touched across hlo-getting-started.adoc, arduino-c-codegen.adoc, and operator-design.adoc. - Literal `xref::` and `include::` mentions in the prose of operator-design.adoc escaped to backticks so Antora stops parsing them as real xrefs. This was the one build error after the initial pandoc pass. Root `ARCHITECTURE.md` was a 4-line stub — its content moved to `reference/architecture.adoc` with a cleaner title and a pointer comment. The root file now contains a 3-line link to the published page under the new site URL. Nav populated with all four Divio sections. A new `reference/api.adoc` page stubs the link to the Dokka-generated API (which will be bundled at publish time by commit 6). Local build verified: docker run --rm -v "$PWD:/antora" -w /antora \ skainet-antora:local docs/antora-playbook.yml Zero errors. 13 non-blocking warnings remain as cleanup debt: pandoc section-level artifacts in skainet-for-ai.adoc and arduino-c-codegen.adoc, missing attribute references (framework_name, ksp_version, dokka_version, asciidoctorj_version) in operator-design.adoc from the original author, and one kroki mermaid 400 from kroki.io on a large diagram. All tracked for a follow-up cleanup pass. Second step of the six-commit docs-to-Antora migration plan. See issue #494. Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>

Three coupled changes that move the custom `GenerateDocumentationTask` output from the pre-staged `docs/modules/operators/_generated_/` tree into the Antora `modules/ROOT/pages/reference/operators/generated/` tree so the generated operator fragments actually surface on the published site. 1. Root `build.gradle.kts` `documentation { }` block: outputDirectory now points at `docs/modules/ROOT/pages/reference/operators/generated`. Dropped the surrounding underscores from `_generated_` to avoid AsciiDoc's italic-formatting parser mangling the path in xrefs (the Antora build errored with `target of xref not found: reference/operators/<em>generated</em>/index.adoc` until the rename). The `_generated_` convention was pre-staged by the maintainers but never validated through a real Antora build. 2. `GenerateDocumentationTask.kt` — `generateMainIndex` now emits Antora-compatible xrefs. A new private `deriveAntoraXrefPrefix()` helper inspects the output directory's absolute path, looks for a `/pages/` segment, and returns everything below it as the xref prefix (e.g. `reference/operators/generated/`). When the output directory is NOT under an Antora module (any flat layout), the prefix is an empty string and the generator emits bare-filename xrefs, preserving the pre-Antora behavior for any callers still using it that way. 3. `docs/modules/ROOT/nav.adoc` now has a Reference entry `xref:reference/operators/generated/index.adoc[Operator coverage]`. Old `docs/modules/operators/_generated_/` tree deleted; the per-operator `.adoc` files tracked as git-mv renames into the new location (history preserved). Local build verified: `./gradlew generateDocs` populates the new location cleanly, Antora renders it with zero errors and the same 13 pre-existing warnings from the previous commit. Third step of the six-commit docs-to-Antora migration plan. See issue #494. Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>

Extends GenerateDocumentationTask with an emitOpsStatusMatrix pass that writes a single-page operator x backend matrix to docs/modules/ROOT/pages/reference/ops-status-matrix.adoc — sibling of the per-operator pages, one click away from the operator index. Rows are `Operator.function` pairs; columns are every backend that appears in any function's statusByBackend map, sorted for column-order stability across runs. Cells carry emoji: ✅ supported / implemented / done ⚠️ partial ❌ not_supported / missing / unsupported ⏳ planned 🚧 in_progress / wip — no claim for this (function, backend) pair A totals footer row shows "Done = N / total" per backend — counts any status rendered with ✅. A missing entry is "unknown" rather than "not supported", and the preamble text makes that explicit so readers don't over-interpret blanks. The matrix output path is derived from the task's outputDirectory via .parentFile.parentFile so it lands at reference/ops-status-matrix.adoc under the Antora layout (the per-operator pages live at reference/operators/generated/). Falls back to writing next to outputDir when the parents are missing, preserving the task's behavior for non-Antora callers. Also widens shortStatus() to cover the implementation-style vocabulary the KSP processor actually emits today (`implemented`, `in_progress`, `missing`) alongside the planning-style vocabulary (`supported`, `partial`, `planned`, `not_supported`). Unknown values fall back to the raw string so the matrix never silently hides a status the generator didn't anticipate — strictly more permissive than the existing `formatStatus` used by the per-function table. Nav entry added under Reference: xref:reference/ops-status-matrix.adoc[Operator coverage matrix] Local build verified: matrix file generates with real KSP data, renders through Antora with zero errors and the same 13 pre-existing warnings. Fourth step of the six-commit docs-to-Antora migration plan. See issue #494. Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>

New .github/workflows/docs.yml that builds the Antora site on every PR and publishes it to GitHub Pages on pushes to develop. Mirrors the SKaiNET-transformers workflow layout but adds several SKaiNET-specific pre-steps and a buildx cache. Workflow steps (build-docs job): 1. Checkout. 2. actions/setup-java@v4 — JDK 25 on the runner, matching every other workflow in this repo. Runs outside the Antora container so the Gradle wrapper cache is reusable and generateDocs / dokkaGenerate see the right JDK. 3. Cache Gradle caches + wrapper for subsequent runs. 4. ./gradlew generateDocs dokkaGenerate — emits the KSP operator fragments + the ops-status-matrix.adoc into docs/modules/ROOT/pages/reference/operators/ AND builds the full Dokka HTML aggregate. Running dokka here (not just in commit 6) means commit 6 is a pure workflow-step + task-registration change with no extra Gradle cost. 5. docker/setup-buildx-action + docker/build-push-action with gha cache. The Chromium layer makes the image ~400 MB so first build is 3–5 minutes; subsequent runs are sub-minute via cache-from/to. Transformers skipped caching here; this workflow improves on that. 6. Docker run Antora inside the container, volume-mounting the workspace, workdir /antora/docs, entrypoint is the image's baked-in antora binary. 7. actions/upload-pages-artifact@v3 with path docs/build/site. Deploy job (deploy-docs) is gated on push to develop and consumes the uploaded artifact via actions/deploy-pages@v4. Triggers fire on paths that actually affect the docs site: everything under docs/, the workflow itself, the root build.gradle.kts, build-logic/, and skainet-lang-core (the KSP source the operator generator reads). .github/workflows/dokka-pages.yml is deleted. Only one workflow can own GitHub Pages and the new one publishes both Antora and (after commit 6) the bundled Dokka output. Deleting it in the same commit as enabling docs.yml creates a brief ~1 minute 404 window between the merge and the first new-workflow run; trigger a workflow_dispatch immediately post-merge to close it. bundleDokkaIntoSite is NOT invoked yet — that Gradle task does not exist until commit 6. Commit 5 can therefore be tested end-to-end (Antora builds, pages deploy) without Dokka being visible under /api/. Commit 6 adds the task and the one-line workflow edit to wire it in. Yaml linted with `python3 -c 'import yaml; yaml.safe_load(...)'`. Fifth step of the six-commit docs-to-Antora migration plan. See issue #494. Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>

Two coupled changes that activate the sibling-path Dokka bundle under /api/ of the published site. 1. Root build.gradle.kts registers a new `bundleDokkaIntoSite` Copy task under the `documentation` group: tasks.register<Copy>("bundleDokkaIntoSite") { dependsOn("dokkaGenerate") from(layout.buildDirectory.dir("dokka/html")) into(layout.projectDirectory.dir("docs/build/site/api")) } The task is INTENTIONALLY NOT wired into the `build` lifecycle. It's invoked only by the docs workflow, and only AFTER Antora has populated docs/build/site/. Pre-creating the target directory from a Gradle build would cause the later Antora run to wipe the api/ subtree on its fresh write; keeping the task workflow-invoked only avoids that ordering trap. 2. .github/workflows/docs.yml gets a new step between "Build Antora site" and "Upload artifact": - name: Bundle Dokka API into site run: ./gradlew --no-daemon bundleDokkaIntoSite Dokka itself is already generated upstream in the existing "Generate operator docs and Dokka" step from commit 5, so this is a pure Copy — no re-run cost. Net effect: the published site contains a full Antora site at the root and the full Dokka multi-module API aggregate under /api/. The reference/api.adoc page's `link:../api/index.html[API Reference]` stub from commit 2 resolves to a real page. Same origin, no CORS, served from the same GitHub Pages site as the Antora content. End-to-end local verification: rm -rf docs/build/site docker run --rm -v "$PWD:/antora" -w /antora \ skainet-antora:local docs/antora-playbook.yml ./gradlew --no-daemon bundleDokkaIntoSite ls docs/build/site/ # index.html, skainet/, api/ ls docs/build/site/api/ # index.html, skainet-lang/, # skainet-compile/, skainet-backends/, # skainet-io/, skainet-data/, ... Sixth and final step of the docs-to-Antora migration plan. See issue #494. Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>

github-actions · 2026-04-13T13:53:22Z

📖 Documentation Preview

The documentation has been built successfully for this PR.

Generated Files:

Operator documentation: docs/modules/operators/_generated_/
JSON schema output: operators.json

Artifacts:

Download the documentation-preview-495 artifact to view the complete documentation locally.

This comment will be updated automatically when the PR is updated.

Two coupled changes that swap mainline's Antora UI bundle for the new SKaiNET-branded skainet-docs-ui release and rewrite the landing page to use its card-grid component. ## docs/antora-playbook.yml Pin ui.bundle.url to: https://github.com/SKaiNET-developers/skainet-docs-ui/releases/download/v1.1.1/ui-bundle.zip Replaces the upstream gitlab.com/antora/antora-ui-default URL that mainline had been pinned to since the Antora migration in #494/#495. The new bundle is the shared SKaiNET docs UI from the new skainet-docs-ui repo (https://github.com/SKaiNET-developers/skainet-docs-ui) which ships: - SKaiNET red palette (#dc2626) with light + dark mode - Orbitron (display) + Inter (body) + Fira Code (mono) - Class-based dark mode toggle (FOUC-safe via inline head bootstrap) - 6px thin custom scrollbar - Rounded code blocks with GitHub-light / GitHub-dark - Primary-red focus glow on links / buttons - SKaiNET footer (replaces the upstream MPL-2.0 / antora-ui boilerplate) - .card-grid CSS for landing-page card grids - Mobile responsive (hamburger menu from upstream preserved) Pinned to v1.1.1 specifically (not floating on latest) so mainline builds are reproducible. Bumping the pin is a one-line follow-up PR whenever skainet-docs-ui ships a new minor. ## docs/modules/ROOT/pages/index.adoc Replaces the placeholder "Quick links" section with a real [.card-grid] block that surfaces the four Diátaxis sections (Tutorials / How-to / Reference / Explanation) as clickable cards with one-line descriptions. Each card links to the first real page of its section: Tutorials -> tutorials/java-getting-started.adoc How-to -> how-to/build.adoc Reference -> reference/architecture.adoc (with inline xrefs to operator catalog + ops-status-matrix + Dokka API) Explanation -> explanation/skainet-for-ai.adoc The [.card-grid] role is rendered into a responsive grid by the new CSS in skainet-docs-ui's src/css/skainet-polish.css (landed in v1.1.0). Falls back to a normal bulleted list on any consumer that doesn't ship the SKaiNET CSS — never a broken render. The Diátaxis NOTE admonition pointing at SKaiNET-transformers for LLM runtimes stays — that was added in the original migration to make the repo split visible. ## Verification Local Antora build against the published v1.1.1 bundle URL (no --ui-bundle-url override): docker run --rm --user "$(id -u):$(id -g)" \ -v "$PWD:/antora" -w /antora \ skainet-antora:local docs/antora-playbook.yml - exit 0 - rendered index.html contains <div class="ulist card-grid"> with the four cards - rendered footer contains the SKaiNET-attribution links to skainet-docs-ui, mainline, and SKaiNET-transformers - all other pages (tutorials, how-to, reference, explanation, operator catalog, ops-status-matrix) render unchanged but in the new red-on-dark/light theme Closes #501. Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>

michalharakal and others added 6 commits April 13, 2026 15:11

michalharakal merged commit 51285d6 into develop Apr 13, 2026
5 of 6 checks passed

michalharakal deleted the feature/494-antora-migration branch April 13, 2026 13:50

This was referenced Apr 13, 2026

Fact-check and correct migrated Antora docs #499

Closed

Fact-check and correct migrated Antora docs (#499) #500

Merged

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Migrate docs to Antora + publish Dokka as sibling /api/ (#494)#495

Migrate docs to Antora + publish Dokka as sibling /api/ (#494)#495
michalharakal merged 6 commits intodevelopfrom
feature/494-antora-migration

michalharakal commented Apr 13, 2026

Uh oh!

Uh oh!

github-actions Bot commented Apr 13, 2026

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

1 participant

Conversation

michalharakal commented Apr 13, 2026

Summary

The six commits

End-to-end local verification

Known cleanup debt (non-blocking warnings)

Post-merge actions

Uh oh!

Uh oh!

github-actions Bot commented Apr 13, 2026

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

1 participant