Skip to content

Unify docs-website release pipeline: new assets + retire dispatch chain#4982

Merged
rdimitrov merged 3 commits intomainfrom
ship-crd-manifests-as-release-asset
Apr 21, 2026
Merged

Unify docs-website release pipeline: new assets + retire dispatch chain#4982
rdimitrov merged 3 commits intomainfrom
ship-crd-manifests-as-release-asset

Conversation

@rdimitrov
Copy link
Copy Markdown
Member

@rdimitrov rdimitrov commented Apr 21, 2026

Summary

Three related release-workflow improvements bundled into one PR, all in service of simplifying the stacklok/docs-website release-doc regeneration pipeline:

  1. Bundle CRD manifests as thv-crds.tar.gz — lets consumers skip cloning this repo just to read the 13 CRD YAMLs
  2. Re-export toolhive-core schemas as release assets — makes the go.mod-pinned schemas available without a second-repo download
  3. Retire the repository_dispatch chain to docs-website — no longer needed once docs-website adopts the new release assets

Why bundle these together

They're the same thing from different angles: "ship more of what docs-website needs as release assets, then stop doing the fragile push-based thing." Merging them together means one review surface, and the retirement of the dispatch chain lines up with the introduction of its replacement.

Context: docs-website PR #748 migrates the docs repo to a Renovate-driven pipeline that watches version: pins in a tracked YAML and regenerates reference docs from release assets. The pipeline works today via a raw repo clone; landing this PR lets it switch to the cleaner asset-based flow.

Commits

1. Bundle CRD manifests as a release asset

Adds build/thv-crds.tar.gz (the 13 CRD YAMLs from deploy/charts/operator-crds/files/crds/) as a new release asset via goreleaser's extra_files. ~94KB tarball, flat layout.

Eliminates the need for consumers to git clone --depth 1 --branch <tag> https://github.com/stacklok/toolhive.git just to read the CRDs.

2. Re-export toolhive-core schemas as release assets

Reads github.com/stacklok/toolhive-core from go.mod at release time, downloads the four schema JSONs from that core release, and uploads them as toolhive's own release assets:

  • toolhive-legacy-registry.schema.json
  • upstream-registry.schema.json
  • publisher-provided.schema.json
  • skill.schema.json

Eliminates the "read go.mod → download from a second repo" dance consumers currently do.

3. Retire the docs-website repository_dispatch chain

Deletes .github/workflows/update-docs-website.yml and the update-docs-website + extract-release-actor jobs in releaser.yml (the latter only fed the dispatch's assign_to input). docs-website no longer listens for repository_dispatch: published-release once #748 merges — so the dispatch is becoming dead code.

Follow-up for maintainers: the DOCS_REPO_DISPATCH_TOKEN repo secret can be retired after this lands — nothing references it.

Test plan

  • Verify thv-crds.tar.gz appears as a release asset with 13 CRD YAMLs
  • Verify the four schema JSONs appear as release assets
  • Verify the go.mod-derived toolhive-core version matches what the binary was compiled against
  • Verify update-docs-website does NOT appear as a job in the release workflow (both in GitHub Actions UI and as a downstream dispatch)
  • Confirm existing release assets (CLI docs, swagger, operator-crd-api.md) are unaffected

Interop with docs-website

If this PR lands before #748:

If this PR lands after #748:

  • docs-website's old receiver is already gone; the dispatch chain is firing into the void. Deleting it here is pure cleanup.

Either ordering is safe.

Notes

A fourth upstream ask (ship pre-generated CRD JSON schemas, which would have eliminated docs-website's extract-crd-schemas.mjs) was explored and dropped because it would have required adding a Node toolchain to this pure-Go repo. That transformation stays on the docs-website side.

🤖 Generated with Claude Code

Adds a `thv-crds.tar.gz` tarball to each release containing the CRD
YAML manifests from `deploy/charts/operator-crds/files/crds/`.

Motivation: downstream consumers of the CRDs (notably
stacklok/docs-website, which generates per-CRD reference pages)
currently have to clone the entire toolhive repo at each release tag
just to read these 13 manifests. Shipping them as a ~94KB tarball
asset lets those consumers skip the clone and just `gh release
download` like they already do for `thv-cli-docs.tar.gz` and
`swagger.yaml`.

Purely additive — no changes to existing release assets or workflows.

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
@rdimitrov rdimitrov requested a review from JAORMX as a code owner April 21, 2026 17:25
@github-actions github-actions Bot added the size/XS Extra small PR: < 100 lines changed label Apr 21, 2026
ChrisJBurns
ChrisJBurns previously approved these changes Apr 21, 2026
@codecov
Copy link
Copy Markdown

codecov Bot commented Apr 21, 2026

Codecov Report

✅ All modified and coverable lines are covered by tests.
✅ Project coverage is 69.66%. Comparing base (63ee62a) to head (a85b118).
⚠️ Report is 1 commits behind head on main.

Additional details and impacted files
@@            Coverage Diff             @@
##             main    #4982      +/-   ##
==========================================
+ Coverage   69.63%   69.66%   +0.02%     
==========================================
  Files         552      552              
  Lines       55951    55951              
==========================================
+ Hits        38962    38977      +15     
+ Misses      13991    13975      -16     
- Partials     2998     2999       +1     

☔ View full report in Codecov by Sentry.
📢 Have feedback on the report? Share it here.

🚀 New features to boost your workflow:
  • ❄️ Test Analytics: Detect flaky tests, report on failures, and find test suite problems.

Reads the toolhive-core version from go.mod at release time, downloads
the four JSON schema files from that version of stacklok/toolhive-core,
and ships them alongside toolhive's own release assets.

Motivation: downstream consumers (notably stacklok/docs-website)
currently have to replicate this logic: read go.mod, derive the core
version, then fetch from a different repo's release. Re-exporting the
schemas here makes toolhive's release self-contained — one `gh release
download` call gets everything.

Paired with #4982 (CRD
manifests as release asset); together these eliminate the need for
docs-website to clone toolhive or hit a second repo during its
release-doc regeneration.

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
@github-actions github-actions Bot removed the size/XS Extra small PR: < 100 lines changed label Apr 21, 2026
@rdimitrov rdimitrov changed the title Bundle CRD manifests as a release asset Ship CRD manifests + re-export toolhive-core schemas as release assets Apr 21, 2026
@github-actions github-actions Bot added size/XS Extra small PR: < 100 lines changed and removed size/XS Extra small PR: < 100 lines changed labels Apr 21, 2026
docs-website no longer consumes `repository_dispatch: published-release`
events — its reference-doc regeneration now runs via a Renovate-driven
pipeline that reads the new `thv-crds.tar.gz` and re-exported core
schemas introduced earlier in this PR. See
stacklok/docs-website#748 for context.

Changes:

- Delete `.github/workflows/update-docs-website.yml` (the dispatch
  sender, called via workflow_call from `releaser.yml`).
- Remove the `update-docs-website` job from `releaser.yml` along with
  its `extract-release-actor` dependency, which only existed to feed
  the dispatch's `assign_to` input.
- Update `notify-release-failure`'s `needs:` list to drop the removed
  jobs.

Follow-up for maintainers: the `DOCS_REPO_DISPATCH_TOKEN` repo secret
can be retired — nothing references it after this change.

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
@github-actions github-actions Bot added size/S Small PR: 100-299 lines changed and removed size/XS Extra small PR: < 100 lines changed labels Apr 21, 2026
@rdimitrov rdimitrov changed the title Ship CRD manifests + re-export toolhive-core schemas as release assets Unify docs-website release pipeline: new assets + retire dispatch chain Apr 21, 2026
@github-actions github-actions Bot added size/S Small PR: 100-299 lines changed and removed size/S Small PR: 100-299 lines changed labels Apr 21, 2026
rdimitrov added a commit to stacklok/docs-website that referenced this pull request Apr 21, 2026
Assumes stacklok/toolhive#4982 has landed. That PR ships three new
release assets on toolhive:
  - thv-crds.tar.gz (the 13 CRD YAMLs)
  - thv-cli-docs.tar.gz (already existed)
  - toolhive-*.schema.json (re-exported from toolhive-core at the
    go.mod-pinned version)

And retires the repository_dispatch chain that previously triggered
our reference-regen workflow.

On our side, the shell wrapper script collapses entirely:

- .github/upstream-projects.yaml: toolhive gains five new release_asset
  entries (swagger, CLI docs tarball with extract, and the four core
  schemas). sync-assets.mjs handles all of them declaratively.
- .github/workflows/upstream-release-docs.yml: the old "run the shell
  script" step becomes an inline "download + extract CRD tarball,
  run our three Node helpers" step. Runs only for project_id=toolhive.
- A new "Commit regenerated reference assets" step lands the regen as
  its own commit, so the autogen-detect step below sees only skill-
  introduced touches (prevents false-positive warnings).
- scripts/update-toolhive-reference.sh deleted.

Net effect after #4982 merges:
  - Zero repo clones for CRD processing (tarball replaces clone)
  - Zero cross-repo downloads (toolhive re-exports core schemas)
  - Zero shell scripts for release-doc regen
  - Three Node helpers stay (extract-crd-schemas, generate-crd-pages,
    bundle-upstream-schema) — they produce docs-specific outputs and
    genuinely belong on our side.

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
@rdimitrov rdimitrov merged commit 7ee0772 into main Apr 21, 2026
30 of 31 checks passed
@rdimitrov rdimitrov deleted the ship-crd-manifests-as-release-asset branch April 21, 2026 18:02
rdimitrov added a commit to stacklok/docs-website that referenced this pull request Apr 21, 2026
* Automate upstream release docs PRs via Renovate

Adds a Renovate custom manager that watches version: pins in
.github/upstream-projects.yaml for the four upstream ToolHive projects.
When an upstream ships, Renovate opens a version-bump PR; a new
workflow reacts to push source-verified content edits produced by the
upstream-release-docs skill (three passes with docs-review), assigns
reviewers from non-bot release contributors, and augments the PR body
in a marker-delimited section.

- .github/upstream-projects.yaml: source of truth for tracked projects
- renovate.json: customManagers + packageRule (ignoreUnstable,
  rebaseWhen: never, recreateWhen: never, labels)
- .github/workflows/upstream-release-docs.yml: pull_request + dispatch
- scripts/upstream-release/detect-change.mjs: finds the changed project
  and asserts repo: hasn't been tampered with
- scripts/upstream-release/apply-pin-files.mjs: rewrites @latest pins
- docusaurus.config.ts: pins Registry Server Swagger at v1.2.1

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>

* Add manual bootstrap mode to upstream-release-docs workflow

workflow_dispatch now accepts either `pr_number` (retry an existing
Renovate PR) or `project_id` + `new_tag` (bootstrap a fresh PR without
waiting for Renovate). The bootstrap path branches off main, bumps
the YAML via scripts/upstream-release/bump-yaml.mjs, creates the PR
with the same labels Renovate applies, and feeds its number into the
rest of the workflow so content augmentation runs identically.

Useful for debugging, manual updates, and validating the pipeline
end-to-end right after merge without waiting for an upstream release.

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>

* Unify toolhive reference and content docs into a single PR

update-toolhive-reference.yml now also bumps the toolhive entry in
.github/upstream-projects.yaml, applies pin_files substitutions, and
applies the upstream-content label alongside autogen-docs. The PR it
opens is then picked up by upstream-release-docs.yml (via its
relaxed gate that accepts github-actions[bot] authors) which adds
skill-generated content in a second commit.

Renovate is disabled for stacklok/toolhive so it doesn't race the
reference workflow. The other three tracked projects keep their
Renovate-driven path unchanged.

Net: one PR per stacklok/toolhive release instead of two, with the
same review surface shape as Renovate-driven PRs for the other
projects.

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>

* Address Copilot review feedback on PR #748

- Switch upstream-release scripts and the inline node snippet in the
  workflow from `js-yaml` to the already-declared `yaml` package, so
  the workflow does not rely on a transitive dependency that could
  disappear in a future install. Rewrites the `.load()` calls to
  `.parse()` to match the new package's API.
- Filter out null author logins in the `gh api compare` reviewers
  extraction (`.commits[].author.login? // empty`) so a commit with an
  unlinked GitHub user cannot pass an empty value into --add-reviewer.
- Replace the nested `$([ ... ] && ... || ...)` has_gaps expression
  with an explicit `if/else` block so the quoting stays obviously
  correct and resists accidental breakage in future edits.

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>

* Chain reference workflow into content via workflow_call

The unified toolhive path had a broken seam: peter-evans opens PRs
with the default GITHUB_TOKEN, and GitHub intentionally does not
trigger pull_request workflows on those PRs (recursion guard).
Verified on PR #747 - no on-pr.yaml run fired. So the content
workflow's pull_request trigger would never fire for toolhive PRs.

Fix: add workflow_call trigger to upstream-release-docs.yml and call
it as a dependent job from update-toolhive-reference.yml after
peter-evans completes. The Renovate path is unaffected - Renovate is
a GitHub App identity whose PRs DO trigger workflows.

Also:
- Distinct concurrency groups per workflow
  (upstream-release-docs-reference vs -content) so the workflow_call
  from reference to content doesn't deadlock on a shared group.
- Drop the "What will be added next" section from the peter-evans
  body; the content workflow appends its own section below.
- Fix leading-whitespace leak in the bootstrap PR body by using
  a heredoc + sed strip instead of an inline quoted string.

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>

* Host Registry Server Swagger locally instead of via jsdelivr

Per review feedback from Dan: pull the Swagger file from the upstream
repo at the pinned tag into static/api-specs/ and reference the local
file in docusaurus.config.ts, matching how ToolHive's own Swagger is
handled. Keeps the docs build offline-friendly and makes the pin
implicit in git history rather than a CDN URL template.

- New scripts/upstream-release/sync-assets.mjs helper copies declared
  files from the shallow clone into the docs repo. No-op when a
  project declares no assets.
- .github/upstream-projects.yaml gains an `assets:` field per project.
- Replaced the `pin_files` entry for registry-server with assets.
- static/api-specs/toolhive-registry-api.yaml seeded from upstream
  v1.2.1 (1971 lines). Regenerated automatically on every bump.
- Content workflow runs sync-assets right after the shallow clone.

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>

* Unify toolhive under the Renovate-driven content pipeline

Collapses update-toolhive-reference.yml into upstream-release-docs.yml
so all four tracked projects share one trigger path, one workflow, one
PR per release. Renovate now watches toolhive's version: the same as
the others; the content workflow runs a conditional reference-regen
step for toolhive only.

Also generalizes the `assets:` schema to support three source types:
  - source: <path in upstream repo>           (file-in-clone copy)
  - release_asset: <asset name>               (gh release download)
  - release_asset: <asset name>, extract: tar-gz  (download + extract)

toolhive's swagger and CLI-docs tarball move from the shell script
into declarative `assets:` entries. The shell script shrinks to just
the CRD MDX generation + toolhive-core schema download (the parts
that need custom transforms). It reuses the shallow clone the content
workflow already makes.

Changes:
- Delete .github/workflows/update-toolhive-reference.yml
- Remove workflow_call + dual concurrency groups from the content
  workflow (no chain seam needed without a second workflow)
- Remove the Renovate "disable for stacklok/toolhive" packageRule
- Extend sync-assets.mjs with release_asset + extract support
- Slim update-toolhive-reference.sh; accept TOOLHIVE_CLONE_DIR to
  reuse the workflow's shallow clone

Trade-off: toolhive reference docs now inherit Renovate's 24h
minimumReleaseAge. Previously they published within minutes of a
release. Acceptable for content coherence; set minimumReleaseAge: 0
per-dep if sub-hour latency becomes needed.

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>

* Consume toolhive's new release assets, delete the shell script

Assumes stacklok/toolhive#4982 has landed. That PR ships three new
release assets on toolhive:
  - thv-crds.tar.gz (the 13 CRD YAMLs)
  - thv-cli-docs.tar.gz (already existed)
  - toolhive-*.schema.json (re-exported from toolhive-core at the
    go.mod-pinned version)

And retires the repository_dispatch chain that previously triggered
our reference-regen workflow.

On our side, the shell wrapper script collapses entirely:

- .github/upstream-projects.yaml: toolhive gains five new release_asset
  entries (swagger, CLI docs tarball with extract, and the four core
  schemas). sync-assets.mjs handles all of them declaratively.
- .github/workflows/upstream-release-docs.yml: the old "run the shell
  script" step becomes an inline "download + extract CRD tarball,
  run our three Node helpers" step. Runs only for project_id=toolhive.
- A new "Commit regenerated reference assets" step lands the regen as
  its own commit, so the autogen-detect step below sees only skill-
  introduced touches (prevents false-positive warnings).
- scripts/update-toolhive-reference.sh deleted.

Net effect after #4982 merges:
  - Zero repo clones for CRD processing (tarball replaces clone)
  - Zero cross-repo downloads (toolhive re-exports core schemas)
  - Zero shell scripts for release-doc regen
  - Three Node helpers stay (extract-crd-schemas, generate-crd-pages,
    bundle-upstream-schema) — they produce docs-specific outputs and
    genuinely belong on our side.

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>

* Refine reference-asset language and attribute gaps to PR authors

Two improvements from Dan Barr's review of PR #748:

1. Terminology: "regenerate" was overloaded. It now reads accurately:
   - "sync" for release-asset downloads / file copies (swagger, CLI
     tarball, 4 core schemas) — nothing is actually recreated
   - "regenerate" reserved for the toolhive CRD MDX step which truly
     transforms upstream manifests into our opinionated layout
   - "refresh" for the combined commit that may do both
   The commit that lands reference updates is now titled "Refresh
   reference assets for <project> <tag>".

2. Gap attribution: the skill's GAPS.md format now includes the PR
   number and @-mentions the PR author (non-bots only). Previously a
   gap routed to "everyone in the reviewer pool" via a label; now
   each gap directly pings the engineer who built the feature.

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>

* Derive bootstrap base from dispatching branch, not a new input

Simpler than adding a base_ref input: workflow_dispatch always runs
against `github.ref_name` (the branch passed to --ref). For
production, that's main. For pre-merge testing, dispatch from the
feature branch with `gh workflow run --ref feat/branch` and the
bootstrap flow branches from, bumps, and opens its PR against that
branch.

- Bootstrap checkout uses github.ref_name
- Bootstrap PR's --base uses github.ref_name
- eff step emits base_ref for detect-change.mjs to use as BASE_REF
  - pull_request trigger: event.pull_request.base.ref
  - workflow_dispatch retry: gh pr view <n> --json baseRefName
  - workflow_dispatch bootstrap: github.ref_name

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>

* Drop labels; gate via bot-author + paths filter

Labels were doing two jobs — neither worth the friction of pre-creating
the labels in every repo that adopts this:

  1. Gating which PRs trigger augmentation. Replaced with the existing
     pull_request paths: filter (only YAML edits) + user.login check
     (must be renovate[bot]). Human PRs editing the YAML are out of
     scope; they should be reviewed normally without skill augmentation.
  2. Human filtering of "failed" / "needs-context" PRs. Replaced with
     PR body sections and a failure-path PR comment that carries the
     run URL.

Changes:
- Drop labels from Renovate packageRule in renovate.json.
- Drop --label from bootstrap gh pr create.
- Drop "Add needs-human-context label" step (gaps already shown in PR
  body).
- Simplify failure step to comment-only; drop upstream-docs-failed
  label. The PR comment already includes the run URL + retry hint.
- Drop labeled trigger type; no longer needed without label-race
  concerns.
- Drop label check from workflow_dispatch retry validation.

Also unblocks the first real dispatch, which failed on
"upstream-content not found" since the label was never pre-created.

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>

* Inline node setup after PR-branch checkout

The `./.github/actions/setup` composite starts with its own
actions/checkout that defaults to the dispatching ref, so calling it
after "Checkout PR branch" clobbers the PR-branch working tree with
the dispatching branch's content. detect-change.mjs then compared the
dispatching branch's YAML against itself and reported "No version
changes detected" — blocking the first dispatch.

Fix: inline the setup-node + cache + npm ci steps after the PR-branch
checkout instead of calling the composite. The composite is still
used in the bootstrap flow where re-checkout of the dispatching
branch is harmless.

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>

* Don't track .claude/scheduled_tasks.lock

* Restore id-token: write for claude-code-action OIDC

Removed earlier as 'unused'. anthropics/claude-code-action@v1 needs
id-token: write to fetch an OIDC token. Skill step was failing with
"Could not fetch an OIDC token" on the first real dispatch.

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>

---------

Co-authored-by: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
rdimitrov added a commit to stacklok/toolhive-registry-server that referenced this pull request Apr 22, 2026
## Context

The \`update-docs-website\` job in \`releaser.yml\` was a legacy of the
pre-Renovate docs pipeline. Back then docs-website pulled the Registry
Server Swagger via jsdelivr (a remote reference), so every release
needed Vercel to rebuild docs-website to pick up the new spec. This job
triggered that rebuild:

\`\`\`yaml
update-docs-website:
  name: Trigger Docs Website Rebuild
  needs: [release-binaries]
  runs-on: ubuntu-latest
  steps:
    - name: Trigger Vercel Deploy Hook
      run: |
echo \"Triggering docs website rebuild to refresh the API spec...\"
curl -f -X POST \"\${{ secrets.DOCS_VERCEL_DEPLOY_HOOK }}\" || exit 1
\`\`\`

## Why it's dead weight now

docs-website now pins the Swagger as a **static file** synced by
\`sync-assets.mjs\` inside a Renovate-opened version-bump PR. Vercel
auto-redeploys on push to docs-website's \`main\` branch when that PR
merges.

The hook currently fires after every release against docs-website
\`main\` at a state where nothing has changed — pure no-op that consumes
runner minutes and creates a redundant deploy in Vercel.

Noticed on the v1.3.0 release run:

https://github.com/stacklok/toolhive-registry-server/actions/runs/24778037006/job/72503570856

## Scope

Cross-checked the other three upstreams that docs-website tracks —
\`toolhive\`, \`toolhive-studio\`, \`toolhive-cloud-ui\` — none have an
equivalent job. \`toolhive\` was cleaned up via stacklok/toolhive#4982
when the unified docs pipeline shipped; this is the last leftover.

## Follow-up (not in this PR)

The \`DOCS_VERCEL_DEPLOY_HOOK\` repo secret can be deleted once this
merges — the only consumer is this job.
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

size/S Small PR: 100-299 lines changed

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants