fix(release): make the v0.23.1 tag pass its own publish gates

[kovtcharov]

Summary

v0.23.1 cannot be tagged from main today — publish.yml's validate job rejects it twice before anything is built. This makes the tag pass, and adds the PR-time checks whose absence let the release PR merge untaggable.

Why

The release PR bumped version.py to 0.23.1 / Lemonade 11.8.1 but left the docs navbar reading v0.23.0 · Lemonade 11.5.0, and shipped release notes with no ## What's New section. Both are hard gates in publish.yml, and neither runs on a PR — they only fire against a pushed tag, so the first time anyone finds out is mid-release.

The notes also hand new users two commands that do not exist in any GAIA binary: gaia list is not a subcommand, and gaia install is the Lemonade installer, which rejects a trailing agent id. A user following the v0.23.1 notes to install the flagship agent gets an argparse error. The working commands are gaia hub install gaia --trust / gaia hub list — the same pair the TUI already tells users to run.

Linked issue

Closes #3353
Refs #1128

Changes

• Navbar label and release notes corrected so the tag passes publish.yml unchanged.
• Release notes now name real commands — the previous ones fail for every user who copies them.
• tests/unit/test_docs_json_release.py checks docs.json against version.py at PR time; docs.yml runs the release-note validator for the declared version. These are the two gates that previously existed only behind a tag.

Test plan

• python util/validate_release_notes.py docs/releases/v0.23.1.mdx --tag v0.23.1 → exits 0 (was 1)
• python -m pytest tests/unit/test_docs_json_release.py -q → 5 passed; reverting docs.json alone fails 3 of them
• publish.yml's inline docs.json check executed verbatim with TAG_NAME=v0.23.1 → exits 0
• The new docs.yml step extracted from the workflow and run against a simulated bump to 0.24.0 → exits 1
• python util/lint.py --all → Black / isort / Flake8 pass; the 9 Pylint errors are pre-existing Windows-only os.killpg / os.geteuid false positives in files this PR does not touch (identical count on the unmodified base)

Evidence

CLI — the commands the notes told users to run (.venv\Scripts\python.exe -m gaia.cli):

$ gaia list
cli.py: error: argument action: invalid choice: 'list' (choose from prompt, chat, talk,
email, api, telegram, schedule, knowledge, download, stats, test, youtube, kill, llm,
eval, report, perf-vis, mcp, lemonade, daemon, hub, cache, memory, diagnostics, agent,
connectors, skill, config, init, install, uninstall)

$ gaia install -h
usage: cli.py install [-h] ... [--lemonade] [--yes] [--silent]      # no agent id accepted

$ gaia hub install --help
usage: cli.py hub install [-h] [--version VERSION] [--trust] agent_id
  agent_id           Agent to install (e.g. email)
  --trust            Explicitly trust a non-verified agent ...

Release-notes gate, before → after:

$ python util/validate_release_notes.py docs/releases/v0.23.1.mdx --tag v0.23.1   # before
❌ docs/releases/v0.23.1.mdx:
   - Missing required section: '## What's New' or '## Key Changes'
   (exit 1)

$ python util/validate_release_notes.py docs/releases/v0.23.1.mdx --tag v0.23.1   # after
✅ 1 file(s) validated successfully
   (exit 0)

publish.yml's docs.json gate, run verbatim with TAG_NAME=v0.23.1:

before:  ERROR: docs/docs.json navbar version label does not reference v0.23.1   (exit 1)
after:   docs.json has release entry for v0.23.1
         docs.json navbar references v0.23.1                                     (exit 0)

New PR-time gates:

$ python -m pytest tests/unit/test_docs_json_release.py -q
5 passed in 3.35s

# with docs.json reverted to the v0.23.0 label:
FAILED test_navbar_label_matches_gaia_version
FAILED test_navbar_label_matches_lemonade_version
FAILED test_version_label_carries_both_versions
3 failed, 2 passed

# the new docs.yml step, against a simulated bump with no notes written:
src/gaia/version.py declares v0.24.0
❌ docs/releases/v0.24.0.mdx: File not found                                      (exit 1)

• Agent exposed in the Agent UI — N/A, no runtime code changed.
• MCP tools / servers — N/A, no MCP surface changed.
• CLI — above; used only to establish which commands actually exist.
• HTTP API / REST — N/A, no API surface changed.

Not in this PR

• update-release-branch.yml and release_components.yml still fire on the same tag independently of validation and approval — update-release-branch.yml force-pushes release and exits 1 on pre-release tags; release_components.yml redeploys the production Cloudflare Worker before any approval. Chaining those after github-release is a separate PR.
• approve-publish hard-requires email-eval, which is currently failing on Anthropic billing (credit balance is too low). Even with these validators fixed the tag stalls before approval until that is cleared — an ops action, not a code change (eval: route remaining direct-Anthropic-API eval paths through Claude Code (subscription auth) #1344).

Checklist

• I have linked a GitHub issue above.
• I have described why this change is being made.
• I have run linting and tests locally.
• I have attached real-world evidence matched to the surface I changed, or marked each surface N/A.
• I have updated documentation if user-visible behavior changed — the release notes are the documentation this PR corrects.

[github-actions]

Verdict: Approve with suggestions

This makes the v0.23.1 tag pass the gates that would otherwise fail it mid-release, and fixes release notes that handed users two commands no GAIA binary accepts. Both corrections check out: the version/navbar/notes now agree, and the replacement commands match what the CLI actually parses and what the README already tells people to run.

One gap worth closing before merge, and it's small: the new docs.json guard is a unit test, and the unit-test workflow doesn't run on documentation-only changes. So the exact drift this PR is guarding against — someone editing the navbar label in a docs-only pull request — still slips through to the tag, which is the moment this PR is trying to stop being the moment of discovery. Adding the two documentation paths to the unit-test workflow's trigger list closes it; that file already does this for one other doc.

Two smaller things: the new "What's New" entry is a prose block where every other entry in the file is a single bullet, and it repeats the same fact the opening paragraph and the bug-fix entry already state — which also pushes the notes past the release process's own word budget for a patch release. And both new gates now fail loudly whenever the declared version has no notes file yet, which is intended for a release pull request but will also redden unrelated documentation changes if a version bump ever lands ahead of its notes.

Real-world evidence

No evidence bundle was produced for this run (evidence-bundle.md is absent) — this PR touches docs, CI, and a test, none of which is an Agent UI or inference surface, so nothing was deferred to the strix-halo lane. The PR description carries its own CLI transcripts, and I re-ran the two release gates in this checkout:

$ python util/validate_release_notes.py docs/releases/v0.23.1.mdx --tag v0.23.1
Validating docs/releases/v0.23.1.mdx...
✅ 1 file(s) validated successfully
exit=0

publish.yml's inline docs.json check, reproduced against this tree: navbar label v0.23.1 · Lemonade 11.8.1, release page releases/v0.23.1 present, notes file present — all pass. I also confirmed against the CLI's own parser that gaia list is not a subcommand and gaia install accepts no agent id, so the command correction in the notes is real and not cosmetic.

Not run here: pytest isn't installed in this environment, so I verified the five new assertions by reproducing their logic rather than executing them. The PR reports them passing.

🔍 Technical details

🟡 The new docs.json guard can't fire on the PR shape it most needs to catch

tests/unit/test_docs_json_release.py is the only check on docs.json ↔ version.py, but test_unit.yml's paths filter (.github/workflows/test_unit.yml:15-26 and 31-42) covers src/** and tests/** only — not docs/docs.json. docs.yml runs the notes validator but never touches docs.json.

Practical effect: a release PR is covered (it bumps src/gaia/version.py, so src/** matches). A docs-only PR that edits or drops the navbar label is not — and that lands on main untested until some later src/** PR fails a test whose subject it didn't touch, or until publish.yml fails against the tag. That workflow already carries precedent for exactly this: it lists docs/guides/custom-agent.mdx because a docs-only change can break a unit test. Same reasoning applies here:

      # tests/unit/test_doc_examples_runnable.py execs this guide's own code
      # blocks, so a docs-only change can still break the guard.
      - 'docs/guides/custom-agent.mdx'
      # tests/unit/test_docs_json_release.py pins docs.json + the release notes
      # to src/gaia/version.py, so a docs-only change can break it too.
      - 'docs/docs.json'
      - 'docs/releases/**'

(needs adding to both the push and pull_request blocks).

🟢 ## What's New is a prose block where the house style is one bullet

docs/releases/v0.23.1.mdx:23-27. Every other entry in the file is a single bullet; this one is an H3 plus a paragraph, and it restates what line 8 and line 33 already say — third telling of the same fact, with gaia hub list named twice in one sentence. It also takes the narrative section to 358 words against the release skill's 350-word patch cap (awk '/^## (Bug Fixes|…)/{exit}' … | wc -w).

The validator only needs the heading to exist, so a bullet satisfies it:

## What's New

- **The flagship GAIA agent installs from the terminal hub** — `gaia hub install gaia --trust` downloads it and starts its sidecar, and `gaia hub list` now shows it next to `email`.

That lands the section at ~310 words and drops the duplicate command mention.

🟢 Both new gates go red repo-wide if a version bump ever precedes its notes

.github/workflows/docs.yml:62-70 and test_release_notes_file_exists_for_current_version both key off the declared version, so version.py = 0.24.0 with no docs/releases/v0.24.0.mdx fails every docs PR and every unit-test run, not just the release PR. The current release flow bumps version and notes in one commit, so this is fine today — but the repo used to bump to a dev version separately (8aeefad6 "release: v0.17.3 notes + bump to 0.17.4 for development"). Worth a line in the release skill noting the bump and the notes must now land together, so a future maintainer doesn't rediscover it as mystery CI failures.

Strengths

• The unit test mirrors publish.yml:126-168 assertion-for-assertion (navbar label, releases/v<version> page, notes file), and test_version_label_carries_both_versions closes a hole the workflow itself has — two labels each half-right would pass the tag gate but ship a wrong navbar.
• Failure messages name publish.yml as the downstream consequence, so whoever hits the red test knows what it is protecting rather than just what string mismatched.
• The command correction is verified, not guessed: gaia install (src/gaia/cli.py:3013) takes no positional agent id, list is absent from the top-level subparsers, and gaia hub install <id> --trust (src/gaia/cli.py:2689-2705) is the pair tui/README.md:111-113 already documents.
• Adding util/validate_release_notes.py to docs.yml's trigger paths means a change to the validator re-validates the current notes rather than going unexercised until a tag.

[kovtcharov]

All three addressed in ccfffaf8.

The guard now covers docs-only PRs. docs/docs.json and docs/releases/** are in both the push and pull_request blocks of test_unit.yml, commented the same way as the docs/guides/custom-agent.mdx line above them. You were right that this was the gap that mattered — the test I added was blind to the exact PR shape that produced the bug.

What's New is one bullet now. Took your suggested edit. The narrative section is 333 words against the 350-word patch cap.

On the version-bump question, I kept the gates strict and wrote the constraint down instead of loosening them. A bump with no notes is not a benign intermediate state — it is the untaggable state this PR exists to make loud, and CLAUDE.md's fail-loudly rule cuts against adding a tolerance to the check that catches it. The release skill's Phase 2 already commits version.py and the notes in one git add, so today's flow never hits it; what was missing was anyone saying why it has to stay that way. .claude/skills/gaia-release/SKILL.md now carries it as a hard rule naming the old separate-bump pattern and what breaks if it comes back.

What I consciously did not do: make test_release_notes_file_exists_for_current_version skip when the notes are absent. That would mean a bump could sit on main with no notes and nothing would say so until someone pushed the tag — the same late-discovery this PR is trying to end, just moved one step further out.

🔍 Technical details

Verified on ccfffaf8:

$ python -m pytest tests/unit/test_docs_json_release.py -q
5 passed in 3.10s

# regression probe, docs.json reverted via `git apply -R` (not stash):
FAILED test_navbar_label_matches_gaia_version
FAILED test_navbar_label_matches_lemonade_version
FAILED test_version_label_carries_both_versions
3 failed, 2 passed

$ python util/validate_release_notes.py docs/releases/v0.23.1.mdx --tag v0.23.1
✅ 1 file(s) validated successfully

$ awk '/^## (Bug Fixes|Known Issues|Contributors|Full Changelog)/{exit} {print}' \
    docs/releases/v0.23.1.mdx | wc -w
333

$ python util/lint.py --all
Black PASS · isort PASS · Flake8 PASS · Pylint [X] 9 errors

The 9 Pylint errors are unchanged from the base: Windows-only os.killpg / os.getpgid / os.geteuid no-member false positives in daemon/sidecars/{ledger,manager}.py and installer/lemonade_installer.py, none of which this PR touches.

test_unit.yml's paths after the change, parsed from the file so both blocks are confirmed identical:

push: ['src/**', 'tests/**', 'util/check_pe_resources.py', 'installer/tui/**',
       'setup.py', 'pyproject.toml', '.github/workflows/test_unit.yml',
       'docs/guides/custom-agent.mdx', 'docs/docs.json', 'docs/releases/**']
pr:   (identical)