Skip to content

ci: gate every version site against pyproject.toml in one place - #393

Merged
cdeust merged 1 commit into
mainfrom
ci/version-surfaces-gate
Aug 8, 2026
Merged

ci: gate every version site against pyproject.toml in one place#393
cdeust merged 1 commit into
mainfrom
ci/version-surfaces-gate

Conversation

@cdeust

@cdeust cdeust commented Aug 8, 2026

Copy link
Copy Markdown
Owner

Phase 1 of #392.

What was wrong

Three gates each compared a version number, and each trusted a different canonical source:

Gate Canonical source Covers
check_doc_claims.py (check_versions) pyproject.toml manifest, server, package, badge
check_marketplace_pins.py the marketplace's own primary pin server + manifest only
tests_py/scripts/test_cross_host_manifests.py package.json four more manifests

Two sites were covered by nothing: server.json's packages[0].version — served to every MCP registry client — and .claude-plugin/marketplace.json's metadata.version.

And the gate that came closest is path-filtered: marketplace-pins.yml only runs when marketplace.json itself changes, so it sits outside ci-green. A PR that bumps five surfaces and forgets the sixth is green, reaches main, and is caught up to seven days later by the Monday cron.

What this does

One module, one canonical value ([project].version in pyproject.toml), all 14 sites across 11 files. SURFACES is data — adding a 15th site of an existing kind is one tuple row, not a new branch (OCP).

Reuse over reimplementation: badge parsing comes from doc_claim_structural.check_badge, the pyproject regex from doc_claim_sources, and the marketplace primary-entry predicate mirrors check_marketplace_pins.main verbatim so the two cannot disagree about which entry is primary. Every check fails closed on a missing file, unparseable JSON, or an absent key.

Wired into the lint job, which ci-green already covers — a partial bump now turns the PR red.

Deduplication, not a fourth overlapping gate

check_doc_claims.check_versions is removed: the new gate is a strict superset. Its orphaned tests, its dead fixture entries, and a then-unused json import go with it, and its docstring no longer advertises a claim family it does not check. test_cross_host_manifests.py now asserts through the new module instead of hand-maintaining a parallel list of five of the fourteen sites.

check_marketplace_pins.py is deliberately untouched — its PIN_BEHIND_TAG and third-party pin-staleness checks are a different and still-valuable concern.

Proof the gate bites

On one of the two previously-uncovered sites:

$ # desync server.json packages[0].version → 9.9.9
$ python scripts/check_version_surfaces.py ; echo exit=$?
Version surfaces disagree with pyproject.toml:
  server.json: packages[0].version: version '9.9.9', pyproject says '4.17.2'
exit=1

$ python scripts/check_doc_claims.py >/dev/null ; echo exit=$?
exit=0        # ← the pre-existing gate is blind to this site

Verification

python scripts/check_version_surfaces.py            → exit 0 (14/14 agree)
python scripts/check_doc_claims.py                  → exit 0 (still green after the dedup)
pytest test_version_surfaces test_check_doc_claims test_cross_host_manifests
                                                    → 129 passed, 14 subtests
ruff check .                                        → All checks passed
ruff format --check .                               → 1154 files already formatted

Tests cover the nominal case, one desync per surface, dedicated tests for both previously-uncovered sites, fail-closed paths (missing uv.lock package block, zero self-hosted marketplace entries), the real repo tree, and main()'s three exit codes.

🤖 Generated with Claude Code

https://claude.ai/code/session_01263uv1QqR8TVzw2jXYUrXn

Three gates each compared a version number, and each trusted a DIFFERENT
canonical source: check_doc_claims.py against pyproject.toml (manifest,
server, package, badge), check_marketplace_pins.py against the marketplace's
own primary pin (server + manifest only), test_cross_host_manifests.py
against package.json (four more manifests). Two sites were covered by
nothing at all — server.json's packages[0].version, served to every MCP
registry client, and .claude-plugin/marketplace.json's metadata.version.

Worse, the gate that came closest is path-filtered: marketplace-pins.yml
only runs when marketplace.json itself changes, so it sits outside the
ci-green aggregate. A PR that bumps five surfaces and forgets the sixth is
green, reaches main, and is caught up to seven days later by a Monday cron.

Replace the version half of that fragmentation with one module reading ONE
canonical value — [project].version in pyproject.toml — and comparing it to
all 14 sites across 11 files. SURFACES is data: adding a 15th site of an
existing kind is one tuple row, not a new branch. Badge parsing and the
pyproject regex are reused from doc_claim_structural/doc_claim_sources
rather than reimplemented; the marketplace primary-entry predicate mirrors
check_marketplace_pins.main verbatim so the two cannot disagree about which
entry is primary. Every check fails closed on a missing file, unparseable
JSON, or an absent key.

Wired into the lint job, which ci-green already covers, so a partial bump
now turns the PR red instead of surfacing a week later.

check_doc_claims.check_versions is removed rather than left alongside: the
new gate is a strict superset of it. Its orphaned tests and fixture entries
and a then-unused json import go with it, and its docstring no longer
claims a version family it does not check. test_cross_host_manifests.py
asserts through the new module instead of hand-maintaining a parallel list
of five of the fourteen sites.

Proof the gate bites, on one of the two previously-uncovered sites:
desyncing server.json's packages[0].version to 9.9.9 makes
check_version_surfaces.py exit 1 naming that exact site, while
check_doc_claims.py still exits 0.

Refs #392

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01263uv1QqR8TVzw2jXYUrXn
@cdeust
cdeust merged commit 749e7fb into main Aug 8, 2026
22 checks passed
@cdeust
cdeust deleted the ci/version-surfaces-gate branch August 8, 2026 16:20
cdeust added a commit that referenced this pull request Aug 8, 2026
cdeust added a commit that referenced this pull request Aug 8, 2026
Re-applies the exact tree validated at 8b97994, which passed 22 checks
on the head of #394. Verified byte-identical: `git diff 8b97994` is empty.

What this contains, per reverted PR:
- #391 scripts/check_ci_gate_complete.py — the `CI Green` aggregate.
- #387 .github/actions/test-suite/action.yml — one test suite shared by
  ci.yml and release.yml instead of two drifting copies.
- #393 scripts/check_version_surfaces.py — one gate over all 14 version
  sites, pyproject.toml as the single canonical source.
- #394 the tag becomes an output of a green tree rather than a
  hand-pushed input.

No branch-protection change is required or requested. All eleven required
contexts keep their exact names — Lint, Type Check, Build Package,
Docker Smoke, Test (Python 3.10/3.11/3.12/3.13), Test (SQLite backend),
Test (Windows, SQLite backend) — this only adds jobs alongside them.
`CI Green` is not proposed as a required context; it gates the tag,
which needs no protection change to be enforced.
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant