docs(ci): self-hosted MCP Toplist rank badge, refreshed monthly from upstream - #242
Merged
Conversation
Adds a static SVG badge recording that Cortex ranks in the top 1.2% of MCP servers tracked by MCP Toplist, and links it to the source listing. The asset is committed rather than hotlinked. A remote badge URL is a third-party beacon on every README view and lets its host restate the claim at any time with no commit in our history; a static file cannot drift. The cost is that it cannot self-update, so the date is part of the claim and must be re-checked when the rank moves. Provenance is recorded in the SVG itself: mcptoplist.com ranked hypermnesia-mcp #964 of 81,919 tracked servers on 2026-07-28, giving 964/81919 = 1.18% -> "Top 1.2%", with the verify URL alongside it. Wording is deliberately attributive. Per their published methodology the score is "a popularity and activity signal, not a quality assessment", and ~25% of its weighting (organic reach, package downloads) is undisclosed -- so the badge, alt text and title all say Cortex is ranked in the top 1.2% by MCP Toplist, never that it is a top-1.2% server. Palette (#3b3129 / #a53e00) is taken from assets/banner.svg. No <style> block or external font, since GitHub's SVG sanitizer strips both; textLength/lengthAdjust pin every string so substituted fonts still fit. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01BJt9KN23rUNQzwoJv4jRWw
The badge is a committed file, so it cannot restate itself without a commit
here -- and equally cannot self-update. Its date is part of the claim and
goes stale by INACTION, which never opens a PR. This closes that loop.
Monthly, not weekly: the badge stamps a month, so that is the least frequent
cadence keeping the date honest and the most frequent one that does not
churn a PR proposing an identical file. No pull_request trigger -- a
required check must not depend on a third party's uptime. The 51 unit tests
cover the logic with no network at all.
Two extraction paths, both feeding one validator:
1. /data/leaderboard.json -- the structured export the site links from its
homepage. Returns HTTP 503 (measured 2026-07-28: 3/3 attempts, 8-14s
each under a browser UA, so a server-side generation timeout rather
than UA gating or rate limiting). Its schema has therefore never been
observed, and writing a parser for a shape I have not seen would be
invention -- so it accepts only documented candidate shapes under
strict validation and refuses anything else rather than guessing.
2. The server page's "ranks #N of M servers tracked" sentence. This is the
ONLY construct on that page carrying both numbers: the <title>, the
og/twitter meta tags and all four JSON-LD blocks give the rank without
the total, so none of them can yield a percentile.
Fail closed. The validator refuses a non-numeric, zero, negative or
beyond-the-field figure, guarding both the division in percentile() and the
semantics of the claim. A figure that fails is never written: the script
exits non-zero, the badge keeps its last good value, and the run goes red. A
fallback is never silent -- the failed path is reported even when a later
one succeeds.
Found by the tests before shipping: a top-of-field rank renders the tier as
"Top <0.1%", whose unescaped '<' made the badge invalid XML. Text and
attributes are now escaped, verified at the rank-1, mid-field and last-place
boundaries.
Wording stays attributive. Upstream's methodology page says its score "is a
popularity and activity signal, not a quality assessment" and keeps ~25% of
the weighting undisclosed, so the badge says Cortex is RANKED in this tier,
never that it IS -- and each generated SVG carries the source, raw figures,
arithmetic and verify URL in a comment so the claim is re-derivable from the
file alone.
Caveat recorded in the workflow: PRs opened with the default GITHUB_TOKEN do
not trigger workflows, so main's required checks will not start on them. An
optional BADGE_REFRESH_TOKEN secret is preferred when present.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01BJt9KN23rUNQzwoJv4jRWw
The suite's advertised count is pinned in six files and gated by scripts/check_doc_claims.py, which the Python 3.12 job runs against a live `pytest --collect-only -q`. This branch's 51 new tests moved the canonical count 6297 -> 6348, so every pinned site disagreed with the repository and the gate failed the job -- correctly, and on my change. Updated: the README test badge and its alt text, the README and CONTRIBUTING run-command comments, CLAUDE.md's test line, the assurance case's CI claim, and the four .bestpractices.json justifications that quote the figure. The CHANGELOG's historical "Suite grows 6275 -> 6297" is deliberately left alone: it records what a past PR did, not what the suite holds now. Verified: `check_doc_claims.py --test-count 6348` reports "doc claims OK", and the local collection agrees with CI's at 6348. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01BJt9KN23rUNQzwoJv4jRWw
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
Adds a self-hosted MCP Toplist rank badge to the README, and the monthly cron that keeps its claim from going stale.
The badge is a committed static SVG, not a hotlinked remote image. A remote badge URL is a third-party beacon on every README view and lets its host restate our claim with no commit in our history; a committed file cannot drift. The cost of that choice is that it cannot self-update — the date is part of the claim and goes stale by inaction, and inaction never opens a PR. The second commit closes that loop with a monthly workflow that regenerates the SVG from upstream and proposes it as a PR.
Provenance is carried inside the SVG itself: source path, raw rank and field size, the percentile arithmetic, the date read, and a verify URL — so the claim is re-derivable from the file alone. mcptoplist.com ranked
hypermnesia-mcp#964 of 81,919 tracked servers on 2026-07-28 → 964/81919 = 1.18% → "Top 1.2%".Wording is deliberately attributive. Upstream's published methodology says its score "is a popularity and activity signal, not a quality assessment" and leaves ~25% of the weighting (organic reach, package downloads) undisclosed. The badge, its alt text and its title therefore all say Cortex is ranked in the top 1.2% by MCP Toplist — never that it is a top-1.2% server.
Closes #
Type of change
Test plan
51 new unit tests, no network in any of them (the script's fetch seam is injected):What the tests would catch. Flipping any validator comparison (
rank > 0→>=,rank <= total→<) is killed by the boundary cases; dropping the XML escaping is killed by the rank-1 case (see below); returning a successful parse from a malformed candidate shape is killed by the strict-shape refusals; swallowing a failed fallback path is killed by the assertions on the emission of the failure report, not only on the final value.Audit notes
Two extraction paths, one validator.
/data/leaderboard.json— the structured export the site links from its homepage. Returns HTTP 503 (measured 2026-07-28: 3/3 attempts, 8–14s each under a browser UA — a server-side generation timeout, not UA gating or rate limiting). Its schema has therefore never been observed, and writing a parser for a shape I have not seen would be invention. It accepts only documented candidate shapes under strict validation and refuses anything else rather than guessing."ranks #N of M servers tracked"sentence. This is the only construct on that page carrying both numbers — the<title>, the og/twitter meta tags and all four JSON-LD blocks give the rank without the total, so none of them can yield a percentile.Fail closed. The validator refuses a non-numeric, zero, negative, or beyond-the-field figure — guarding both the division in
percentile()and the semantics of the claim. A figure that fails is never written: the script exits non-zero, the badge keeps its last good value, and the run goes red. A fallback is never silent — the failed path is reported even when a later one succeeds.Found by the tests before shipping: a top-of-field rank renders the tier as
Top <0.1%, whose unescaped<made the badge invalid XML. Text and attributes are now escaped, verified at the rank-1, mid-field and last-place boundaries.Rendering constraints. Palette (
#3b3129/#a53e00) taken fromassets/banner.svg. No<style>block and no external font — GitHub's SVG sanitizer strips both;textLength/lengthAdjustpin every string so a substituted font still fits.BADGE_REFRESH_TOKENThe workflow opens its refresh PR with
secrets.BADGE_REFRESH_TOKEN || secrets.GITHUB_TOKEN. The secret is not currently set. With the defaultGITHUB_TOKEN, GitHub deliberately does not trigger workflows on the resulting PR, somain's required checks never start and the refresh PR cannot be merged until a maintainer pushes to its branch. This is documented at the step in the workflow file. Setting a PAT withreposcope asBADGE_REFRESH_TOKENmakes those PRs self-checking. This does not affect this PR — only the monthly ones it will open from 1 August onward.Coding-standards compliance
scripts/, imported by nothing in the package.any/ untyped dicts at boundaries — the fetched payload is validated into typed values at the trust boundary before any use.refresh_mcp_toplist_badge.pyis 406.Completion Ledger (§13.2)
Every code path introduced by
git diff origin/main...HEAD, mapped to its asserting evidence.test_refresh_mcp_toplist_badge.pyshape-acceptance casesranks #N of Mmatchedpercentile()— division guarded by validatorTop <0.1%XML escapingadd-paths+ create-pull-request no-op on identical fileCHANGELOG.md+1README.md+1 (badge line)BADGE_REFRESH_TOKENis referenced by name only, never a value. Caveat disclosed above.check_doc_claims.py --test-count 6348→doc claims OK; CITest (Python 3.12)green oncfec4daCI history — one real failure, fixed
The first push failed
Test (Python 3.12), the only job running the doc-claims gate. The 51 new tests moved the canonical suite count 6297 → 6348, which the repo pins in 11 places; the gate compares them against a livepytest --collect-only -qand failed the job. Fixed incfec4da— all 11 sites updated, including the README badge URL. The CHANGELOG's historicalSuite grows 6275 → 6297is deliberately left alone: it records a past PR, not the current suite.Not silently changed, flagged for the maintainer: the README test badge is a hotlinked
img.shields.ioURL — the very pattern this PR argues against. Its number was updated; converting the README's badge row to committed SVGs is a separate decision and is not in this diff.Reviewer checklist
cfec4da.🤖 Generated with Claude Code
https://claude.ai/code/session_01BJt9KN23rUNQzwoJv4jRWw