Skip to content

feat(api): aggregate endpoints — subnet overview + registry summary - #243

Merged
JSONbored merged 1 commit into
mainfrom
feat/aggregate-endpoints
Jun 10, 2026
Merged

feat(api): aggregate endpoints — subnet overview + registry summary#243
JSONbored merged 1 commit into
mainfrom
feat/aggregate-endpoints

Conversation

@JSONbored

Copy link
Copy Markdown
Owner

What

Two frontend-enabling composed endpoints so the UI renders pages with far fewer round-trips:

  • GET /api/v1/subnets/{netuid}/overview → R2-tier overview/{netuid}.json composing the subnet's profile + health + curation + gaps + counts (surfaces/endpoints/candidates) + gap priorities in one call (slug aliases resolve too). Replaces ~6 round-trips for a subnet page.
  • GET /api/v1/registry/summary → R2-tier registry-summary.json: subnet count, completeness rollup, surface/endpoint/provider/candidate counts, curation + profile level counts, top-10 subnets by completeness, and the latest change feed (from changelog.json). Powers a homepage/leaderboard in one call.

How

Both compose already-computed build data (no new probing), emitted in build-artifacts.mjs; R2-tier (high-churn, never committed). Follows the per-subnet evidence/gaps contract pattern (#231): contracts + schema components (SubnetOverviewArtifact, RegistrySummaryArtifact), validate-schemas dir mapping + validate-api checks (45 routes now), docs rows, and tests/subnet-overview.test.mjs.

Verification

  • 233 tests (4 new), lint, prettier, validate:schemas/api/openapi/types/docs/artifact-budgets all green.
  • Reproducibility: CI-mirroring build reproduces the committed contract artifacts exactly; changelog/build-summary/r2-manifest stay at main's version (subset-commit discipline). The new artifacts are R2-served (rebuilt every refresh).

Frontend-enabling composition so the UI renders pages with far fewer round-trips:

- GET /api/v1/subnets/{netuid}/overview -> R2-tier overview/{netuid}.json
  composing the subnet's profile + health + curation + gaps + counts (surfaces/
  endpoints/candidates) + gap priorities in one call (slug aliases resolve too).
- GET /api/v1/registry/summary -> R2-tier registry-summary.json: subnet count,
  completeness rollup, surface/endpoint/provider/candidate counts, curation +
  profile level counts, top-10 subnets by completeness, and the latest change
  feed (from changelog).

Both compose already-computed build data (no new probing); R2-tier (high-churn,
never committed). Follows the per-subnet evidence/gaps contract pattern (#231):
contracts + schema components (SubnetOverviewArtifact, RegistrySummaryArtifact),
validate-schemas dir mapping + validate-api checks (45 routes), docs, and tests
(tests/subnet-overview.test.mjs). 233 tests, lint, prettier, all validators
green; committed contract artifacts reproduce (subset-commit discipline).
@superagent-security

Copy link
Copy Markdown

Superagent didn't find any vulnerabilities or security issues in this PR.

@JSONbored
JSONbored merged commit 13c450d into main Jun 10, 2026
8 checks passed
@JSONbored
JSONbored deleted the feat/aggregate-endpoints branch June 10, 2026 20:41
JSONbored added a commit that referenced this pull request Aug 3, 2026
…sholds to the producer (#9311)

Two gaps in the alarm coverage over the two outputs of the poller's one Alpha
scan.

THE COUNTS LANE HAD NO WATCHDOG. #9302 gave it a D1 sink and metagraphed-infra
#243 re-enabled its producer, but nothing watched it -- which is the exact
shape of the failure that produced the lane's outage in the first place. Its
writer targeted a Postgres that went away, nominator_count degraded to null or
to a frozen 2026-08-02 mirror covering 564 of 1,031 validators, and no probe,
no red check and no exception fired. The read path degrades so gracefully that
the outage was invisible. nominator_positions -- the other output of the same
scan -- got a watchdog in #9273 for precisely this reason; its sibling now has
the matching one, same shape as neurons/positions/chain-detail before it: one
MAX() read, a pure rule, a summary rather than a throw, one exception per stale
tick. An EMPTY table alerts too, since that is the state in which every
nominator_count is still coming from the frozen mirror.

THE POSITIONS THRESHOLD UNDERCUT ITS OWN PRODUCER. It was 6 hours, chosen while
that lane had no producer at all, on the reasoning that six hours was "several
missed passes at any plausible cadence". The producer now feeding it runs on a
24h tick (VALIDATOR_NOMINATORS_POLL_SECS defaults to 24*3600, and one scan
writes both tables), so a healthy lane presents an age anywhere in
[0h, 24h+scan] and a 6h threshold would have alerted for roughly three quarters
of every day. It has not fired yet only because the table is still empty and
takes the no_rows branch instead; it would have started the moment the
re-enabled producer posted. An alarm that always fires is one nobody reads.

Both thresholds are now 30h -- one missed pass plus slack for the scan itself
(~4 min at the measured ~3,100 rows/sec) and cron jitter. Derived from the
cadence, not picked, and stated as such in both headers so the next cadence
change has somewhere to land. Tests pin the regression directly: a capture from
the middle of a 24h cycle must be quiet.

Cron 19,49 collides with nothing in workers/config.ts and stays off the */5
raw-capture and */15 probe grids; the suite asserts both that uniqueness and
that wrangler.jsonc actually declares the trigger, since dispatch keys on the
literal string and an undeclared cron is silently dead code.

Closes #9310
Part of #9146
JSONbored added a commit that referenced this pull request Aug 3, 2026
#9276 added a serving-Worker overlay filling nominator_count from the frozen
lakehouse mirror. It was the bridge for the period when D1 had no data, and
that period is over: migration 0012 gave the field a live table, #243/#267
gave it a live producer, and #9334 made the tier answer COMPLETELY -- absence
from a fresh scan reads as a confirmed zero, so nominator_count is now non-null
on 1,028 of 1,028 validators.

The overlay could only ever fire on a null count, because
validatorHotkeysNeedingCount collects exactly those and returns early when
there are none. There are none left to fill.

It was not simply dead, which is why this is worth doing rather than leaving.
The one situation that still reached it was a partial failure -- the counts
query throwing while the neurons query succeeded -- and there it served ~557
counts stamped 2026-08-02 from a mirror nothing refreshes, values that age
indefinitely. That is the failure the retired module's own header warned about:
a degraded count is worse than no count, because a card cannot tell its reader
which one it got. Null is the honest answer in that case.

Removed at all eight call sites across the three surfaces that shared it --
REST (workers/request-handlers/entities.ts), GraphQL (src/graphql.ts) and MCP
(src/mcp-server.ts) -- plus the module and its suite. Each site was a
multi-line expression wrapper, so this is unwrapping rather than deletion; the
2,877 tests across those three suites are what makes that safe to assert.

Closes #9337
Part of #9146
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant