feat(console): net origin offload, and catch up to plugin 0.64.1; v0.12.0

[harper-joseph]

What

Origin offload, gross and net. The offload tile the rollout is judged on counts only crawler requests proxied to the origin live. It ignores every origin request this system makes — renders, change probes, sitemap fetches — so a deployment can post a 95% gross offload while sending the origin more requests than the crawlers would have. This adds the net figure:

net = 1 − (proxied serves + renders + change probes + sitemap fetches) ÷ crawler requests arrived

All four terms come from series already in the one analytics scan (bot_serve source=origin, render/outcome — exactly one row per posted result, probe_probed and sitemap_sitemaps read by value). originLoad() in charts.js owns the arithmetic so Traffic and the Overview cannot disagree. Traffic gains a "What the origin actually saw" panel (each term over time + a bar per cause); the Overview tile carries the net figure under the gross one.

Stated on the panel rather than assumed:

• a render counts as one origin request (the document; the page's subresources reach the origin only if the CDN doesn't cache them for the renderer);
• the probe/sitemap counters land in the bucket where a pass finished, so short ranges lump them — quote the 24h figure.

The term nothing here can count is stated, and it is missing from both sides. The requests a page's own scripts make when a rendering crawler (Googlebot, Bingbot, Applebot) runs it — the XHR/API calls no CDN caches — never pass through the plugin (the CDN forwards the document to us and sends subrequests straight to the origin). Without this system every such page-view costs the origin 1 + k, so the "crawlers asked for" baseline understates what the origin was spared; with it, a snapshot served without scripts triggers none of the k (a saving not credited), while a snapshot that keeps scripts, a proxied origin page, and every one of our own renders still trigger them (a cost not charged). The figure is documents-only on both sides and says so; the net tile reads "before crawler follow-up requests"; the panel reports the exposure as every page handed to a crawler — for every crawler, not a guessed list of bots, and never multiplied by a guessed factor. Where snapshots ship with scripts stripped, the true net for rendering crawlers is higher than shown.

Catch-up to plugin 0.61 → 0.64.1 (console was current to 0.60.0):

• verified cache status (v0.63.0) was invisible in the worst way: outside every cache-served sum (the hit rate read low) and in the "other" non-hit family. One shared CACHE_SERVED set now drives the freshness chart, the per-route table and the staleness sums; verified joins invalidated in one Invalidation family — one population, split into rescued and refused.
• Invalidations gains a third panel: refused vs rescued serves, page_verification writes (faults on the tile), every invalidation_reenqueue outcome including v0.64.0's forwarded / forward-failed (forwarded is never added to lowered — the owner counts it), and invalidation_error with lkg-expired as the failed-OPEN alarm. Reads the same cached 1h window the Overview holds. Two catalog-guard waivers removed — the guard's page_verification failure on current main is what started this.
• Change probe (v0.62.0): stateAvailable: false is named per node as "unknown, not idle" (the old shape made an unreadable state look like the probe being off); a running sweep shows its heartbeat count on the header, the sweep card and the node table.

Verification

• cd packages/console && node --test — 257 pass (was 227/1 fail on main: the guard's page_verification catch).
• npm run lint && npm run format:check clean.
• New: test/invalidationsView.test.js; extended traffic/overview/probe/aggregate suites pin the arithmetic (value vs count for pass counters, bot_request as denominator, negative net, all-bots tag under a filter, the two-sided caveat, verified cache-served, forwarded ≠ lowered).

No plugin change; console-only release train (prerender-console-v0.12.0).

Follow-up (not in this PR): measure k

The renderer already runs a cache policy on every same-origin response (renderer.ts response hook). Counting uncacheable same-origin subrequests per render gives k per page; stored on the cached page and applied — by what the registry says each crawler runs — to the baseline, the proxied serves, the cache-served pages (zero where scripts are stripped) and our own renders, it turns the exposure into a counted term on both sides. Browser → plugin → console train; happy to open the issue.

🤖 Generated with Claude Code

[gemini-code-assist]

Code Review

This pull request updates the prerender console to version 0.12.0, introducing a new 'net offload' metric alongside the gross offload figure, and adding a new panel to the Invalidations view to show what invalidations are doing (including page verification and demand-driven heals). It also adds support for displaying probe sweep progress and handling unreadable probe state rows. The changes are accompanied by comprehensive unit tests. The review feedback suggests using the ratioOf helper function in charts.js to safely calculate the net offload ratio and handle potential division issues.

[gemini-code-assist]

When calculating ratios or dividing metrics, use the ratioOf helper function to safely handle division and potential null/NaN values. This serves as an explicit guard at the call site and avoids relying on implicit null comparisons.

Suggested change

	net: arrived > 0 ? (arrived - total) / arrived : null,
	net: ratioOf(arrived - total, arrived),

References

1. When calculating ratios or dividing metrics, use a helper function (such as ratioOf) to safely handle division and potential null/NaN values. This serves as an explicit guard at the call site, even if the inputs are currently guaranteed to be numbers.

[harper-joseph]

Taken — ratioOf(arrived - total, arrived) in edd9af1. Same result (both inputs are finite sums), but it is the module's convention for every division and the guard should not be re-derived at one call site.

[harper-joseph]

Addressed the one inline finding (ratioOf for the net ratio). 257 tests, lint and format clean locally — this repo has no PR CI.