Follow-up to #152. Console v0.12.0 states net origin offload documents-only and says the fifth term is missing from both sides of the ledger: the XHR/API calls a page's own scripts make when a rendering crawler runs it. This issue measures that term so the console can count it instead of stating it.
The quantity
k(page) = same-origin subrequests a page load makes whose response no shared cache would serve — the calls that reach the origin whoever runs the page. Then, per page-view by a crawler that executes scripts:
|
origin requests |
| without prerender |
1 + k (document, then the page's calls) |
| cache-served, snapshot without scripts |
0 — k is saved |
| cache-served, snapshot with scripts |
k — incurred |
| proxied origin page |
1 + k — incurred |
| our own render |
1 + k — incurred (the renderer runs the page too) |
Where each piece is measured
Browser (v1.22.0) — the renderer's response hook already sees every same-origin response and already inspects cache headers for its own resource cache. Add a pure classifier over (request, response) → uncacheable (explicit: non-GET, no-store/private/no-cache, Set-Cookie, uncacheable status), cacheable (explicit positive freshness: s-maxage/max-age/Expires), or unspecified (no freshness info — CDN-default dependent; reported, counted on neither side). Also count same-origin requests our block list aborted (blocked), so the undercount is visible. Carry the counts on the attempt and post them with the result as subrequests: { sameOrigin, cacheable, uncacheable, unspecified, blocked }, plus scriptsStripped: <postProcess.stripScripts> so the plugin knows whether the stored snapshot can hydrate at all. Cost: string checks per response; ~5 ints on the wire.
Plugin (prerender-v0.65.0)
PrerenderedPage gains uncacheableSubrequests: Int and scriptsStripped: Boolean (nullable; older rows read as unknown). Written on store.render gains series subrequests (method = kind) — a value per posted result, so Σ = mean × count; this is the render fleet's own k cost.- Registry:
analytics.bots[].rendersJs (boolean). Defaults true for the documented renderers only — Googlebot, Google InspectionTool, Bingbot, Applebot, YandexBot. Every AI crawler in the registry runs nothing and stays unflagged.
- Serve path: for a request from a flagged crawler, emit
hydration_calls (new metric; it needs its own three slots): path = side (saved | incurred | unknown), method = bot, type = source, value = k. saved = cache serve of a script-stripped snapshot; incurred = cache serve of a snapshot with scripts, or any origin serve; unknown = no page record, or a page that predates the upgrade. One in-memory counter bump; ~30 combo rows/period/node.
Console (prerender-console-v0.13.0) — originLoad() becomes
net = 1 − (proxied + renders + Σk_renders + probes + sitemaps + Σk_incurred) ÷ (arrived + Σk_saved + Σk_incurred), the exposure tile becomes "script calls: saved X · incurred Y", the unknown count is printed (it decays over one render cycle after the plugin deploys), and unspecified is shown as the CDN-default caveat.
Order
Every stage tolerates the others being absent (the console falls back to v0.12.0's exposure count; the plugin ignores fields an older fleet does not post). Browser first so k starts landing on pages; plugin; console. Plugin + console land in one PR here (the console's catalog guard scans the plugin's emit sites, so a plugin-only merge would fail the console suite), released as two tags.
Not resource-intensive
Measurement rides on hooks that already run per response; storage is two small nullable fields per page; the serve path adds one counter bump for rendering-crawler serves. The cost is coordination and the one-render-cycle warm-up.
Follow-up to #152. Console v0.12.0 states net origin offload documents-only and says the fifth term is missing from both sides of the ledger: the XHR/API calls a page's own scripts make when a rendering crawler runs it. This issue measures that term so the console can count it instead of stating it.
The quantity
k(page)= same-origin subrequests a page load makes whose response no shared cache would serve — the calls that reach the origin whoever runs the page. Then, per page-view by a crawler that executes scripts:1 + k(document, then the page's calls)0—kis savedk— incurred1 + k— incurred1 + k— incurred (the renderer runs the page too)Where each piece is measured
Browser (
v1.22.0) — the renderer's response hook already sees every same-origin response and already inspects cache headers for its own resource cache. Add a pure classifier over(request, response)→uncacheable(explicit: non-GET,no-store/private/no-cache,Set-Cookie, uncacheable status),cacheable(explicit positive freshness:s-maxage/max-age/Expires), orunspecified(no freshness info — CDN-default dependent; reported, counted on neither side). Also count same-origin requests our block list aborted (blocked), so the undercount is visible. Carry the counts on the attempt and post them with the result assubrequests: { sameOrigin, cacheable, uncacheable, unspecified, blocked }, plusscriptsStripped: <postProcess.stripScripts>so the plugin knows whether the stored snapshot can hydrate at all. Cost: string checks per response; ~5 ints on the wire.Plugin (
prerender-v0.65.0)PrerenderedPagegainsuncacheableSubrequests: IntandscriptsStripped: Boolean(nullable; older rows read as unknown). Written on store.rendergains seriessubrequests(method = kind) — a value per posted result, so Σ = mean × count; this is the render fleet's ownkcost.analytics.bots[].rendersJs(boolean). Defaultstruefor the documented renderers only — Googlebot, Google InspectionTool, Bingbot, Applebot, YandexBot. Every AI crawler in the registry runs nothing and stays unflagged.hydration_calls(new metric; it needs its own three slots): path = side (saved|incurred|unknown), method = bot, type = source, value =k.saved= cache serve of a script-stripped snapshot;incurred= cache serve of a snapshot with scripts, or any origin serve;unknown= no page record, or a page that predates the upgrade. One in-memory counter bump; ~30 combo rows/period/node.Console (
prerender-console-v0.13.0) —originLoad()becomesnet = 1 − (proxied + renders + Σk_renders + probes + sitemaps + Σk_incurred) ÷ (arrived + Σk_saved + Σk_incurred), the exposure tile becomes "script calls: saved X · incurred Y", theunknowncount is printed (it decays over one render cycle after the plugin deploys), andunspecifiedis shown as the CDN-default caveat.Order
Every stage tolerates the others being absent (the console falls back to v0.12.0's exposure count; the plugin ignores fields an older fleet does not post). Browser first so
kstarts landing on pages; plugin; console. Plugin + console land in one PR here (the console's catalog guard scans the plugin's emit sites, so a plugin-only merge would fail the console suite), released as two tags.Not resource-intensive
Measurement rides on hooks that already run per response; storage is two small nullable fields per page; the serve path adds one counter bump for rendering-crawler serves. The cost is coordination and the one-render-cycle warm-up.