Skip to content

v27.8.0

Choose a tag to compare

@sebyx07 sebyx07 released this 11 Oct 01:45
· 25 commits to main since this release
1f647c8

Added

Tier 0 — core. Tier 1 — cache. Tier 4 — manifest, render. Tier 5 — cli. Both tracked apps.

  • revalidate.onInvalidate: 'purge' — a tag bust takes an isr page down. A bust marked the stored page stale and the next request was answered that copy, however old, per stored key and per replica: a withdrawn article reached its next visitor — a crawler, days later — once more. revalidate: { tags, ttl, onInvalidate: 'purge' } deletes every stored page of the route carrying the tag (each locale, each keyed query, stored 404s included) and evicts any render of them in flight, so a request after the bust renders and waits — it never joins a render that began before it, and one that had already joined renders again. The bound: the next request on the replica that ran the bust, and on every replica that receives the invalidation broadcast; a replica that misses it serves its copy until ttl and nothing stale after, which is why 'purge' requires a ttl as well as tags (X_ROUTE_MODE_INVALID). The response carries no stale-while-revalidate and an s-maxage of what is left of the ttl, so an edge cannot outlive it either. 'stale' is the default and today's behaviour. A replica still on an earlier release treats a purge as 'stale': finish the rollout first.
  • revalidate.maxStale — a bound on a copy served past its ttl. revalidate: { ttl: '10m', maxStale: '1h' }: older than ttl + maxStale, a stored page is dropped and the request waits for a render instead of being answered a copy of any age; 0 is "never serve stale". It is also the stale-while-revalidate a shared cache is told (a day when omitted). Unbounded by default — 0 under 'purge' — and needs a ttl. Past the bound the copy is dropped before the re-render, so a render that fails there answers its 5xx.
  • IsrStore.tagFence — a shared ISR store that refuses a purged page. A process's own fence only knows the busts it heard, so over a store two replicas write, one whose render began before a purge wrote the page straight back. A store may now carry a generation per tag entity (sample / bump / an atomic setIfCurrent); the controller reads it before a render and writes through it, and a bust moves it. memoryIsrStore implements it. A runtime.isrStore without one is refused at boot when any route purges (X_ROUTE_MODE_INVALID, naming the route).
  • revalidate.query — the query an isr page is keyed on is the route's to declare. revalidate: { query: ['page'] } (or [] for none): only the listed parameters are part of the stored page's key and of the url / query that load, meta and the page receive — one value each, sorted — so /blog?utm_source=… is the /blog page, with /blog as its canonical, and no query string mints an entry. Nothing is redirected. The default is unchanged: a route that declares no query still keys on the whole query string, where /blog?x=1, ?x=2, … are each a render and a stored page; x verify names every such route (the seo step's warnings) and the server logs isr.query.undeclared once per route. Declare it on every isr route — the default becomes [] in the next major. A declaration that OMITS a parameter the page reads is one stored document for every value of it, so x verify also warns for a query.<name> / searchParams.get('<name>') read in the route module that the declaration does not list, and outside production the query a page receives logs isr.query.unkeyed-read once for such a read. RouteDescriptor gains optional revalidateOnInvalidate, revalidateMaxStale, revalidateQuery; @ultimat3/render/server exports isrRequestUrl and undeclaredQuery.
  • noStore(data) — a loader's "do not keep this answer". From @ultimat3/render, a mark on the data like withStatus and composable with it. An isr render so marked is served private, no-store and never written to the store; a stored copy of the page is dropped. IsrRendered gains noStore.
  • x.manifest.json carries an isr route's onInvalidate, maxStale and query. onInvalidate is written for every isr route, so x manifest diff reports a route flipping 'purge' back to 'stale', a changed bound, or a dropped query declaration (internal, each on its own path). Regenerate the manifest: x manifest.
  • singleFlight().evict(key) and .keys() (@ultimat3/core). Free a key so the next run leads a fresh load instead of joining one already known to be wrong; the evicted load keeps its own callers.
  • registerRevalidator(byPath, byTags?). @ultimat3/cache's fan-out also asks the ISR holder which pages its store holds under the busted tags (TagRevalidator) and reports them in report.isr — see the shared-store fix below.
  • The reference app withdraws a post. examples/dummy gains withdrawPost (published → draft), its blog routes declare ttl: '5m', onInvalidate: 'purge' and query: [], its pricing page query: ['currency'] (apps/web/e2e/pricing-currency.e2e.test.ts: two currencies are two documents), and apps/web/e2e/blog-locales.e2e.test.ts proves a withdrawal answers 404 for the article and drops the card from the index, in both locales and under a tracking parameter, on the very next request.

Fixed

Tier 0 — core. Tier 1 — cache. Tier 3 — realtime. Tier 5 — cli.

  • A page held open through a NATS outage is live again without a reload. While a sync node's bus is away it answers the page's heartbeat — the subscribe re-sent every 10 s for each channel — with an ack carrying X_TRANSPORT_UNAVAILABLE, because the presence write under the beat cannot land. The client took every refusal as a decision: the membership went failed and nothing cleared it, so when the bus returned events reached the page again while useChannel() still read failed, until a reload. A new tab was live at once. X_TRANSPORT_UNAVAILABLE is now the one transient refusal (client-channels.ts, isTransientRefusal): the membership reads joining, error() stays undefined, the beat keeps re-asking as it always did, and the node's first answer — the room's roster, a records frame, a replay-gap — makes it live. A page opened during the outage recovers the same way. Every other refusal is still terminal: a policy denial or an undeclared channel (X_TOPIC_FORBIDDEN), a cap, an auth failure. Presence rides the same frames and comes back with the roster; with one socket per origin the SharedWorker's own beat is the re-ask and every tab holding the topic is told. A live query is not on this path — its subscribe reads the database and never the bus, and the beat re-sends channels only — so its ack handling is unchanged. Nothing to change in an app; one that rendered failed as a dead end sees joining instead. Realtime → A bus outage, seen from an open page.
  • A KV request made with no live connection is refused at once. publish was (27.7.0); request was not: the nats client buffers a request made while it reconnects and lets it time out, so on a sync node whose bus was away every presence write held its socket's frame lane for the 5 s request timeout before saying what the client already knew. The in-memory bus the unit suites run on always refused it — the real client now agrees (nats-lib-client.ts). Found by the black-hole test below.
  • A NATS that accepts the socket and then goes silent is detected in 30 s, and that is now measured. nats-black-hole.live.test.ts puts a publisher and a sync node with a seated page behind a relay that stops forwarding — sockets open, no byte either way — against a real server: both connections are given up 30.0 s after the silence begins (10 s pings, two allowed unanswered; the library's default is six minutes). Inside the window a publish is taken and lost, and the node already refuses the page's beat once its presence write times out; after it both are refused at once; the page reads joining throughout and is live on the same socket when the relay forwards again.
  • An outage is logged as one fact, at a level that says what the process can still do. Each line below is written on its 1st, 2nd, 4th, 8th, … failure and once more when it ends (outageLog, isOutageMilestone on @ultimat3/core). warn is a process that is degraded and serving; error is one that cannot do its job. A sync node wrote presence.sweep failed at error every 10 s for the whole outage (19 lines in three minutes on one node) and sent the error monitor one event per refused beat per socket: both are now sync.bus_unavailable (error, with operation and failures), one count for the node, the monitor thinned with the log, and sync.bus_unavailable recovered (after) when the bus answers. nats transport error was thinned for the background dial only and always error: it now counts every failure of the connection — the drop, each dial — carries failures, is warn on web, worker and scheduler and error on sync and the replicator (NatsTransportOptions.outageLevel), and ends with nats transport recovered (after); the library giving up on a client is still written every time, once per client. cache.invalidate.partial was written once per refused bust on every publisher: it is thinned per failing tier, stays warn, carries failures, and cache.invalidate.partial recovered (tier, after) ends it — every bust's report still carries its errors and /_x still records every bust. cache.broadcast.subscribe-failed gains its closing … recovered (after). A failure that is not the bus being away is written every time, as before. An alert keyed on the text presence.sweep failed for a bus outage must move to sync.bus_unavailable. Realtime → What an outage logs.
  • x dev no longer goes down for the bus. It runs every role in one process and took the strictest reading: with realtime on and NATS down the sync role's 15 s refusal ended the whole boot, so no page was served; and NATS lost under a running one failed plain /readyz. x dev now waits 3 s (DEV_BUS_WAIT_MS), logs the refusal a container would have exited on once at warn — ultimate bus is away: x dev serves without live until it answers, with its code, cause and fix — and boots: pages, jobs and tasks run, the dial goes on in the background, and the sync node stays mounted and not ready (an upgrade is shed with a 503 and a retry delay) until the bus answers, then starts with no restart (sync node started: the bus answered). checks.transport is degraded there, so /readyz stays 200. A container is unchanged: ROLE=sync and the replicator keep the 15 s X_TRANSPORT_UNAVAILABLE, having nothing else to serve.
  • New names, nothing BREAKING. outageLog, isOutageMilestone, OutageLog, OutageLogOptions (@ultimat3/core); NatsTransportOptions.outageLevel, PresenceRegistry.rooms, FrameRouterOptions.busAnswered (@ultimat3/realtime/server). startServices takes a sixth, optional dev argument and RunningServices an optional busTolerated.

Tier 1 — cache. Tier 4 — render. Tier 5 — cli.

  • An isr page never stores a 5xx. A load answering withStatus(503, …) — the app's "the read failed, try again" — was stored with its document for the whole ttl plus one stale serve, and replaced the good page it was regenerating: one upstream blip was minutes of 503. A 5xx render is now never written and never replaces a stored page: while a good copy exists it keeps answering, stale, and isr.render.unstored is logged; with none, the 5xx answers its own request private, no-store with no purge key. For one second (isrController({ failureCooldownMs })) requests for that page are answered the same failure rather than each rendering again; then the next one renders. A render that throws was already never stored (the last good page, else the error page) and is now covered by tests. A 2xx or 4xx is stored as before.
  • A tag bust purges the edge last. invalidateTags cleared tiers farthest first with the cdn tier among them, so the CDN was purged before the origin dropped its own page: a request in between was answered the old page as a public hit, and the edge — already purged — kept it for a whole s-maxage. The order is now the read tiers (farthest first, as before) → the ISR pages → the broadcast to peers → the cdn tier. A page rendered from rows read before a bust is answered private, no-store with no purge key. When the broadcast is refused (the bus is away) the edge is purged with the bust and again when the deferred broadcast is finally published (purgeEdgeAgain): the peers only drop their copies then, and whatever they answered in between had gone back to the edge. A flush (flushProcessTiers, a peer's flush-all, a replica that was deaf) is at least a purge: it asks the ISR holder for every page it holds (EVERY_TAG), so a 'purge' route's pages are deleted rather than marked stale, and it purges the edge last by the tags of the ISR pages that replica held.
  • controller.revalidateByTags() fences by tag. Called directly it marked only the pages it found, and a cold page whose first render was in flight could be in neither the graph nor the store (another page's reconciliation had forgotten it): the bust returned [] and the pre-bust render was stored. It now voids the tags' fence first, and a page with a render in flight is never forgotten.
  • A tag bust reaches a page another controller rendered into a shared IsrStore. The invalidation graph only held the pages a process rendered itself, so a replica that had merely served a hit for an entry in a runtime.isrStore — or any process after a restart over a store that outlived it — found nothing to invalidate while report.isr read clean. A bust now also asks the store which keys it holds under the tags, by the route table (no route carrying the tag: no store read), and a hit registers the page in the graph.
  • /sitemap.xml and /robots.txt keep the last good answer when prerender() fails. The running web role forgot its memoised sitemap before re-enumerating at the hourly refresh, so a failed data read answered a crawler a 500. The hourly refresh now runs BEHIND the previous answer, which every request keeps getting meanwhile — so a read that hangs blocks nobody; a failed one, or one with no answer in 30 s, is logged (seo.enumeration.kept, an error from the third in a row) and asked again a minute later. A first enumeration that fails still fails the request. A prerender() that catches its own failed read and returns [] still shrinks the sitemap: let it throw.

Commits

  • fix(realtime): a channel held open through a bus outage is live again without a reload; an outage is logged as one fact, thinned, at the role's level; x dev serves without the bus; a silent NATS is detected in 30 s, measured (#759)
  • feat(render): isr invalidation can purge, a route declares its keyed query, a failed render is never stored (#757)