Repository navigation
v27.8.0
Added
Tier 0 — core. Tier 1 — cache. Tier 4 — manifest, render. Tier 5 — cli. Both tracked apps.
revalidate.onInvalidate: 'purge'— a tag bust takes anisrpage down. A bust marked the stored page stale and the next request was answered that copy, however old, per stored key and per replica: a withdrawn article reached its next visitor — a crawler, days later — once more.revalidate: { tags, ttl, onInvalidate: 'purge' }deletes every stored page of the route carrying the tag (each locale, each keyed query, stored 404s included) and evicts any render of them in flight, so a request after the bust renders and waits — it never joins a render that began before it, and one that had already joined renders again. The bound: the next request on the replica that ran the bust, and on every replica that receives the invalidation broadcast; a replica that misses it serves its copy untilttland nothing stale after, which is why'purge'requires attlas well astags(X_ROUTE_MODE_INVALID). The response carries nostale-while-revalidateand ans-maxageof what is left of thettl, so an edge cannot outlive it either.'stale'is the default and today's behaviour. A replica still on an earlier release treats a purge as'stale': finish the rollout first.revalidate.maxStale— a bound on a copy served past itsttl.revalidate: { ttl: '10m', maxStale: '1h' }: older thanttl + maxStale, a stored page is dropped and the request waits for a render instead of being answered a copy of any age;0is "never serve stale". It is also thestale-while-revalidatea shared cache is told (a day when omitted). Unbounded by default —0under'purge'— and needs attl. Past the bound the copy is dropped before the re-render, so a render that fails there answers its 5xx.IsrStore.tagFence— a shared ISR store that refuses a purged page. A process's own fence only knows the busts it heard, so over a store two replicas write, one whose render began before a purge wrote the page straight back. A store may now carry a generation per tag entity (sample/bump/ an atomicsetIfCurrent); the controller reads it before a render and writes through it, and a bust moves it.memoryIsrStoreimplements it. Aruntime.isrStorewithout one is refused at boot when any route purges (X_ROUTE_MODE_INVALID, naming the route).revalidate.query— the query anisrpage is keyed on is the route's to declare.revalidate: { query: ['page'] }(or[]for none): only the listed parameters are part of the stored page's key and of theurl/querythatload,metaand the page receive — one value each, sorted — so/blog?utm_source=…is the/blogpage, with/blogas itscanonical, and no query string mints an entry. Nothing is redirected. The default is unchanged: a route that declares noquerystill keys on the whole query string, where/blog?x=1,?x=2, … are each a render and a stored page;x verifynames every such route (theseostep'swarnings) and the server logsisr.query.undeclaredonce per route. Declare it on everyisrroute — the default becomes[]in the next major. A declaration that OMITS a parameter the page reads is one stored document for every value of it, sox verifyalso warns for aquery.<name>/searchParams.get('<name>')read in the route module that the declaration does not list, and outside production thequerya page receives logsisr.query.unkeyed-readonce for such a read.RouteDescriptorgains optionalrevalidateOnInvalidate,revalidateMaxStale,revalidateQuery;@ultimat3/render/serverexportsisrRequestUrlandundeclaredQuery.noStore(data)— a loader's "do not keep this answer". From@ultimat3/render, a mark on the data likewithStatusand composable with it. Anisrrender so marked is servedprivate, no-storeand never written to the store; a stored copy of the page is dropped.IsrRenderedgainsnoStore.x.manifest.jsoncarries anisrroute'sonInvalidate,maxStaleandquery.onInvalidateis written for everyisrroute, sox manifest diffreports a route flipping'purge'back to'stale', a changed bound, or a droppedquerydeclaration (internal, each on its own path). Regenerate the manifest:x manifest.singleFlight().evict(key)and.keys()(@ultimat3/core). Free a key so the nextrunleads a fresh load instead of joining one already known to be wrong; the evicted load keeps its own callers.registerRevalidator(byPath, byTags?).@ultimat3/cache's fan-out also asks the ISR holder which pages its store holds under the busted tags (TagRevalidator) and reports them inreport.isr— see the shared-store fix below.- The reference app withdraws a post.
examples/dummygainswithdrawPost(published → draft), its blog routes declarettl: '5m',onInvalidate: 'purge'andquery: [], its pricing pagequery: ['currency'](apps/web/e2e/pricing-currency.e2e.test.ts: two currencies are two documents), andapps/web/e2e/blog-locales.e2e.test.tsproves a withdrawal answers 404 for the article and drops the card from the index, in both locales and under a tracking parameter, on the very next request.
Fixed
Tier 0 — core. Tier 1 — cache. Tier 3 — realtime. Tier 5 — cli.
- A page held open through a NATS outage is live again without a reload. While a
syncnode's bus is away it answers the page's heartbeat — thesubscribere-sent every 10 s for each channel — with anackcarryingX_TRANSPORT_UNAVAILABLE, because the presence write under the beat cannot land. The client took every refusal as a decision: the membership wentfailedand nothing cleared it, so when the bus returned events reached the page again whileuseChannel()still readfailed, until a reload. A new tab wasliveat once.X_TRANSPORT_UNAVAILABLEis now the one transient refusal (client-channels.ts,isTransientRefusal): the membership readsjoining,error()staysundefined, the beat keeps re-asking as it always did, and the node's first answer — the room's roster, arecordsframe, areplay-gap— makes itlive. A page opened during the outage recovers the same way. Every other refusal is still terminal: a policy denial or an undeclared channel (X_TOPIC_FORBIDDEN), a cap, an auth failure. Presence rides the same frames and comes back with the roster; with one socket per origin the SharedWorker's own beat is the re-ask and every tab holding the topic is told. A live query is not on this path — its subscribe reads the database and never the bus, and the beat re-sends channels only — so itsackhandling is unchanged. Nothing to change in an app; one that renderedfailedas a dead end seesjoininginstead. Realtime → A bus outage, seen from an open page. - A KV request made with no live connection is refused at once.
publishwas (27.7.0);requestwas not: thenatsclient buffers a request made while it reconnects and lets it time out, so on asyncnode whose bus was away every presence write held its socket's frame lane for the 5 s request timeout before saying what the client already knew. The in-memory bus the unit suites run on always refused it — the real client now agrees (nats-lib-client.ts). Found by the black-hole test below. - A NATS that accepts the socket and then goes silent is detected in 30 s, and that is now measured.
nats-black-hole.live.test.tsputs a publisher and asyncnode with a seated page behind a relay that stops forwarding — sockets open, no byte either way — against a real server: both connections are given up 30.0 s after the silence begins (10 s pings, two allowed unanswered; the library's default is six minutes). Inside the window a publish is taken and lost, and the node already refuses the page's beat once its presence write times out; after it both are refused at once; the page readsjoiningthroughout and isliveon the same socket when the relay forwards again. - An outage is logged as one fact, at a level that says what the process can still do. Each line below is written on its 1st, 2nd, 4th, 8th, … failure and once more when it ends (
outageLog,isOutageMilestoneon@ultimat3/core).warnis a process that is degraded and serving;erroris one that cannot do its job. Asyncnode wrotepresence.sweep failedaterrorevery 10 s for the whole outage (19 lines in three minutes on one node) and sent the error monitor one event per refused beat per socket: both are nowsync.bus_unavailable(error, withoperationandfailures), one count for the node, the monitor thinned with the log, andsync.bus_unavailable recovered(after) when the bus answers.nats transport errorwas thinned for the background dial only and alwayserror: it now counts every failure of the connection — the drop, each dial — carriesfailures, iswarnonweb,workerandscheduleranderroronsyncand the replicator (NatsTransportOptions.outageLevel), and ends withnats transport recovered(after); the library giving up on a client is still written every time, once per client.cache.invalidate.partialwas written once per refused bust on every publisher: it is thinned per failing tier, stayswarn, carriesfailures, andcache.invalidate.partial recovered(tier,after) ends it — every bust's report still carries itserrorsand/_xstill records every bust.cache.broadcast.subscribe-failedgains its closing… recovered(after). A failure that is not the bus being away is written every time, as before. An alert keyed on the textpresence.sweep failedfor a bus outage must move tosync.bus_unavailable. Realtime → What an outage logs. x devno longer goes down for the bus. It runs every role in one process and took the strictest reading: with realtime on and NATS down thesyncrole's 15 s refusal ended the whole boot, so no page was served; and NATS lost under a running one failed plain/readyz.x devnow waits 3 s (DEV_BUS_WAIT_MS), logs the refusal a container would have exited on once atwarn—ultimate bus is away: x dev serves without live until it answers, with itscode,causeandfix— and boots: pages, jobs and tasks run, the dial goes on in the background, and thesyncnode stays mounted and not ready (an upgrade is shed with a 503 and a retry delay) until the bus answers, then starts with no restart (sync node started: the bus answered).checks.transportisdegradedthere, so/readyzstays 200. A container is unchanged:ROLE=syncand the replicator keep the 15 sX_TRANSPORT_UNAVAILABLE, having nothing else to serve.- New names, nothing
BREAKING.outageLog,isOutageMilestone,OutageLog,OutageLogOptions(@ultimat3/core);NatsTransportOptions.outageLevel,PresenceRegistry.rooms,FrameRouterOptions.busAnswered(@ultimat3/realtime/server).startServicestakes a sixth, optionaldevargument andRunningServicesan optionalbusTolerated.
Tier 1 — cache. Tier 4 — render. Tier 5 — cli.
- An
isrpage never stores a 5xx. AloadansweringwithStatus(503, …)— the app's "the read failed, try again" — was stored with its document for the wholettlplus one stale serve, and replaced the good page it was regenerating: one upstream blip was minutes of 503. A 5xx render is now never written and never replaces a stored page: while a good copy exists it keeps answering, stale, andisr.render.unstoredis logged; with none, the 5xx answers its own requestprivate, no-storewith no purge key. For one second (isrController({ failureCooldownMs })) requests for that page are answered the same failure rather than each rendering again; then the next one renders. A render that throws was already never stored (the last good page, else the error page) and is now covered by tests. A 2xx or 4xx is stored as before. - A tag bust purges the edge last.
invalidateTagscleared tiers farthest first with thecdntier among them, so the CDN was purged before the origin dropped its own page: a request in between was answered the old page as a public hit, and the edge — already purged — kept it for a wholes-maxage. The order is now the read tiers (farthest first, as before) → the ISR pages → the broadcast to peers → thecdntier. A page rendered from rows read before a bust is answeredprivate, no-storewith no purge key. When the broadcast is refused (the bus is away) the edge is purged with the bust and again when the deferred broadcast is finally published (purgeEdgeAgain): the peers only drop their copies then, and whatever they answered in between had gone back to the edge. A flush (flushProcessTiers, a peer's flush-all, a replica that was deaf) is at least a purge: it asks the ISR holder for every page it holds (EVERY_TAG), so a'purge'route's pages are deleted rather than marked stale, and it purges the edge last by the tags of the ISR pages that replica held. controller.revalidateByTags()fences by tag. Called directly it marked only the pages it found, and a cold page whose first render was in flight could be in neither the graph nor the store (another page's reconciliation had forgotten it): the bust returned[]and the pre-bust render was stored. It now voids the tags' fence first, and a page with a render in flight is never forgotten.- A tag bust reaches a page another controller rendered into a shared
IsrStore. The invalidation graph only held the pages a process rendered itself, so a replica that had merely served a hit for an entry in aruntime.isrStore— or any process after a restart over a store that outlived it — found nothing to invalidate whilereport.isrread clean. A bust now also asks the store which keys it holds under the tags, by the route table (no route carrying the tag: no store read), and a hit registers the page in the graph. /sitemap.xmland/robots.txtkeep the last good answer whenprerender()fails. The running web role forgot its memoised sitemap before re-enumerating at the hourly refresh, so a failed data read answered a crawler a 500. The hourly refresh now runs BEHIND the previous answer, which every request keeps getting meanwhile — so a read that hangs blocks nobody; a failed one, or one with no answer in 30 s, is logged (seo.enumeration.kept, an error from the third in a row) and asked again a minute later. A first enumeration that fails still fails the request. Aprerender()that catches its own failed read and returns[]still shrinks the sitemap: let it throw.
Commits
- fix(realtime): a channel held open through a bus outage is live again without a reload; an outage is logged as one fact, thinned, at the role's level; x dev serves without the bus; a silent NATS is detected in 30 s, measured (#759)
- feat(render): isr invalidation can purge, a route declares its keyed query, a failed render is never stored (#757)