feat: async render() + component-level Suspense (bare-await data fetch) by vivek7405 · Pull Request #470 · webjsdev/webjs

vivek7405 · 2026-06-10T23:57:48Z

Summary

Closes #469
Closes #471
Closes #473

Adds bare-await async render plus the full component-level Suspense stack to webjs. A component writes the natural const data = await getData() inside render() and it resolves server-side, co-located, with no page orchestration; slow regions stream via <webjs-suspense>, progressively on soft navigation too. This is a clean breaking change to the render contract (webjs is pre-1.0): render() may now be async, and every render path accounts for it.

The model (decoupled, do not conflate)

SSR always blocks by default so the resolved DATA is in the first paint, no fallback markup (PE-safe; JS-off reads it).
The client re-fetch default is stale-while-revalidate. A prop change re-runs async render(); the current content stays until the new render resolves. A monotonic render token drops a superseded resolution (sync or async).
renderFallback() is the optional re-fetch loading UI (re-fetch only, never first paint, never SSR streaming).
Per-component error isolation is a default (no user code): a thrown await getData() renders a component-scoped error state while siblings render. renderError() customizes it.
<webjs-suspense .fallback=${...}> streams a slow region (feat: <webjs-suspense> streaming SSR boundary + per-component streaming (follow-up to #469) #471): fallback on the first byte, data streams in, concurrent across boundaries, progressive on soft navigation (feat: progressive soft-nav streaming in the client router (follow-up to #469) #473), error-isolated.

What is in this PR

Client async render (component.js): update() routes a promise-returning render() to a stale-while-revalidate commit, a per-commit __renderToken race guard, rejection to renderError(); _performRender defers the post-commit half (incl. an in-flight counter so a non-committing cycle never resolves updateComplete early). New renderFallback() hook.
SSR error isolation (render-server.js): the per-component catch renders renderError() (or a dev box / silent prod element), wrapped in a DSD template for shadow components; siblings render.
<webjs-suspense> streaming (feat: <webjs-suspense> streaming SSR boundary + per-component streaming (follow-up to #469) #471): renderTemplate captures .fallback to data-webjs-fallback; injectDSD's processSuspenseElements pre-pass streams the boundary (concurrent via Promise.all) or renders inline when not streaming. A rejected boundary renders an error instead of hanging on the fallback. Client element is layout-neutral.
Progressive soft-nav streaming (feat: progressive soft-nav streaming in the client router (follow-up to #469) #473): the SSR stream flushes a  sentinel with the shell; the router's readStreamedShell swaps the shell in immediately and streamBoundariesProgressively applies each boundary as it streams, guarded by the nav token.
Elision (component-elision.js): an async-render component is always shipped.
Docs across every surface (see below).

Deferred (filed, with a written architectural analysis)

feat: seed SSR action results into hydration so async render does not re-fetch (follow-up to #469) #472 (seed SSR action results into hydration to skip the redundant load-time RPC). A transparent seed needs a Node ESM loader hook to wrap server-action exports for SSR-time result capture, which is out of webjs's no-build / raw-ESM-SSR thesis and risky; the skip-static and request-coalescing alternatives each have correctness traps (signals/events; collapsing distinct writes). Stale-while-revalidate (feat: async render() + component-level Suspense (bare-await data fetch) #470 in this PR) already makes the redundant RPC invisible. Full analysis is on the issue. Moved back to Todo.

Test plan

Unit: async-render SSR + error isolation; <webjs-suspense> streaming (fallback-first, concurrency timing, error isolation); the progressive-stream reader helpers (sentinel / </html> / nested-template). Counterfactuals for the sync-supersession race and the updateComplete early-resolve.
Browser: client async render (SWR, renderFallback re-fetch-only, error isolation, two race guards, updateComplete ordering); <webjs-suspense> client; progressive streaming apply + supersede guard.
E2E: progressive soft-nav streaming (fallback live at URL-advance), full blog suite.
Full suites: 2380 node, 359 browser.

Dogfood

Blog e2e (reported below); website / docs / ui-website boot 200 in dist mode (rebuilt against this core), no broken modulepreloads. webjs-core-browser.js is ~26.4 KB gzipped (small delta; the async-render + streaming branches ship unconditionally, no lazy-split needed).

Docs surfaces updated

Root + packages/core AGENTS.md (public API table incl. <webjs-suspense>, the module map), agent-docs/{components,advanced,recipes,lit-muscle-memory-gotchas,testing}.md, the docs-app pages (NEW data-fetching decision guide registered in the sidebar, plus suspense / loading-states / client-router / lifecycle / components / ssr / error-handling, which feed the live llms.txt corpus), the scaffold per-agent configs, the website marketing pillar, the @webjsdev/mcp fetch_data_in_component prompt, and the renderFallback type declaration.

An async (promise-returning) render() suspends on the client: it awaits data, re-renders with the resolved value, reads the SSR seed, and may re-fetch via RPC on a prop change. That is real client work, so an async-render component must never be elided as display-only even though its SSR first paint looks static. Detect `async render(`, the `render = async` arrow field, and the `renderFallback` companion in the elision analyser and force the module to ship. Part of #469 (async render + component-level Suspense).

A render that throws during SSR (most commonly a rejected await getData() in an async render, but any render throw) is now caught per component: the walker loop already continues so siblings render, and the failing element renders a component-scoped error state instead of leaving its raw, unprocessed children in the output or bubbling to the route error.js. renderError() customizes the UI; the default surfaces the tag and message in dev and renders an empty, silent element in prod so no internal detail leaks. SSR already awaits a promise-returning render, so async render bakes resolved data into the first paint with no fallback. Part of #469 (async render + component-level Suspense).

The client renderer now awaits a promise-returning render(). The default is stale-while-revalidate: the current DOM (the SSR first paint on hydration, or the prior content on a re-fetch) stays visible until the new template resolves, so there is no blank or flash and no user code is needed. A monotonic render token drops a superseded resolution so an out-of-order fetch never commits stale DOM, and a rejection routes to the per-component renderError() boundary. renderFallback() is the optional override: on a client re-fetch it swaps in a loading state instead of stale content; it is never shown on the first paint and never triggers SSR streaming. _performRender defers the post-commit half of the cycle (hostUpdated, firstUpdated/updated, updateComplete) until the async commit lands. renderFallback joins the elision lifecycle signal list (an async component must ship). Part of #469.

Add the bare-await async render reference to the framework AGENTS.md (WebComponent essentials), packages/core/AGENTS.md (the component.js lifecycle row), the agent-docs/components.md deep-dive (decision rules, anti-patterns, and the canonical snippets), and a lit-muscle-memory gotcha (Lit has no async render; SSR blocks by default, renderFallback is a re-fetch concern, error isolation is automatic). The streaming wrapper, hydration seed, and progressive soft-nav streaming are noted as follow-ups #471 / #472 / #473. Part of #469.

…rors) (#469) Add the user-facing reference for bare-await async render: a render section in the lifecycle page (async render + renderFallback), a fetching-data decision guide with anti-patterns in the components page, the per-component error-isolation default in the error-handling page, and a first-paint-data note in the ssr page. The docs site serves these into the live llms.txt corpus automatically. Part of #469.

…nfigs (#469) Add the fetch-in-the-component (async render) convention and the renderFallback / Task decision rule to the scaffolded CONVENTIONS.md, AGENTS.md (lit gotchas table), .cursorrules, .agents/rules/workflow.md, and .github/copilot-instructions.md, so every agent a new app ships with (Claude, Cursor, Copilot, Antigravity, Aider) gets the same guidance. Part of #469.

vivek7405

Review: async-render core, first pass

Went over the client update cycle in component.js, the SSR error-isolation catch in render-server.js, and the elision signal. The __renderToken race guard holds (a superseding render bumps the token so a stale resolution drops its commit), _performRender's async branch defers the post-commit half and resolves updateComplete exactly once, and _changedProperties is reset on dispatch so a change during an in-flight fetch starts a fresh superseding cycle without being lost. The sync path is a straight refactor into _handleRenderError / _postCommit with identical logic. On the SSR side the catch computes the closing-tag span correctly for both self-closing and normal custom elements, and the prod default never emits the error message. Nothing to change in this pass.

The __renderToken race guard was stamped only inside _commitAsync, so it guarded an async render superseded by another async render but NOT by a synchronous one: a fast sync cycle committed fresh DOM without bumping the token, then the in-flight async resolution passed its token check and clobbered the fresh DOM back to stale content (and ran updated() twice). Stamp the token once per commit in _performRender (sync and async alike), so any later cycle supersedes an in-flight async render. A shouldUpdate= false cycle does not reach the stamp, so it never invalidates an in-flight render. Also strengthen the race-guard browser test to resolve the NEW render first then the STALE one (the prior order passed with or without the guard), and add an async-superseded-by-sync regression test. Part of #469.

vivek7405

Review: async update-cycle edge cases, one real race

Deep pass on the reactive cycle with async render. Five scenarios check out (updateComplete ordering, async-by-async supersession, disconnect mid-flight, the renderFallback first-paint guard, the signal-watcher interaction). One real hole: the render token was bumped only on the async commit path, so an async render superseded by a SYNCHRONOUS render kept a matching token and clobbered the fresh sync DOM when it resolved late (and ran updated() twice). The race-guard test also resolved stale-first, so it passed with or without the guard. Both addressed in the follow-up commit.

#469) 1. updateComplete could resolve early when a non-committing cycle (shouldUpdate=false) ran during an in-flight async render: the sync tail resolved the promise the pending async commit owned. Track in-flight async commits with a counter and skip the non-committing resolve while one is pending, so await el.updateComplete never returns before the async DOM lands. The counter decrements in every async .then, including a superseded one, so it cannot leak. 2. A throwing SHADOW component rendered its SSR error state as light children instead of a DSD template, so the client (which renders the error into the shadow root) left the light error box lingering underneath. Wrap the shadow error state in a shadowrootmode template to match the success path. Both found in the pre-merge review loop. Part of #469.

vivek7405

Review: verify the race fix, plus two error/non-committing-path edge cases

The token fix holds (stamped once per commit in _performRender, sync and async, _commitAsync only reads it, a shouldUpdate=false cycle never re-stamps). While confirming that I found two smaller edge cases worth closing: updateComplete could resolve early if a non-committing cycle runs during an in-flight async render, and a throwing SHADOW component emitted its SSR error state as light DOM instead of a DSD template. Both fixed; everything else (updateComplete ordering, async-by-async supersession, disconnect mid-flight, the renderFallback first-paint guard, the SSR closing-span + prod-no-leak, the docs, the elision guard) checks out.

vivek7405

Review: clean

Re-traced the in-flight-commit counter (no leak, no negative, no deadlock: a sync supersede resolves updateComplete and A's late .then decrements back to zero and bails on the token), confirmed the shadow-error path wraps in a DSD template with the right styles and imports, and swept the whole diff and docs. Nothing left to change.

Adds the explicit SSR-streaming opt-in for async render. renderTemplate captures <webjs-suspense .fallback=${html...}> to a data-webjs-fallback attribute (a TemplateResult is not serializer-safe, so it must not ride the data-webjs-prop path). injectDSD gains a processSuspenseElements pre-pass: in a streaming context each boundary flushes its fallback now and pushes its raw children to ctx.pending (streamed later via the existing <template data-webjs-resolve> + swap-script engine, so multiple boundaries resolve via Promise.all and fetch concurrently); without a streaming context the children render inline (blocking), so a plain renderToString returns real content. A throwing component inside a boundary is isolated by injectDSD's per-component error boundary (#469); a rejected page-level Suspense boundary now renders a boundary-scoped error instead of hanging on the fallback forever. The client <webjs-suspense> element is layout-neutral (display:contents) and the registration home for the soft-nav apply (#473); first-load streaming needs no client runtime (the inline swap script replaces innerHTML and the children upgrade natively). Closes #471.

A soft navigation previously buffered the whole response (resp.text()) before swapping, so a streamed Suspense page showed correct final content but not progressive fallback-then-stream. Now the router reads the response progressively: readStreamedShell returns the shell (up to the first <template data-webjs-resolve>) as soon as it arrives, the existing apply path swaps it in (fallbacks visible, URL advanced), then a detached streamBoundariesProgressively applies each resolved boundary as it streams (fast-before-slow), upgrading the custom elements inside. takeResolveUnit depth-tracks nested templates (a streamed shadow component's DSD). A non-streaming response reads to completion and is byte-identical to the old path. The progressive applier is guarded by the nav token (a newer navigation stops it and cancels the reader) and a mid-stream transport failure is non-destructive (applied boundaries stay, the rest keep their fallback). Closes #473.

…rker (#473) The SSR stream flushes the whole shell (prefix + body with fallbacks + </body></html>) FIRST and appends each boundary template only after its slow data resolves. Delimiting the shell on the first <template data-webjs-resolve> marker therefore blocked the shell swap until the slow boundary arrived, defeating progressive streaming. Delimit on </html> instead so the shell swaps in immediately and the boundaries stream into the live DOM afterward; a fast/buffered boundary already in hand still splits at the marker, and a partial fragment with neither reads to completion (non-streaming).

…ogressive (#473) streamingHtmlResponse flushes the shell, then PAUSES for the slow data before streaming the boundary templates and emitting </body></html> last. So neither the boundary marker nor </html> reaches the client until the slow boundary resolves, and the progressive soft-nav reader had nothing to delimit the shell on, blocking its early swap. Emit a  sentinel in the same chunk as the shell; the router's readStreamedShell splits there and swaps the shell (with fallbacks) in immediately, then streams the boundaries. The sentinel is an inert comment for the native initial-load parse and absent from the buffered (non-streaming) response. Add the e2e (#473) and the sentinel unit case.

#473) Update the now-implemented streaming surface across the framework AGENTS.md (public API table + async-render section), packages/core AGENTS.md (webjs-suspense.js module-map entry + suspense rows), the agent-docs/components.md deep-dive (a full <webjs-suspense> section with grouping, concurrency, error isolation, and progressive soft-nav), and the lit gotcha. Replaces the earlier 'tracked in #471' placeholders now that streaming ships; only the hydration-seed optimization (#472) stays deferred.

, #471, #473) Add the canonical docs/app/docs/data-fetching page (the HOW and WHEN decision guide with the four canonical snippets and the anti-patterns), register it in the sidebar, extend the suspense page with component-level <webjs-suspense>, add the renderFallback re-fetch-loading section to loading-states, the progressive-streaming section to the client-router page, and update the lifecycle + components pages to drop the 'forthcoming' notes now that streaming ships. These feed the live llms.txt corpus automatically.

…py (#471, #473) Add the <webjs-suspense> component-level streaming + progressive soft-nav sections to agent-docs/advanced.md, a fetch-data-in-a-component recipe (the four canonical snippets) to recipes.md, async-component/streaming test guidance per layer to testing.md, and reframe the website's streaming pillar around co-located async render + component Suspense.

…#469) Add a fetch_data_in_component guided-workflow PROMPT to @webjsdev/mcp so an agent introspecting the framework gets the async-render recipe (the decoupled model, webjs-suspense streaming, renderFallback, error isolation), and declare the optional renderFallback() hook in component.d.ts so a TS subclass can type the override. render() already types Promise<TemplateResult>.

, #471, #473) Add a /stream-demo blog page exercising the whole stack: an <async-greeting> component that fetches its data IN an async render() (so the greeting is in the first paint, JS-off readable) plus interactivity to prove hydration, and a slow <slow-fact> wrapped in <webjs-suspense> so its fallback flushes first and the content streams in. Cover it at the server-pipeline layer in the blog smoke test (async-render data in the first paint, the boundary placeholder + fallback, the wj-stream-shell sentinel, the streamed resolve template, and the JS-off PE baseline).

…469, #471, #473) On the /stream-demo fixture: async render data is in the raw SSR HTML (PE, network-probed) and the component hydrates and stays interactive; the <webjs-suspense>-wrapped slow component streams in behind its fallback on initial load; and a progressive soft navigation keeps the boundary fallback live at URL-advance (a buffered swap would already show the resolved content) before the boundary streams in.

…itial load (#473) applyStreamedResolve replaced the boundary's innerHTML, leaving the transient <webjs-suspense> / <webjs-boundary> wrapper in the DOM, whereas the initial-load boot resolver and the prefetched-buffered path both replaceWith the wrapper away. So a structural selector (webjs-suspense > .x) behaved differently after a live progressive soft-nav than after initial load. Converge: applyStreamedResolve now replaces the boundary element with the resolved content (upgrading the custom elements inside), matching the other two paths exactly. Found in review.

vivek7405

Review: streaming half (webjs-suspense + progressive nav), one convergence nit

Went over the streaming work added on top of the async-render core: processSuspenseElements + the .fallback capture, readStreamedShell / takeResolveUnit / streamBoundariesProgressively, the fetchAndApply integration, and the wj-stream-shell sentinel. The depth-tracking, the shell delimiting, the supersede/cancel guards, the prefetch skip, and the sentinel emit/absent split all hold. One real inconsistency: a live progressively-streamed boundary settled to a different DOM shape than the same page reached by initial load or a prefetched-buffered nav (innerHTML kept the wrapper vs replaceWith removing it), so a structural selector could differ. Converged it.

…e comments (#471) The core streamSuspenseBoundaries inline swap script still did b.innerHTML=t.innerHTML (keeping the transient boundary wrapper), while ssr.js's __webjsResolve (the real-app resolver) and the soft-nav applyStreamedResolve both replaceWith it away. So a direct renderToStream consumer settled to a different DOM shape than the server pipeline. Converge the inline script to b.replaceWith(t.content.cloneNode(true)), so every streamed-boundary swap path removes the wrapper. Also restore the findClosingTagInString doc comment that a careless insertion orphaned, and drop the 'reads the SSR seed' phrasing from the elision comment (the seed is deferred, #472). Found in review.

vivek7405

Review: streaming swap convergence + two stale comments

The prior round's applyStreamedResolve fix matches the real-app initial-load resolver (ssr.js __webjsResolve uses replaceWith), but a sharper pass found the CORE streamSuspenseBoundaries inline swap still did innerHTML, so a direct renderToStream consumer settled to a different DOM shape than the server pipeline. Converged it to replaceWith so every swap path removes the transient wrapper. Also fixed a JSDoc block I orphaned when inserting processSuspenseElements, and dropped a 'reads the SSR seed' phrasing from the elision comment (the seed is deferred, #472). All four streaming e2e cases still green after the change.

…ap convergence (#471) The previous commit changed the streamed-boundary swap from innerHTML to replaceWith but left three comments describing the old behaviour: the webjs-suspense.js module doc, the packages/core/AGENTS.md module-map row, and the webjs-suspense-client test header. Update all three to say the swap replaces the boundary element (removing the transient wrapper), and note that every swap path now converges on that shape. Found in review.

vivek7405

Review: swap convergence verified, three comment drifts fixed

Confirmed all three streamed-boundary swap paths (the core inline script, the boot __webjsResolve, and the soft-nav applyStreamedResolve) now replaceWith the wrapper away and converge on the same settled DOM, and the orphaned JSDoc is well-formed. The swap-script change had left three comments still saying innerHTML (the webjs-suspense.js module doc, the packages/core AGENTS.md row, and a test header); corrected all three in dcd23095. No logic change, comments only.

vivek7405

Review: clean

Confirmed all three streamed-boundary swap paths converge on replaceWith (wrapper removed), the corrected comments match, the one 'wrapper stays' note correctly describes the distinct blocking path, and no stale innerHTML descriptions remain. Swept the async-render cycle, streaming SSR, progressive router, sentinel, and the new unit/browser/e2e/smoke tests; nothing left to change.

#470 made async render() a blanket interactivity signal, shipping every async component plus a redundant on-hydration re-fetch. A BARE async leaf (async render() with no other client signal, light DOM) renders identical first-paint HTML with or without JS, so it is elidable like any display-only component. Drop async render() from the standalone-signal short-circuit in analyzeComponentSource. Add two carve-outs that always ship: static shadow (Declarative Shadow DOM only attaches during HTML parsing, so a streamed/soft-navigated shadow component needs its module to re-run attachShadow) and the explicit static refresh = true opt-in (keeps the stale-while-revalidate on-load re-fetch eliding would drop). renderFallback stays a ship signal via CLIENT_LIFECYCLE_HOOKS.

A superseding sync render that throws while an async render is in flight must not hang await el.updateComplete. The render-error boundary catches the throw and the cycle completes as a failed sync commit, which resolves updateComplete via _postCommit; the later-settling stale async render bails on the token check without clobbering. This locks that behaviour in (a fresh-context review hypothesised a hang here; the test proves there isn't one).

#470 made async render() a blanket interactivity signal, shipping every async component plus a redundant on-hydration re-fetch. A BARE async leaf (async render() with no other client signal, light DOM) renders identical first-paint HTML with or without JS, so it is elidable like any display-only component. Drop async render() from the standalone-signal short-circuit in analyzeComponentSource. Add two carve-outs that always ship: static shadow (Declarative Shadow DOM only attaches during HTML parsing, so a streamed/soft-navigated shadow component needs its module to re-run attachShadow) and the explicit static refresh = true opt-in (keeps the stale-while-revalidate on-load re-fetch eliding would drop). renderFallback stays a ship signal via CLIENT_LIFECYCLE_HOOKS.

* feat(elision): elide bare async-render components #470 made async render() a blanket interactivity signal, shipping every async component plus a redundant on-hydration re-fetch. A BARE async leaf (async render() with no other client signal, light DOM) renders identical first-paint HTML with or without JS, so it is elidable like any display-only component. Drop async render() from the standalone-signal short-circuit in analyzeComponentSource. Add two carve-outs that always ship: static shadow (Declarative Shadow DOM only attaches during HTML parsing, so a streamed/soft-navigated shadow component needs its module to re-run attachShadow) and the explicit static refresh = true opt-in (keeps the stale-while-revalidate on-load re-fetch eliding would drop). renderFallback stays a ship signal via CLIENT_LIFECYCLE_HOOKS. * test(elision): bare-async corpus, differential route, import-rule cases Add the /async-leaf blog route rendering <inline-quote> (a bare async-render display-only leaf) to the differential-elision corpus: the module is elided ON, shipped OFF, and the SSR'd quote is byte-identical + first-paint present on both sides. Add import-rule unit cases proving a bare-async parent that imports an interactive child ships (the nested-child fence), and a bare-async parent whose child registers elsewhere is elided while the child still ships. * test(e2e): network-probe the bare-async elision (#474) Two real-browser probes: /async-leaf proves the elided <inline-quote> module is never fetched while the async-baked quote renders JS-off (raw HTML) and JS-on with no page errors; /stream-demo proves the streamed bare-async <slow-fact> is elided (its light-DOM content streams without the module) while its interactive sibling <async-greeting> still ships. * docs: correct the async-render elision story (#474) Drop the 'an async-render component is always shipped (never elided)' claim from root AGENTS.md, agent-docs/components.md, the core AGENTS elision invariant, and the scaffold AGENTS lit-gotcha table. Document that a bare async leaf (no other client signal, light DOM) is elided, the two always-ship carve-outs (static shadow, static refresh = true), and the renderFallback opt-out. Add a 'bare async leaf ships zero JS' section + decision rule + anti-pattern to the data-fetching doc page. --------- Co-authored-by: t <t@t>

vivek7405 self-assigned this Jun 10, 2026

t added 2 commits June 11, 2026 05:32

t added 3 commits June 11, 2026 05:41