Opt-in request batching for server functions

[brenelz]

A page that mounts several independent reads sends one HTTP request per server function call. Over HTTP/1.1 the browser runs at most six at a time per host, so the seventh read waits for a free connection, and every request pays its own per-request cost on the server: session lookup, auth middleware, and serverless invocation. I prototyped opt-in batching that sends reads issued together as one request while each answer still streams back to its caller separately. I'd like feedback on the shape before going further.

What it looks like

A function opts in at its declaration, and the app opts in once on the client:

import { withMeta } from "@solidjs/web/server-functions";

export const getUser = withMeta(
  async (id: string) => {
    "use server";
    return db.users.find(id);
  },
  { batch: true }
);

import { enableBatching } from "@solidjs/web/server-functions/batch";

enableBatching();

No server code is needed: handleServerFunctionRequest answers <endpoint>/batch itself. Calls are written and consumed exactly as before.

How it works

• The client groups calls to declared functions that are issued in the same microtask into one POST to /_server/batch. A lone call still goes to its own data address. The response is a framed stream: each member's status, headers, and body chunks are relayed as they happen, so every caller gets its own Response, a fast call never waits for a slow one, and a streamed result still streams.
• The server re-enters handleServerFunctionRequest once per member, as a real Request with the outer request's options and its Cookie, Origin, and Sec-Fetch-* headers. A member cannot supply its own. CSRF gating, createEvent, and wrapInvocation therefore apply per call exactly as they do unbatched. Response metadata that lives in headers (redirects, the unknown-function label, single-flight data) reaches the caller unchanged.
• The server admits a member only when its registered function carries { batch: true } and is not GET-declared. The declaration is read from the binding it was made about, the same rule GET follows (A GET() declaration outlives the function it was made about #3129), so a rebound id does not inherit it. Without this check, a hand-built batch of 32 calls to a login function would get around rate limits keyed on that function's URL.
• enableBatching() follows the enableRichArguments() pattern: it is its own entry, and apps that never import it carry none of the batching code. The core transport holds only the small hook it installs into.

Why a declaration and not "batch everything"

Browsers apply Set-Cookie only from a real HTTP response, so a batched response cannot deliver cookies. If every POST joined a batch, login, logout, and session-refresh functions would break, but only when they happened to coincide with another call. That makes for an intermittent bug. With the declaration, only a function's author can make it batchable, and a declared function that sets a cookie fails with an error naming it. It is never re-sent, because it has already run.

GET(fn) reads stay out even when marked, so they keep their cacheable URL. If the server has no batch route (a 404 or 405), nothing ran, and each call is sent again on its own.

I also looked at inferring reads from the call site: batch calls made while a computation is tracking, and never from actions. It is feasible, but it misses router preloads, which are the biggest bursts, and it brings back the intermittent cookie failure.

Numbers

These come from a demo dashboard with 12 widgets plus one streamed feed, all loaded in the same tick. Each HTTP request pays a simulated 250 ms per-request cost, and the numbers are from Chromium against the dev server and a production preview:

	batching off	batching on
HTTP requests	13	1
last widget ready	~1,670 ms	~1,190 ms
streamed feed finished	~2,900 ms	~2,050 ms

Over HTTP/2 the connection limit goes away. A Node benchmark without it measured 75 ms against 73 ms for 12 reads, so what remains is one request's worth of server overhead instead of twelve. Large results pay for the framing: a 4 MiB result was 15–40% slower batched.

Questions

1. Is this worth having in core, or should it stay a userland transport? It can be built outside core with the existing fetch and prepareRequest hooks, but only with both hooks taken over, and without the server-side declaration check.
2. Should the declaration be applied by data layers instead of per function? A router's query() wraps reads by definition, preloads included, so it could mark what it wraps and apps would batch without annotating anything.
3. Is failing a cookie-setting declared function the right behavior, or should the batch hold its response headers until every member has answered, so cookies can be moved onto the outer response? That costs the fast-call-first property.
4. withMeta is currently typed (fn: any, meta: any): any, so wrapping a function erases its type. That's worth fixing either way.

The prototype has 17 tests covering the behavior above, plus a demo app with a request waterfall. I can open it as a draft PR if the direction looks right.

[brenelz]

Follow-up after reading the server-components design notes, which already cover part of this.

The server-component request grouping seed describes the same idea for server components: N same-tick invocations could ride one request, since the response side already multiplexes. The prototype follows the rule stated there, that the batch shares the pipe but never the completion, and avoids the trap it names: the batch response is never buffered, so each member's answer streams as soon as it starts.

That seed is on hold because the heavy cases are covered (t=0 by the document, mutations by single-flight), and because the persistent connection in Stage 8 would make the question moot. Stage 8 isn't built yet, and client-driven reads after load, such as navigation, filters, or widgets mounting together, aren't covered by the document or by single-flight. That is the case this targets. If Stage 8 lands, batching could ride its connection instead of competing with it.

The router-side data batching note is a different kind of batching: one function called with many arguments, coalesced into one call in the router, with nothing on the wire. This proposal is about different functions sharing one request, so the two can coexist.

A correction to my post: the latency numbers were measured against Vite's dev server, which is HTTP/1.1. In production over HTTPS, browsers use HTTP/2 or HTTP/3, the six-connection limit doesn't apply, and the latency gain mostly disappears (75 ms against 73 ms for 12 reads in a local benchmark without the limit). What batching still saves there is per-request server work: one pass through session lookup, auth, and middleware instead of N, and one serverless invocation instead of N. I haven't measured that on a real host yet, and that measurement is what should decide whether this is worth doing.

One design question the notes raise: the grouping seed assumes grouped calls share one request scope, so siblings awaiting the same query dedupe. The prototype instead runs each member as its own request with its own event, which keeps CSRF gating, createEvent, and wrapInvocation per call. Sharing a scope would bring back request-scoped dedupe but make per-call policy harder to reason about. Which would you prefer?

[ryansolid]

Thanks for this — the prototype follows the §9.4 principle where it matters (shares the pipe, never the completion; nothing buffered), and the correction in your follow-up is the right one. A few things, roughly in order of weight, starting with the one that sets the bar.

Batching pulls against the read model we've chosen. RFC 10's position is that a read is its URL: GET, cacheable, preloadable, keyed by address. The runtime is built to push reads there — router query() and liveQuery() declare GET on the author's behalf, and GET is the idiom for server components. Your admission rule excludes GET, correctly: per-URL caching and many-per-request batching are mutually exclusive per call, and choosing between them by whether two calls happened to coincide would make caching intermittent (the same smell you rightly avoided with cookies). But that means in a well-declared app the batchable set is POST reads — a category the runtime is actively shrinking. So either the feature serves very little, or it admits GET reads and URL identity is given up on coincidence. Question 2 makes this concrete: query() would mark what it wraps and the server would then refuse it.

That's not "the prototype is wrong." It's that these are two read models and core has already picked one. The bar for this is therefore not "does it save something" but "does it justify a second read model in core." If the real answer is "the router should choose batch over GET for its own reads," that's a router design question that has to be argued on its own terms first — and it's the data layer's deliberate choice, never inferred from timing. I want to be upfront about that now rather than after you've done the measuring.

The Stage 8 premise in the seed is gone, which helps your case. The grouping seed was held partly because "Stage 8's persistent connection dissolves the question." Stage 8 is settling as one event-stream response per live source, nothing shared between them — so there's no persistent pipe for batching to ride, and nothing there addresses coincident bounded reads. Your target (post-load client-driven reads) is genuinely uncovered by the document face, single-flight, or Stage 8. §9.4 needs that correction regardless of what happens here.

Per-member is right, and it decides what the saving is. Your closing question — shared request scope vs. one real Request per member — is the proposal, not a detail. Per-member is correct by the principle you're already following: co-occurrence shares the connection, never semantics; a shared request scope (dedupe across unrelated callers, one createEvent for calls that merely landed in the same tick) asserts a relation nobody declared. Keep it.

But per-member means everything the runtime does per request — createEvent, wrapInvocation, CSRF gating, session lookup inside any of them — still happens N times. What one outer request saves is the HTTP round trip (which HTTP/2 already removes, as you found) and one serverless invocation instead of N, plus whatever middleware the host runs before dispatch. So the case rests on "one invocation vs. N on a real serverless host." You said that measurement should decide; I agree. One consistency note: we recently declined to add surface for platforms that cap response duration, on the grounds that a platform's cost model is the deployment's to solve. A serverless invocation count is the same shape of argument, so the number would have to be large enough to say why this one is different.

Cookies (question 3): the error is right; holding headers is not. Holding the outer response until every member has answered converts shared-connection into atomic-completion — the trap the seed names. One refinement: a read that sets a cookie conditionally (a session refresh) fails only sometimes under your rule, which is the bug class you set out to remove. State the declaration as "reads that touch no response headers" and make the failure loud in dev.

Core vs. userland (question 1): core if it lands, for one reason. The server-side admission check — only functions declared batchable on the binding they were declared about, so a hand-built batch can't multiply calls to a rate-limited function — is a security property only the registry can enforce. The client half following enableRichArguments() (its own entry, a small hook) is the right cost model. Three things a draft would have to answer:

• Abort. invoke(fn, { signal }) aborts a call by cancelling its request. A member has no request of its own, and the registry's rule is that a caller's signal can't own a wire other callers are reading. Either abort is silently different under batching, or the batch grows a control channel.
• prepareRequest. Relaying only the outer Cookie/Origin/Sec-Fetch-* and refusing member headers drops the documented withMeta({ requiresAuth }) → prepareRequest → Authorization pattern, or applies one member's headers to all. Per-call transport middleware has to compose.
• Framing. Don't add a third envelope beside length-prefixed and event-stream. Members are records with an id, which the codec's stream already expresses.

Smaller. The dev waterfall you measured is HTTP/1.1 Vite; server.https gives the dev server HTTP/2 and removes it without batching. Question 4 — withMeta typed any — is a standalone fix and a PR for that is welcome now. And "same microtask" is a fragile grouping window against Solid's own scheduling; if this proceeds, define it in terms of the runtime's flush.

So: principle followed, prototype careful. The honest version of the case is one serverless invocation vs. N on a real host, for POST reads only, weighed against carrying a second read model in core. That's a high bar, and I'd rather say so now than after the work.

— Claude via Cursor