fix(v3): chunk large IPC payloads to bypass WebView2 2 MB body limit#5369
fix(v3): chunk large IPC payloads to bypass WebView2 2 MB body limit#5369leaanthony merged 6 commits intomasterfrom
Conversation
WebView2's WebResourceRequested event fails to deliver POST bodies larger than ~2MB via ICoreWebView2WebResourceRequest::GetContent(), causing service calls with large string arguments to fail with a network/CORS- style error on the JS side. Fix: when the serialised request body exceeds 512 KB the JS runtime splits it into 512 KB chunks and sends them serially with coordination headers (x-wails-chunk-id / x-wails-chunk-index / x-wails-chunk-total). The Go HTTPTransport accumulates chunks, assembles them when complete, and processes the reassembled body identically to a single-request call. Partial chunk state is cleaned up after 30 s to prevent memory leaks. Fixes #4428 Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com> Co-authored-by: multica-agent <github@multica.ai>
|
No actionable comments were generated in the recent review. 🎉 ℹ️ Recent review info⚙️ Run configurationConfiguration used: Path: .coderabbit.yaml Review profile: CHILL Plan: Pro Run ID: 📒 Files selected for processing (1)
🚧 Files skipped from review as they are similar to previous changes (1)
WalkthroughClient and bundled runtime now conditionally split large serialized RPC payloads (>512 KiB) into sequential POST chunks with chunk headers; server HTTP transport accumulates chunks keyed by ID, validates ordering, assembles the full body, processes the RPC payload, and runs a background cleanup for stale chunk assemblies. ChangesLarge Payload Chunking for Wails Runtime
Sequence Diagram(s)sequenceDiagram
participant Frontend as Frontend Client
participant Runtime as Runtime Transport
participant Server as HTTP Server
participant Store as Chunk Store
Frontend->>Runtime: invoke(method, large_args)
Runtime->>Runtime: stringify and measure payload
alt Payload exceeds 512 KiB
Runtime->>Server: POST chunk 0 (id, index=0, total=N)
Server->>Store: store chunk 0
Server-->>Runtime: 200 OK
Runtime->>Server: POST chunk 1
Server->>Store: store chunk 1
Server-->>Runtime: 200 OK
Runtime->>Server: POST chunk N (final)
Server->>Store: store chunk N
Store->>Server: assemble chunks in order
Server->>Server: processBody(fullBytes)
Server-->>Runtime: RPC result
else Payload within limit
Runtime->>Server: POST full body
Server->>Server: processBody(bodyBytes)
Server-->>Runtime: RPC result
end
Runtime-->>Frontend: return result
Estimated code review effort🎯 3 (Moderate) | ⏱️ ~25 minutes Suggested labels
Poem
🚥 Pre-merge checks | ✅ 4 | ❌ 1❌ Failed checks (1 warning)
✅ Passed checks (4 passed)
✏️ Tip: You can configure your own custom pre-merge checks in the settings. ✨ Finishing Touches📝 Generate docstrings
🧪 Generate unit tests (beta)
Warning There were issues while running some tools. Please review the errors and either fix the tool's configuration or disable the tool if it's a critical failure. 🔧 golangci-lint (2.12.1)level=error msg="[linters_context] typechecking error: pattern ./...: directory prefix . does not contain main module or its selected dependencies" Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out. Comment |
![coderabbitai[bot]](https://avatars.githubusercontent.com/in/347564?s=60&v=4){kind=small}
There was a problem hiding this comment.
Actionable comments posted: 3
🧹 Nitpick comments (2)
v3/pkg/application/transport_http.go (1)
111-117: 💤 Low value
Stop()is not idempotent/safe under concurrent calls.
close(t.stopCleanup)followed byt.stopCleanup = nilis a data race with any concurrent observer of the field, and a secondStop()call from another goroutine after the nil-check but before the assignment wouldclose(nil)and panic. In practice this is unlikely in this lifecycle, but async.Once(or an atomic CAS) is essentially free.type HTTPTransport struct { ... - stopCleanup chan struct{} + stopCleanup chan struct{} + stopOnce sync.Once } ... func (t *HTTPTransport) Stop() error { - if t.stopCleanup != nil { - close(t.stopCleanup) - t.stopCleanup = nil - } + t.stopOnce.Do(func() { + if t.stopCleanup != nil { + close(t.stopCleanup) + } + }) return nil }🤖 Prompt for AI Agents
Verify each finding against current code. Fix only still-valid issues, skip the rest with a brief reason, keep changes minimal, and validate. In `@v3/pkg/application/transport_http.go` around lines 111 - 117, The Stop() method on HTTPTransport is not safe for concurrent calls because closing t.stopCleanup without synchronization can panic; change HTTPTransport to include a sync.Once (e.g., stopOnce) or use an atomic CAS and wrap the close(t.stopCleanup) call inside stopOnce.Do(func(){ ... }) so the channel is closed exactly once; update the Stop() implementation to call stopOnce.Do to close the channel and nil out t.stopCleanup (if desired) and ensure any existing tests or callers use the new field.v3/internal/runtime/desktop/@wailsio/runtime/src/runtime.ts (1)
155-181: 💤 Low valueConsider failing fast on a non-final chunk that returns a non-empty body.
The protocol contract is that intermediate chunks reply with empty 200 OK and only the final chunk carries the RPC result. The client currently only checks
resp.okfor intermediates and discards the body. If the server (e.g., a future change or a misrouted endpoint) returns an unexpected payload mid-stream, the call still continues and the user sees a confusing error from the last chunk's response. A debugconsole.warnwhen an intermediate response has non-empty body would catch protocol drift cheaply.🤖 Prompt for AI Agents
Verify each finding against current code. Fix only still-valid issues, skip the rest with a brief reason, keep changes minimal, and validate. In `@v3/internal/runtime/desktop/`@wailsio/runtime/src/runtime.ts around lines 155 - 181, In the chunked upload loop (the for loop that slices bodyStr using CHUNK_THRESHOLD and sends chunks with headers including chunkId, x-wails-chunk-index, x-wails-chunk-total), read the intermediate response body (await resp.text()) and if it is non-empty (or not just whitespace) fail fast: either log a console.warn with the unexpected body and throw an Error containing that payload so the call stops immediately instead of continuing to the final chunk; keep the existing resp.ok check but replace the silent discard of the body with this explicit check on resp before proceeding to the next chunk.
🤖 Prompt for all review comments with AI agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.
Inline comments:
In `@v3/internal/runtime/desktop/`@wailsio/runtime/src/runtime.ts:
- Around line 130-136: The code is currently chunking JSON by string length
which breaks UTF-16 surrogate pairs; change the flow so you call TextEncoder to
produce a Uint8Array (e.g., const bodyBytes = new
TextEncoder().encode(JSON.stringify(body))) and use that byte array against
CHUNK_THRESHOLD, passing bodyBytes into sendChunked instead of bodyStr; update
sendChunked to accept a Uint8Array and perform slicing with subarray(byteOffset,
byteOffset + CHUNK_THRESHOLD) to create each chunk, and send each chunk with
fetch using the Uint8Array (or its buffer/view) so no surrogate pairs are split;
after updating runtime.ts (the main request block and the sendChunked function)
regenerate the bundled runtime.js artifact.
In `@v3/pkg/application/transport_http.go`:
- Around line 165-211: In handleChunkedRequest validate that index < total
immediately after parsing and reject requests where index >= total using
httpError; enforce sane upper bounds by defining constants (e.g., MaxChunkTotal,
MaxChunkSize, MaxAggregateSize) and reject totals > MaxChunkTotal, individual
chunk bodies > MaxChunkSize (check buf.Len()) and the sum of stored chunk
lengths per chunkID > MaxAggregateSize before storing to prevent OOM; when
LoadOrStore returns an existing *pendingChunks, verify that the stored total
equals the incoming total and reject mismatches rather than silently accepting
them; update pendingChunks to track current aggregate size and use that to
enforce the aggregate bound when adding a new chunk.
- Around line 196-224: The current handling in the chunk-join path lets
duplicate requests race to call processBody because Delete on t.chunkStore isn't
sufficient; fix by adding a completed bool field to pendingChunks and guard it
with pc.mu: after determining received>=total check/set pc.completed=true while
holding pc.mu and treat any arrivals where pc.completed is true as a no-op
(return 200 OK) instead of proceeding; keep the existing t.chunkStore.Delete but
use the completed flag to prevent a second goroutine from assembling/processing
the same data, and while assembling preallocate the assembled slice to the total
expected size to avoid repeated reallocations; update references in LoadOrStore
handling, the pc.mu-protected sections, and the call to t.processBody
accordingly.
---
Nitpick comments:
In `@v3/internal/runtime/desktop/`@wailsio/runtime/src/runtime.ts:
- Around line 155-181: In the chunked upload loop (the for loop that slices
bodyStr using CHUNK_THRESHOLD and sends chunks with headers including chunkId,
x-wails-chunk-index, x-wails-chunk-total), read the intermediate response body
(await resp.text()) and if it is non-empty (or not just whitespace) fail fast:
either log a console.warn with the unexpected body and throw an Error containing
that payload so the call stops immediately instead of continuing to the final
chunk; keep the existing resp.ok check but replace the silent discard of the
body with this explicit check on resp before proceeding to the next chunk.
In `@v3/pkg/application/transport_http.go`:
- Around line 111-117: The Stop() method on HTTPTransport is not safe for
concurrent calls because closing t.stopCleanup without synchronization can
panic; change HTTPTransport to include a sync.Once (e.g., stopOnce) or use an
atomic CAS and wrap the close(t.stopCleanup) call inside stopOnce.Do(func(){ ...
}) so the channel is closed exactly once; update the Stop() implementation to
call stopOnce.Do to close the channel and nil out t.stopCleanup (if desired) and
ensure any existing tests or callers use the new field.
🪄 Autofix (Beta)
Fix all unresolved CodeRabbit comments on this PR:
- Push a commit to this branch (recommended)
- Create a new PR with the fixes
ℹ️ Review info
⚙️ Run configuration
Configuration used: Path: .coderabbit.yaml
Review profile: CHILL
Plan: Pro
Run ID: a4212cad-f9ff-490a-b1ea-0740e9c9f91e
📒 Files selected for processing (4)
v3/internal/assetserver/bundledassets/runtime.debug.jsv3/internal/assetserver/bundledassets/runtime.jsv3/internal/runtime/desktop/@wailsio/runtime/src/runtime.tsv3/pkg/application/transport_http.go
| actual, _ := t.chunkStore.LoadOrStore(chunkID, &pendingChunks{ | ||
| chunks: make(map[int][]byte), | ||
| total: total, | ||
| createdAt: time.Now(), | ||
| }) | ||
| pc := actual.(*pendingChunks) | ||
|
|
||
| pc.mu.Lock() | ||
| pc.chunks[index] = chunk | ||
| received := len(pc.chunks) | ||
| pc.mu.Unlock() | ||
|
|
||
| if received < total { | ||
| rw.WriteHeader(http.StatusOK) | ||
| return | ||
| } | ||
|
|
||
| // All chunks received — assemble in order and process. | ||
| t.chunkStore.Delete(chunkID) | ||
|
|
||
| pc.mu.Lock() | ||
| var assembled []byte | ||
| for i := 0; i < pc.total; i++ { | ||
| assembled = append(assembled, pc.chunks[i]...) | ||
| } | ||
| pc.mu.Unlock() | ||
|
|
||
| t.processBody(rw, r, assembled) | ||
| } |
There was a problem hiding this comment.
Race: duplicate chunk submission can run processBody twice on the same ResponseWriter.
If a client (or a flaky retry on the WebView2 side) re-sends the same index after the prior request has already triggered completion but before chunkStore.Delete is observed, two requests can both pass received >= total and both invoke processBody on their own rw. The second one will work on a stale pc that's already been processed, but it will still attempt to write a successful response with an empty/garbage body, and may also race on the assembly read.
Add a "completed" flag guarded by pc.mu and treat post-completion arrivals as 200 OK no-ops (or an error). The store-level Delete alone is not sufficient because both goroutines hold a reference to the same *pendingChunks.
🔒 Suggested fix
type pendingChunks struct {
mu sync.Mutex
chunks map[int][]byte
total int
createdAt time.Time
+ done bool
}
...
- pc.mu.Lock()
- pc.chunks[index] = chunk
- received := len(pc.chunks)
- pc.mu.Unlock()
-
- if received < total {
- rw.WriteHeader(http.StatusOK)
- return
- }
-
- // All chunks received — assemble in order and process.
- t.chunkStore.Delete(chunkID)
-
- pc.mu.Lock()
- var assembled []byte
- for i := 0; i < pc.total; i++ {
- assembled = append(assembled, pc.chunks[i]...)
- }
- pc.mu.Unlock()
+ pc.mu.Lock()
+ if pc.done {
+ pc.mu.Unlock()
+ rw.WriteHeader(http.StatusOK)
+ return
+ }
+ pc.chunks[index] = chunk
+ if len(pc.chunks) < pc.total {
+ pc.mu.Unlock()
+ rw.WriteHeader(http.StatusOK)
+ return
+ }
+ pc.done = true
+ size := 0
+ for _, c := range pc.chunks {
+ size += len(c)
+ }
+ assembled := make([]byte, 0, size)
+ for i := 0; i < pc.total; i++ {
+ assembled = append(assembled, pc.chunks[i]...)
+ }
+ pc.chunks = nil // release memory early
+ pc.mu.Unlock()
+ t.chunkStore.Delete(chunkID)
t.processBody(rw, r, assembled)
}This also pre-sizes the assembled buffer (otherwise the append loop reallocates repeatedly for multi‑MB payloads).
🤖 Prompt for AI Agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.
In `@v3/pkg/application/transport_http.go` around lines 196 - 224, The current
handling in the chunk-join path lets duplicate requests race to call processBody
because Delete on t.chunkStore isn't sufficient; fix by adding a completed bool
field to pendingChunks and guard it with pc.mu: after determining
received>=total check/set pc.completed=true while holding pc.mu and treat any
arrivals where pc.completed is true as a no-op (return 200 OK) instead of
proceeding; keep the existing t.chunkStore.Delete but use the completed flag to
prevent a second goroutine from assembling/processing the same data, and while
assembling preallocate the assembled slice to the total expected size to avoid
repeated reallocations; update references in LoadOrStore handling, the
pc.mu-protected sections, and the call to t.processBody accordingly.
There was a problem hiding this comment.
Fixed. Added processing bool to pendingChunks, guarded by pc.mu. The check received >= pc.total && !pc.processing plus setting pc.processing = true is done atomically inside the lock, so only one goroutine wins the assembly race. The loser gets 200 OK (same as an intermediate chunk).
There was a problem hiding this comment.
🧩 Analysis chain
🏁 Script executed:
#!/bin/bash
rg -n "processing\|pendingChunks\|pc\.mu\|assembled\|handleChunkedRequest" v3/pkg/application/transport_http.go
Length of output: 0
🏁 Script executed:
#!/bin/bash
fd "transport_http.go" --type f
Length of output: 37
🏁 Script executed:
#!/bin/bash
grep -n "processing\|pendingChunks\|pc\.mu\|assembled\|handleChunkedRequest\|done bool" v3/pkg/application/transport_http.go
Length of output: 706
🏁 Script executed:
#!/bin/bash
sed -n '37,225p' v3/pkg/application/transport_http.go
Length of output: 4606
@leaanthony, it looks like the fix hasn't been pushed to the PR branch yet — the file on disk still has the original code without the processing bool field. The pendingChunks struct and handleChunkedRequest function are unchanged from the initial diff.
Once you push the commit, the struct should look something like this and the critical section should be consolidated into a single lock acquisition:
type pendingChunks struct {
mu sync.Mutex
chunks map[int][]byte
total int
createdAt time.Time
processing bool
}pc.mu.Lock()
pc.chunks[index] = chunk
received := len(pc.chunks)
if received < pc.total || pc.processing {
pc.mu.Unlock()
rw.WriteHeader(http.StatusOK)
return
}
pc.processing = true // wins the assembly race atomically
// Pre-size and assemble while still holding the lock
size := 0
for _, c := range pc.chunks {
size += len(c)
}
assembled := make([]byte, 0, size)
for i := 0; i < pc.total; i++ {
assembled = append(assembled, pc.chunks[i]...)
}
pc.chunks = nil // release chunk memory early
pc.mu.Unlock()
t.chunkStore.Delete(chunkID)
t.processBody(rw, r, assembled)The key point is keeping everything — the processing check, the processing = true set, and the assembly — inside a single pc.mu.Lock() block so there's no window between the check and the set. Please push your changes and I'll re-verify!
🧠 Learnings used
Learnt from: symball
Repo: wailsapp/wails PR: 4853
File: v2/internal/system/system.go:128-152
Timestamp: 2026-01-04T08:01:00.038Z
Learning: In v2/internal/system/system.go, shared functions like checkLibrary are defined without build tags but are only invoked from platform-specific files (system_linux.go, system_windows.go, system_darwin.go) that have build constraints. Reviewers should ensure there are no runtime OS checks in system.go and that platform-specific behavior is controlled via build tags. If runtime switches exist, remove them in favor of compile-time platform constraints to reduce overhead and improve correctness.
Learnt from: leaanthony
Repo: wailsapp/wails PR: 4906
File: v3/internal/setupwizard/wizard_windows.go:93-94
Timestamp: 2026-01-26T01:56:49.314Z
Learning: Ensure the Go toolchain minimum version for Wails v3 is 1.25+. Update go.mod to reflect 1.25+ and validate that setup wizard checks enforce Go 1.25+ at release time. Align any CI/go-version checks, documentation, and release notes with this requirement; this guideline applies to all Go files under the v3 module, not just the wizard.
There was a problem hiding this comment.
Pull request overview
This PR introduces a chunked IPC request path for Wails v3 runtime calls to work around WebView2’s ~2MB request-body delivery limit, preventing large frontend→backend service calls from failing on Windows.
Changes:
- Added chunk headers + server-side accumulation/assembly logic in the HTTP transport, with periodic TTL-based eviction of incomplete transfers.
- Updated the JS runtime transport to conditionally send large JSON payloads as multiple sequential POSTs and return only the final chunk response.
- Rebuilt the bundled runtime assets for debug and production distributions.
Reviewed changes
Copilot reviewed 3 out of 4 changed files in this pull request and generated 5 comments.
| File | Description |
|---|---|
v3/pkg/application/transport_http.go |
Adds chunk header detection, per-chunk storage/assembly, and background cleanup for incomplete chunk sets. |
v3/internal/runtime/desktop/@wailsio/runtime/src/runtime.ts |
Adds CHUNK_THRESHOLD and sendChunked helper; routes large runtime calls through chunked POSTs. |
v3/internal/assetserver/bundledassets/runtime.debug.js |
Rebuilt bundle to include chunking behavior (debug build). |
v3/internal/assetserver/bundledassets/runtime.js |
Rebuilt bundle to include chunking behavior (minified production build). |
💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.
…corruption The previous chunking split bodyStr at JS string indices (UTF-16 code units), which could fall mid-surrogate-pair for non-BMP characters such as emoji or CJK extension code points. The lone surrogates in each chunk would corrupt JSON when encoded to UTF-8 for the HTTP body. Use TextEncoder to convert the full serialised body to a UTF-8 Uint8Array first, then slice at CHUNK_THRESHOLD byte offsets with subarray(). Go's assembler already works on raw bytes so no server-side change is needed. Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com> Co-authored-by: multica-agent <github@multica.ai>
Linux QA — review concerns addressedThe Linux functional test already passed (all 7 payload sizes 1 KB–5 MB, WebKitGTK 4.1, Ubuntu 24.04). This comment addresses the automated reviewer threads. Issues found and fixed in
|
Cap chunk count (maxChunkTotal=1024), per-chunk body size (http.MaxBytesReader at 1 MiB), and aggregate assembled size (64 MiB) to prevent OOM from malformed or hostile clients. Validate that subsequent chunks declare the same total as the first chunk; reject mismatches with 422. Addresses remaining review feedback on PR #5369. Co-authored-by: multica-agent <github@multica.ai>
|
Addressed both remaining review items from #5369 (discussion_r3212888838): Issue #2 — Upper bounds added:
Issue #3 — Inconsistent
Bonus fix:
CC @leaanthony Taliesin is an AI agent. CC @leaanthony |
…k large IPC payloads to bypass WebView2 2 MB body limit
Problem
When a frontend service call passes a payload larger than ~2 MB, the
ICoreWebView2WebResourceRequest::GetContent()IStream silently truncates or drops the body. From the browser's perspective the POST tohttp://wails.localhost/wails/runtimefails and DevTools reports it as a CORS error.Repro: call any bound Go service method from JavaScript with a
stringorBlobargument larger than ~2 MB.Fixes #4428
Root cause
WebView2's
WebResourceRequestedevent delivers the request body throughICoreWebView2WebResourceRequest::GetContent()— a COMIStream. The WebView2 team has documented an internal buffering limit of roughly 2 MB; bodies beyond that are not fully delivered to the host process.Fix
JavaScript (
runtime.ts) — if the serialised JSON body exceeds 512 KB the newsendChunkedhelper splits it into 512 KB slices and sends them serially. Each request carries three coordination headers:x-wails-chunk-idx-wails-chunk-indexx-wails-chunk-totalAll intermediate chunks receive a
200 OKwith an empty body. The final chunk's response carries the actual RPC result, so the call-site API is completely unchanged.Go (
transport_http.go) —handleRuntimeRequestdetects thex-wails-chunk-idheader and delegates to the newhandleChunkedRequestpath. Chunks are accumulated in async.MapofpendingChunks. When all slices arrive they are assembled in order and forwarded to the existingprocessBodypath. A background goroutine (10 s tick) evicts accumulators older than 30 s to prevent leaks from aborted transfers.Files changed
v3/pkg/application/transport_http.gov3/internal/runtime/desktop/@wailsio/runtime/src/runtime.tsCHUNK_THRESHOLDconstant,sendChunkedhelper, conditional dispatchv3/internal/assetserver/bundledassets/runtime.debug.jsv3/internal/assetserver/bundledassets/runtime.jsTesting
go build ./v3/pkg/application/...— cleantsc --noEmitin the runtime package — cleanCHUNK_THRESHOLD/sendChunkedsymbolsManual end-to-end test: bind a Go method that accepts a
string, call it from JS with a 3 MB payload; the call completes successfully in both debug and production bundles.CC @leaanthony
Summary by CodeRabbit
New Features
Bug Fixes
Other