sync: latch the crypto worker bridge off after repeated failures - #968
Merged
Conversation
A worker that is alive but never answers kept being chosen every batch, because isRunning only knew the thread had not exited. Each push and pull batch paid the full 60s request timeout before degrading to main-thread crypto, so sync completed at a rate that reads as broken. The bridge now counts consecutive infrastructure failures and reports isRunning false after three, sending sync-crypto-batch straight to main-thread crypto with no round trip. A successful batch resets the count, so a transient hiccup does not cost the session its worker.
|
React Doctor found no new issues. 🎉 Reviewed by React Doctor for commit |
Codecov Report❌ Patch coverage is
📢 Thoughts on this report? Let us know! |
h4yfans
marked this pull request as ready for review
August 5, 2026 18:27
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Closes #964
Builds directly on #960.
Problem
#960 made a rejected worker crypto request degrade to main-thread crypto instead of failing the batch. It deliberately did not stop asking the worker, and the two failure shapes are very far apart in cost.
A worker that crashes is already fine — the
exithandler nullsthis.worker,isRunninggoes false, and later batches skip it. A worker that answers with an error is fine too: the reply is immediate and the fallback costs sub-millisecond IPC.The gap is the worker that is alive but silent.
isRunningwasthis.worker !== null, which only knows the thread has not exited, so a hung-but-alive worker kept reporting healthy and kept being chosen. Nothing anywhere short-circuits:sendRequestarms a fresh 60s timer per request, andisRunninghas exactly two production consumers, both of them the per-batch gate insync-crypto-batch.ts. Every push batch and every pull batch therefore paid a full minute before degrading. Sync still completed — that is #960 working — but at a rate that reads as broken.Fix
SyncWorkerBridgecounts consecutive failed requests. After three it latches:isRunningreports false, andsync-crypto-batch.tstakes the main-thread path with no round trip. A successful batch resets the count to zero.Both
encryptBatchanddecryptBatchare wrapped, so the count also covers the post-response throws ({ type: 'error' }protocol drift, unexpected response type), not justsendRequestrejections.Design decisions
N = 3. One failure is noise — a single transient timeout under load should not cost the session its worker. Three consecutive failures is not noise. Because the penalty is paid in whole minutes, a larger N is expensive: three silent batches is a bounded ~3 minutes of degraded-but-correct sync, after which the worker costs nothing for the rest of the session. The reasoning lives in a comment on the constant.
Latch scope: session-lifetime, not a timed retry. The acceptance criterion asks for a bounded, one-time penalty. A cooldown-and-reprobe scheme cannot deliver that — every expiry buys another 60s stall — whereas a session latch gives a hard ceiling of three timeouts total. It is also simpler and deterministic: no timers, no clock. The cost of being wrong is small and bounded in the right direction: main-thread crypto is correct (the whole point of #960), just slower, so losing the worker for one session is a performance regression and never a correctness one. Re-probing a wedged worker, by contrast, costs the user another frozen minute each time.
The reset-on-success still does the work that matters, because it applies before the latch trips — that is exactly the transient-hiccup case. In-session recovery after a latch is
stop()thenstart(), which spawns a fresh thread and clears the count; a fresh thread is not the thread that failed.The worker thread is not terminated. Raised as an open question in the issue. Terminating buys nothing once the bridge has stopped routing to it, and it would add a failure mode (terminating a wedged thread,
exitracingrejectAll) while removing the only in-session way back.stop()deliberately still keys offthis.workerrather thanisRunning, so a latched-but-alive thread is still shut down cleanly and does not leak.Why this cannot mask a crypto or auth failure
Re-verified against current
worker.ts, not assumed from #960. Per-item crypto outcomes still come back in-band:encrypt-batch-resultcarrieserrors[],decrypt-batch-resultcarriesfailures[]including signature mismatches. Neither rejects the request. The only producer of{ type: 'error' }ishandleUnknownMessage, i.e. protocol drift. So anencryptBatch/decryptBatchrejection is purely infrastructure, and the latch counts nothing else. Latching cannot suppress a bad item — the main-thread path re-runs the identical encryption and signature verification.No key material or plaintext is logged: the latch warning carries only the failure count and the transport error text, and crypto error messages never travel that path.
Backward compatibility
No schema, contract, wire-protocol, or persisted-format change. Purely in-process bridge state. The main↔worker protocol is untouched, so mixed-build installs behave exactly as they do today. Behaviour when the worker answers normally is unchanged — the new path is reachable only after three consecutive infrastructure failures, which previously meant three 60s stalls.
Tests
worker-bridge.test.ts— 4 new: still routes to the worker below the threshold; latches at the threshold and stops issuing round trips; a success clears the count and the threshold is then re-counted from that success; a restarted bridge is no longer latched.sync-crypto-batch.test.ts— 2 new, pinning the other half of the contract: anisRunning: falsebridge encrypts and decrypts on the main thread without ever calling the worker.Whole
src/main/syncsuite green:Mutation check —
worker-bridge.tsstashed, tests re-run:Fix restored, green again.
pnpm typecheck16/16 successful,pnpm lint0 errors (10 pre-existing renderer warnings, none in changed files),pnpm docs:impact --strictcovered,pnpm docs:buildcomplete.The first draft of the reset test asserted only
isRunning === true, which passes with no latch at all; it was tightened to also assert the latch still fires on the next failure, so all four are mutation-sensitive.Deliberately out of scope
encryptItemForPushthrowing in the fallback (e.g. "Item too large for sync") still propagates out ofencryptPushBatchinstead of becoming a per-itemqueue.markFailed. That is how the main-thread path has always behaved when no worker is configured, so it is a pre-existing rough edge rather than a regression.