Add support for sending ReadableStream and WritableStream over RPC, with automatic flow control.#132
Merged
Add support for sending ReadableStream and WritableStream over RPC, with automatic flow control.#132
Conversation
🦋 Changeset detectedLatest commit: 111a252 The changes in this PR will be included in the next version bump. This PR includes changesets to release 1 package
Not sure what this means? Click here to learn what changesets are. Click here if you're a maintainer who wants to add another changeset to this PR |
commit: |
Instead of storing an array of RpcStub, we now store an array of the underlying StubHooks. This will make it easier to add support for new types like streams, which aren't RpcStubs, but they will wrap / be wrapped in StubHooks.
(Flow control is left for a future commit.) Written using Claude+Opencode: https://share.opencode.cloudflare.dev/share/gJU0pT8p (I cleaned some stuff up manually.)
As described in protocol.md, the basic idea here is that whenever we want to send a ReadableStream, we first send a message to the other side creating a "pipe". We pump our ReadableStream to the pipe's WritableSteam end, and we deliver the pipe's ReadableStream end to the remote peer. This way, we can begin pushing bytes immediately upon sending a ReadableStream, without waiting for the remote end to call back asking for the bytes (which would be an unnecessary round trip). Written using Claude+Opencode: https://share.opencode.cloudflare.dev/share/ctbbBnOu
If more than 256kb of writes are in flight, we pause writes until past writes complete so that the number drops back below 256kb. Future commits will expand the window size. Written using Claude+Opencode: https://share.opencode.cloudflare.dev/share/D1bqkx2K This was not the best Claude session. I could probably have done it faster by hand.
We add a new "stream" message to the protocol which skips these. See protocol.md for explanation. (Since this is only used for streams, which were just introduced in this PR, this is not a breaking change.) Written using Claude+Opencode: https://share.opencode.cloudflare.dev/share/OfY838e7
kentonv
force-pushed
the
kenton/streams
branch
from
dbad315 to
df03a6a
Compare
With this change, we'll automatically update a stream's window size based on the observed bandwidth-delay product, in order to fully saturate the stream with minimal additional buffer bloat. The algorithm works by observing when each stream chunk is sent and acknowledged (via return from the RPC), allowing us to calculate: 1. Minimum round trip time. 2. Running average bandwidth over the last RTT. From that we calculate bandwidth-delay product and adjust the window to match. We actually set the window a bit bigger than the calculated BDP during startup (2x) and steady-state (1.25x) so that we can observe if the actual bandwidth is greater than expected, and thus update the window accordingly. I worked with Claude+Opencode to design the algorithm and implement, although I significantly refactored almost everything it wrote as the code was pretty meh: https://share.opencode.cloudflare.dev/share/rGV0SKLW
Member
Author
|
Update: I went ahead and added adaptive window size adjustment! |
dmmulroy
reviewed
Feb 7, 2026
kentonv
added a commit
to capnproto/capnproto
that referenced
this pull request
Feb 7, 2026
The controller measures the stream's RTT and bandwidth and tries to keep the window equal to RTT * bandwidth, plus some margin so that if more bandwidth is available, it'll notice and adjust. This is a transliteration of the algorithm introduced in Cap'n Web here: cloudflare/capnweb#132 The transliteration was performed with the help of Claude + Opencode: https://share.opencode.cloudflare.dev/share/xZgIlJe4
kentonv
added a commit
to capnproto/capnproto
that referenced
this pull request
Feb 8, 2026
The controller measures the stream's RTT and bandwidth and tries to keep the window equal to RTT * bandwidth, plus some margin so that if more bandwidth is available, it'll notice and adjust. This is a transliteration of the algorithm introduced in Cap'n Web here: cloudflare/capnweb#132 The transliteration was performed with the help of Claude + Opencode: https://share.opencode.cloudflare.dev/share/xZgIlJe4
kentonv
added a commit
to capnproto/capnproto
that referenced
this pull request
Feb 8, 2026
The controller measures the stream's RTT and bandwidth and tries to keep the window equal to RTT * bandwidth, plus some margin so that if more bandwidth is available, it'll notice and adjust. This is a transliteration of the algorithm introduced in Cap'n Web here: cloudflare/capnweb#132 The transliteration was performed with the help of Claude + Opencode: https://share.opencode.cloudflare.dev/share/xZgIlJe4
kentonv
added a commit
to capnproto/capnproto
that referenced
this pull request
Feb 8, 2026
The controller measures the stream's RTT and bandwidth and tries to keep the window equal to RTT * bandwidth, plus some margin so that if more bandwidth is available, it'll notice and adjust. This is a transliteration of the algorithm introduced in Cap'n Web here: cloudflare/capnweb#132 The transliteration was performed with the help of Claude + Opencode: https://share.opencode.cloudflare.dev/share/xZgIlJe4
kentonv
added a commit
that referenced
this pull request
Feb 8, 2026
This bug was caught and fixed by AI (prompted by @dmmulroy): #132 (comment) This commit applies exactly the suggestion from the comment.
kentonv
added a commit
to capnproto/capnproto
that referenced
this pull request
Feb 9, 2026
The controller measures the stream's RTT and bandwidth and tries to keep the window equal to RTT * bandwidth, plus some margin so that if more bandwidth is available, it'll notice and adjust. This is a transliteration of the algorithm introduced in Cap'n Web here: cloudflare/capnweb#132 The transliteration was performed with the help of Claude + Opencode: https://share.opencode.cloudflare.dev/share/xZgIlJe4
kentonv
added a commit
to capnproto/capnproto
that referenced
this pull request
Feb 9, 2026
The controller measures the stream's RTT and bandwidth and tries to keep the window equal to RTT * bandwidth, plus some margin so that if more bandwidth is available, it'll notice and adjust. This is a transliteration of the algorithm introduced in Cap'n Web here: cloudflare/capnweb#132 The transliteration was performed with the help of Claude + Opencode: https://share.opencode.cloudflare.dev/share/xZgIlJe4
dmmulroy
reviewed
Feb 12, 2026
Collaborator
dmmulroy
left a comment
dmmulroy
left a comment
There was a problem hiding this comment.
Two bugs found during review. Failing tests and fixes on separate branches (both parented to this PR's HEAD).
This bug was caught and fixed by AI (prompted by @dmmulroy): #132 (comment) This commit applies exactly the suggestion from the comment.
This was a pain to track down, when the code goes into a busy loop, vitest gets very confused. It fails to display console.log()s that happened before the hang and doesn't even correctly report which test is running -- often claiming it's still working on some previous test. Ugh! Anyway, it turns out that the stream implementations on some platforms require macro tasks to make progress, so pumping only microtasks doesn't get there. Weirdly, it seems to be non-deterministic. I was seeing workerd hang maybe 1/4 of the time, and webkit also hang sometimes but less often. It's possible the other platforms were also affected but even more rarely and I just never saw it happen.
kentonv
added a commit
that referenced
this pull request
Feb 12, 2026
Fix and test provided by @dmmulroy here: #132 (comment)
kentonv
force-pushed
the
kenton/streams
branch
from
a01900f to
df1558a
Compare
dmmulroy
approved these changes
Feb 12, 2026
Fix and test provided by @dmmulroy here: #132 (comment)
I noticed that the `state.stream.cancel()` call in `ReadableStreamStubHook.dispose()` was often failing, throwing an exception because the stream was already locked. But we were ignoring the exceptions. This actually fixes the problem in two ways: 1. When we pipe from a ReadableStream (which locks it), we also take a reference on its StubHook, which we dispose then the pipe completes. This makes sense and solves the case seen in the tests. 2. I also just made it skip the cancel() call if the stream is locked. Throwing the exception and ignoring it is just a waste of cycles.
kentonv
force-pushed
the
kenton/streams
branch
from
df1558a to
111a252
Compare
Dr-Emann
added a commit
to Dr-Emann/capnweb_types
that referenced
this pull request
Feb 13, 2026
See cloudflare/capnweb#132 This is a breaking change: it changes the types of message/expressions that can be sent/recieved.
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
2 participants
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
When you send a WritableStream over RPC, the remote side gets a WritableStream. They can write to it. If they write faster than the connection can handle, or faster than your app actually consumes the data, they'll experience backpressure.
When you send a ReadableStream over RPC, the RPC system immediately begins reading from the stream and sending the chunks over the wire so that they are already ready for the remote end to read when it starts reading. This again applies backpressure appropriately. Under the hood, we ask the other end to create a "pipe" -- exposing a WritableStream back to us -- and then we pump chunks into that stream. Meanwhile, the call receiver receives the read end of the pipe.
The flow control is at present based on a fixed window size of 256kb per WritableStream. I intend to fix that in a subsequent change but this is enough to unblock remote bindings that use streams.