Say so when an interrupt stops a streamed statement - #338
Merged
Conversation
A streamed run breaks out of its morsel loop when the stop flag is up, and that flag is up for two different things: a LIMIT that has been filled, which is an answer that is all there, and a caller who asked the statement to stop, which is an answer that ends partway. Breaking out for both handed the second one back as a whole result: rows fewer than the statement would have returned, stopped false, no condition, and nothing in it for a caller to notice. The interrupt is now asked at the top of the loop, before the flag the quota shares, so an interrupted stream ends the way the buffered run already ends it, with ZuError::Interrupted and the session exactly as it was. One relaxed load per morsel, which is a load per thousand rows and more. The test covers both executors, because a client raising an interrupt does not know which one took its statement, and it needs a table of more than one morsel to have a partway to stop at.
tamnd
added a commit
to tamnd/zu-node
that referenced
this pull request
Aug 18, 2026
conn.stream(...) runs a statement and hands its rows over as they are made. Reading it is a for await over rows, a batches() over the arrays they crossed the boundary in, or a toReadableStream() for anything that already speaks Web Streams, and all three are the same statement read once. The engine's shape is a push and JavaScript wants a pull, so a thread of the statement's own sits between them with a queue of two batches. That queue is the whole of the buffering: a reader slower than the scan stops the scan rather than filling memory behind it. The thread is its own rather than libuv's, because a statement parked on the threadpool waiting for a loop body to come round again is a quarter of every other library's file reads gone. Stopping is the part worth the machinery. A break, a throw, a return() on the iterator, a cancel() or the end of an await using block all end the statement and wait for it to let go of the connection, so the next statement runs rather than queueing behind a scan nobody is reading. A cursor nobody holds closes its queue when it is collected, which is the backstop under a program that forgot. The statement starts on the first read and not before, so a stream made and never read holds nothing at all. The summary says what the statement did once it is over: the columns, the rows handed over, whether the reader stopped it, whether it was streamed rather than run whole and cut up afterwards, and the notices. ORDER BY, DISTINCT and the aggregates are the second kind, and the loop over them is the same either way. batchRows is read as a double and checked here rather than narrowed by the runtime, because JavaScript's own narrowing turns -1 into four billion and 1.5 into 1. Engine pinned forward to 92c9a5e for tamnd/zu#338, without which an interrupted stream ended cleanly and truncated instead of failing. 50k rows, fastest of nine: a stream read to the end costs 463ns a row against 372ns for query, a batch at a time 320ns, and reading the first batch and stopping 1.1ms against 18.6ms for the whole scan.
33 tasks
tamnd
added a commit
that referenced
this pull request
Aug 19, 2026
A streamed run breaks out of its morsel loop when the stop flag is up, and that flag is up for two different things: a LIMIT that has been filled, which is an answer that is all there, and a caller who asked the statement to stop, which is an answer that ends partway. Breaking out for both handed the second one back as a whole result: rows fewer than the statement would have returned, stopped false, no condition, and nothing in it for a caller to notice. The interrupt is now asked at the top of the loop, before the flag the quota shares, so an interrupted stream ends the way the buffered run already ends it, with ZuError::Interrupted and the session exactly as it was. One relaxed load per morsel, which is a load per thousand rows and more. The test covers both executors, because a client raising an interrupt does not know which one took its statement, and it needs a table of more than one morsel to have a partway to stop at.
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
A streamed run breaks out of its morsel loop when the stop flag is up, and that flag stands for two different things: a LIMIT that has been filled, which is an answer that is all there, and a caller who asked the statement to stop, which is an answer that ends partway. Breaking out for both handed the second one back as a whole result: fewer rows than the statement would have returned,
stoppedfalse, no condition raised, and nothing in it for a caller to notice.The client this turned up in is the JavaScript one, where an
AbortSignalon a stream ended the loop cleanly after 1024 of 60000 rows. A truncated result reported as a complete one is the one outcome nobody can defend against.The interrupt is now asked at the top of the loop, before the flag the quota shares, so an interrupted stream ends the way the buffered run already ends it:
ZuError::Interrupted, and the session exactly as it was. It costs one relaxed load per morsel, which is a load per thousand rows and more.The test raises the interrupt from inside the sink, which is where a client's signal fires, and runs under both executors, because a client raising an interrupt does not know which one took its statement. It needs a table of more than one morsel to have a partway to stop at, so it seeds its own.
Checked:
cargo test -p zu -p zu-execgreen,cargo fmt --all --check,cargo clippy -p zu -p zu-exec --all-targets. Reverting the one-line fix fails the new test withStreamed { rows: 1024, stopped: false, .. }, which is the bug.