int_proxy reduce: tidy the backend interface and the retire harness - #810
Open
frankmcsherry wants to merge 4 commits into
Open
int_proxy reduce: tidy the backend interface and the retire harness#810frankmcsherry wants to merge 4 commits into
frankmcsherry wants to merge 4 commits into
Conversation
`retire` held fourteen scratch buffers as inline locals and distinguished its two phases by comment alone. Those buffers and the times a retire reasons against now live on a `Retire` struct, whose `run` walks the windows and whose `determine` / `apply` / `flush` are the phases the comments named. Pure motion; the only behavioral difference is that `tile_deltas` is cleared at the start of `apply` rather than the start of the window, which nothing between the two observes. Also drops a vestigial `'static` on `ProxyReduceBackend::ROut` and corrects a comment that credited `drain` for what `append` does. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
…ns in place `reduce_corrections` took five undocumented parallel arrays and returned two more. The inputs are now a `ReduceRound` and the outputs a `ReduceCorrections`, each documenting how its lists segment by key. The corrections are populated in place rather than returned owned, so the harness's allocation survives the round instead of being replaced by a fresh pair on every one of them. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
`seed_times` scanned the whole instance up front and returned a time for every changed key, so a retire materialized its entire seed set before doing any work. The harness then derived the changed keys from it and handed them back to the backend to window. `advance` replaces both it and `next_window`, on join's model: progress is an inclusive lower bound on key hash that the backend advances to the exclusive upper bound of the range it covers, and the window it populates now carries the seeds for its own keys. What the harness contributes is the times earlier retires withheld, presented flat and key-ordered for the backend to move through and block its windows against. The window is populated in place rather than returned owned, so its three buffers survive from one window to the next. The output session now opens on the first window with work, preserving the behavior of the old changed-is-empty short circuit: a retire with nothing to do ships no batches rather than an empty one per tile. `advance` correspondingly draws a non-empty window unless it reports exhaustion. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Twenty-one fields to eighteen, none of them costing an allocation. `pended` becomes a local. It looked like a twin of `moments`, but the two are drained differently: `append` leaves `moments`' capacity behind for the next key, where `new_pending` takes `pended`'s allocation whole and leaves nothing. So it either never allocated or was carried off, and a local does the same work. The `moments` field now documents that asymmetry, since it is the reason only one of the pair is a field. `n_states` becomes a value returned by `determine` and passed to `apply`. It was per-window state in a per-retire struct, and the dependency between the phases was invisible. `held` and `tile_of` were parallel by held index, and become one `tiles` list. Also renames `cur_out` to `out_accum`, matching `in_accum` beside it. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
No description provided.