flowctl: preview runs on the V2 runtime, remove V1 preview - #3442
Merged
Conversation
jgraettinger
force-pushed
the
johnny/flowctl-preview-v2-migration-856bc7
branch
from
August 28, 2026 21:45
9306303 to
d2606b7
Compare
jgraettinger
force-pushed
the
johnny/flowctl-preview-v2-migration-856bc7
branch
from
August 31, 2026 23:13
d2606b7 to
6ace886
Compare
`flowctl preview` now dispatches to the runtime-next + shuffle
implementation developed as `flowctl raw preview-next`, and the legacy
V1-runtime `preview/{mod,journal_reader}.rs` is deleted. `raw
preview-next` remains as a hidden, deprecated alias onto the same
`clap::Args` struct, so existing callers keep working untouched.
The flag surface is a superset -- `--shards` and `--debug-port` are
additive and nothing is removed -- but three behaviors do change:
* Derivation documents on stdout gain the `["collection/name",<doc>]`
framing that captures and materializations already used; stripping the
wrapper reproduces the old output exactly. Nothing downstream parses
it. The three site/ derivation tutorials show bare documents and are
now stale, but site/ no longer serves docs.estuary.dev (90e8261),
so that correction belongs in estuary/docs.
* `--delay` combined with `--fixture` is now an error. A fixture's
transaction boundaries are fixed by its own commit markers, so there
is nothing for a raised minimum transaction duration to batch.
* An unset `--sessions` no longer means the same thing as `--sessions -1`
for captures: unset holds an EOF'd session through the capture's poll
interval as production does, while `-1` stops at EOF (f259b8f).
The module is deliberately NOT moved to `preview/`. The in-flight
`johnny/rebase-runtime-v2-tech-debt-bab0a9` renames five of its files out
to a new `crates/runtime-local` crate and rewrites what remains, so
moving now would mean rename-vs-rename conflicts on every one -- and
would land the wrong layout regardless, since after that split the code
deserving the name `preview/` is only the flowctl-specific residue. A
TODO on `raw::Command::PreviewNext` records the deferred move, along with
the in-repo callers that must migrate before the alias can be dropped.
`flowctl preview` was the sole consumer of `runtime::harness`, and of the private `exchange` combinator that only the harness's derive and materialize drivers called. Both die with V1 preview. No dependency is orphaned: `coroutines` and `tempfile` remain in use elsewhere in the crate, so Cargo.toml is unchanged.
jgraettinger
force-pushed
the
johnny/flowctl-preview-v2-migration-856bc7
branch
from
September 1, 2026 02:53
6ace886 to
bc2ca1f
Compare
williamhbaker
approved these changes
Sep 1, 2026
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Replaces the V1-runtime-backed
flowctl previewwith the runtime-next + shuffleimplementation developed as
flowctl raw preview-next, and deletes V1 previewalong with the runtime harness it was the sole consumer of.
raw preview-nextis retained as a hidden, deprecated alias onto the sameclap::Argsstruct, so existing callers — estuary/connectors CI, the in-repopytest suite — keep working untouched.
Commits
flowctl: preview runs on the V2 runtime— re-pointCommand::Preview,hide + deprecate the
raw preview-nextalias, deleteflowctl/src/preview/{mod,journal_reader}.rs(−733).runtime: remove the now-dead V1 preview harness— deleteruntime/src/harness/and the privateexchangecombinator (−1131). Splitout so the first commit proves the harness has no remaining callers and
this one reviews as a pure deletion.
Behavior changes
The flag surface is a strict superset (
--shardsand--debug-portareadditive; nothing is removed). Three behaviors do change:
["collection/name",<doc>]framing, whichcaptures and materializations already used. Stripping the wrapper reproduces
the old output exactly. Nothing downstream parses it. The three
site/derivation tutorials show bare documents and are now stale — but
site/nolonger serves docs.estuary.dev (90e8261), so that correction belongs in
estuary/docsand is deliberately not made here.--delaywith--fixtureis now an error. A fixture's transactionboundaries are fixed by its own commit markers, so a raised minimum
transaction duration has nothing to batch.
--sessionsno longer equals--sessions -1for captures. Unsetholds an EOF'd session through the capture's poll interval as production does;
-1stops at EOF (f259b8f).Why the module isn't moved to
preview/flowctl previewis backed by a module still namedraw/preview_next/. That isdeliberate. #3416 renames five of that module's files out to a new
crates/runtime-localcrate and rewrites the rest, so moving now meansrename-vs-rename conflicts on every one — and would land the wrong layout
regardless, since after that split the code deserving the name
preview/is onlythe flowctl-specific residue.
git merge-treeagainst the rebased #3415–#3419 stack reports zero conflictsin either direction. The only file both touch is
raw/mod.rs(this PR edits thePreviewNextarm; #3417 adds aTestarm beside it), and the merged result waschecked by hand to be semantically correct.
A
TODO(johnny)onraw::Command::PreviewNextrecords the deferred move, thein-repo callers that must migrate first (
tests/preview/*.pyand thetests/soak/**comments, both introduced by #3416), and the eventual removal ofthe alias.
Verification
misewith no unused-code warnings — thecompiler is the proof the harness is dead.
flowctl preview --helpcarries the original doc text with the V2 flag set.raw --helpno longer listspreview-next;raw preview-next --helpstilldispatches and reports itself deprecated.
Note that the
tests/*.pysnapshot suite could not verify anything: it is notwired into CI, and has been broken since 85253a1 swapped the fixture reader
without converting the checked-in fixtures. The framing claim above was instead
verified directly against the V1 and V2 snapshot artifacts.