Skip to content

v0.32.0

Choose a tag to compare

@github-actions github-actions released this 18 Oct 11:25
· 101 commits to main since this release
8afde97

Upgrading from 0.31.x or earlier — read these first:

  • Exit codes follow the code's class. A coded refusal exits 5, an integrity stop 3, an internal
    error 6, a usage error 1, an environment fault 2 when it is transient and 1 otherwise. Several
    stops that exited 1 now exit 5; each is named in its entry below. A scheduler that retries on any
    non-zero exit should stop retrying on 5.
  • The state schema moves from v32 to v37 on the first run. Before migrating, rivet now leaves a
    copy of the old state (.rivet_state.db.v32.<stamp>.bak beside a SQLite state, schema
    rivet_state_backup_v32 in a PostgreSQL state) and the way back to the previous release is in
    docs/reference/state.md, "Rolling back an upgrade". If the copy cannot be written, the
    migration does not run (RIVET_STATE_BACKUP_FAILED).
  • A relative destination.path is anchored to the config file's directory, not the working
    directory. A run started from another directory while the export's files still sit under the old
    place is refused (RIVET_DEST_RELATIVE_PATH_MOVED) until the files are moved or the path is made
    absolute.
  • A dropped source table is refused RIVET_SOURCE_TABLE_NOT_FOUND on every SQL engine; remove
    the export from exports: if the table was dropped on purpose.

Known issue in this release (fixed in 0.32.1):

  • rivet load of a batch export reads the manifests under its whole prefix, including another
    export's nested under it.
    rivet init on SQL Server writes a batch prefix exports/<t>/ and a
    CDC prefix exports/<t>/cdc/; with both in one bucket, rivet load of the batch export takes the
    stream's snapshot and change parts as its own and applies their deletes to the batch table
    (exit 0). Present since at least 0.31.0. What to do: never place one export's prefix inside
    another's; in a config rivet init generated for SQL Server, move the CDC export's prefix to a
    sibling (for example cdc/<t>/) before the first run.

  • A plan applied after UTC midnight wrote under the planning day. rivet plan resolved
    {date} in the destination into the sealed plan, so a plan made before midnight and applied
    after it wrote under the planning day while the data and the part files were of the apply
    (P-26, since rivet plan exists). Now the sealed plan keeps {date} as written and
    rivet apply resolves it to the UTC day it runs on, through the same resolver rivet run
    uses: a plan made before midnight UTC and applied after now writes under the apply date.
    rivet plan --date YYYY-MM-DD freezes that day into the plan instead. A plan sealed by an
    earlier release holds an already-resolved path and applies as it did. What to do:
    nothing; pass --date to rivet plan when the planning day is the one wanted.

  • rivet apply answers a replay of an applied plan in the summary document, and gains --summary-output / --json.
    A replay of a full, chunked or keyset plan that already applied to exit 0 (0.31.0, #616) answered
    only in prose on stderr: no summary file, no JSON line, so a scheduler could not tell "already
    applied by run X" from "succeeded, result missing" without parsing text. Now rivet apply
    takes --summary-output <PATH> and --json like rivet run (in both plan-artifact and config
    modes), and the replay writes the same document a run writes, built from the run that applied the
    plan: its run_id, rows, files and bytes from its export_metrics row, status: "already_applied"
    and answered_by_run: "<run_id>". Every per_export[] entry now carries answered_by_run
    (null unless the entry answers a replay); the stderr line is unchanged, the exit stays 0, and
    the replay still writes no run, manifest or metrics row. ADR-0039 lists the new status word and
    key. What to do: a scheduler reads status on the entry; a status table that treats an
    unknown word as failed needs already_applied added.

  • An upgrade could not be rolled back: the first run of a newer rivet migrated the state schema in
    place, and the previous release then refused it (RIVET_STATE_SCHEMA_NEWER, exit 5) with no copy
    of what it had written.
    Release 0.31.0 knows schema v32; this release moves it to v37 (0.31.0
    and earlier). Now a migration first writes a copy of the state as the previous release left it,
    inside the migration lock and before its first statement: on SQLite the file
    <state>.v<old>.<UTC time>.bak beside the state (.rivet_state.db.v32.20261017T101530Z.bak), on
    PostgreSQL the schema rivet_state_backup_v<old> in the state database, every state table with its
    rows, keys, indexes and defaults, in the migration's own transaction. One log line names the copy.
    A copy that cannot be written (disk full, a read-only directory, a role without CREATE on the
    database) stops the migration with RIVET_STATE_BACKUP_FAILED (exit 1); the state is untouched and
    the previous release still opens it. What to do: nothing to upgrade. To roll back, follow
    Rolling back an upgrade: stop the schedulers,
    install the previous release, put the copy back (SQLite: replace the file; PostgreSQL: the SQL
    block there) and run once. Progress written after the upgrade is lost, so the rows delivered since
    are delivered again, at least once; what that means per mode and for rivet load is written there.

  • Behaviour change: a replay of a sealed plan that already applied re-ran it, and an incremental plan could be applied only once.
    rivet apply plan.json run again after that plan's apply had exited 0 (a scheduler retry whose
    task process died after rivet finished) extracted the whole table a second time for mode: full,
    chunked and keyset plans. An incremental plan made after a run had stored a cursor was refused
    RIVET_PLAN_ARTIFACT_STALE (exit 5, "cursor has drifted") on every apply after its first, by its
    own cursor commit, and any plan older than 24 h was refused (0.31.0 and earlier). Now a
    full, chunked or keyset plan is a point-in-time snapshot: the run that applies it to exit 0
    records the plan's plan_id (state schema v37, run_status.plan_id), and a replay of it exits 0
    without extracting, printing apply: plan <plan_id> of export '<export>' was already applied by run <run_id> (...) with that run's rows, files and bytes; it writes no run, manifest or metrics
    row. An incremental plan (mode: incremental, keyset_incremental) is a reusable description:
    every apply of it, a retry included, reads the cursor from state and delivers what arrived since,
    new rows or none; a cursor moved by an earlier apply of the same plan is not drift, and the 24 h
    limit no longer applies to it. A cursor moved by another run is still refused
    RIVET_PLAN_ARTIFACT_STALE, and a plan whose apply failed, was interrupted or died still
    replays. What to do: nothing for a retry. To extract a full, chunked or keyset plan again,
    make a new plan with rivet plan, or pass --force. Plans applied before the upgrade recorded no
    plan_id, so a replay of one behaves as before.

  • An export whose source table had been dropped failed with the raw driver error and no code.
    rivet run over a table dropped since the config was generated ended with the engine's own words
    as its Error: line and no RIVET_* code on MySQL and PostgreSQL (Error: MySqlError { ERROR 1146 (42S02): Table 'rivet.drop_probe_b' doesn't exist }, exit 1), or RIVET_SOURCE_QUERY_FAILED
    on SQL Server and Oracle; rivet check refused the same table as RIVET_SOURCE_QUERY_FAILED
    (0.30.0 and 0.31.0). Now one function recognises each engine's own verdict (MySQL 1146,
    PostgreSQL SQLSTATE 42P01, SQL Server 208, Oracle ORA-00942) on every path that reads the table
    (the batch read, the parallel runners, the chunk planner's probe, the CDC snapshot leg, check
    and plan), and the export stops as Error: [RIVET_SOURCE_TABLE_NOT_FOUND] export 'drop_probe_b': the table it reads does not exist (drop_probe_b, read by its query:). Correct the export's table:orquery:, or remove the export from exports: if the table was dropped on purpose: MySqlError { ERROR 1146 (42S02): Table 'rivet.drop_probe_b' doesn't exist } (exit 1). The exit
    stays 1 and the other exports of the same run still deliver (1 success · 1 failed), so alerting
    on the exit code sees no change; rivet plan keeps RIVET_PLAN_SOURCE_UNREADABLE as its outer
    code, as it does for a missing MongoDB collection, with this refusal as the cause. What to do:
    remove the export from exports: if the table was dropped on purpose, or correct its table:.

  • A relative destination.path followed the working directory, not the config. A local
    export whose path is relative (./output/orders, what rivet init writes) was resolved
    against the directory rivet was started from, while its cursor and state live beside the
    config: the same config run from a cron entry and from a shell delivered ids 1..10 under one
    directory and 11..13 under the other, each run green (P-27). Now a relative local path
    is anchored to the config file's directory, resolved once at config load and used the same
    way by run, plan/apply, validate, reconcile, repair, state, doctor and
    load; a cursor stored under the old relative spelling continues under the anchored path.
    Transition guard: a command started from another directory, whose working-directory
    place still holds the export's files (its manifest.json) while the directory beside the
    config holds none, is refused with RIVET_DEST_RELATIVE_PATH_MOVED (exit 5) naming both
    directories, before anything is read or written; the check is one decision in the config
    loader, so run, load, reconcile, repair, validate and every other command that
    resolves the path meet it, and it costs local file reads only. What to do: nothing for
    a run started from the config's directory, where the path resolves to the same place as
    before. After the refusal, move the files to the named directory beside the config (the
    stored cursor continues there), or make destination.path absolute to keep the old location.

  • reconcile and repair read a stored chunk run on an edited chunk_column. Neither
    command compared the chunk plan the stored run was planned under with the config, so after a
    chunk_column (or query, chunk_size, chunk_count, chunk_by_days) edit they re-read the
    stored windows on the new column, and a repair --execute superseded parts: the manifest listed
    39 rows for 40 source ids, exit 0 (P-21). Now both commands run the check rivet run --resume already runs and refuse with RIVET_STATE_CHUNK_PLAN_CHANGED (exit 5) before any window
    is read, naming the remedy. What to do: restore the chunk settings the run had to reconcile or
    repair it, or rivet state reset-chunks -c <config> --export <name> abandons it and the next run
    starts over. Exit codes: a reconcile or repair under a changed chunk plan exited 0 or 2 and
    exits 5 now.

  • Behaviour change: a Snowflake full load emptied its table before the COPY. The load ran
    CREATE OR REPLACE TABLE and then COPY as one script, so a COPY that failed (an unreadable
    part, a lost client, a killed snow) left the table with 0 rows until a later load succeeded
    (0.31.0 and earlier; measured: 20 rows before, 0 after a COPY of a corrupted part). Now the
    parts land in a new table <table>__rivet_load_<16 hex>, the landed row count is checked against
    the runs, and one CREATE OR REPLACE TABLE <table> CLONE <staging> COPY GRANTS puts it in place;
    any failure before that statement leaves the table as it was and drops the staging table. A table
    that existed before the upgrade is replaced like any other. Grants made on the table now survive a
    full load (COPY GRANTS); before, every full load dropped them. What to do: nothing; a load
    killed between the COPY and the clone can leave a <table>__rivet_load_* table, which no view or
    load reads: drop it.

  • A coded stop met by one chunk of a checkpointed export lost its code and its exit class. A
    range-chunked export with chunk_checkpoint: true (with or without parallel:) recorded a
    failed chunk's cause as text in chunk_task only, so the run always ended as the incomplete run
    RIVET_RUN_UNITS_FAILED with exit 1 and a --resume hint, and a --resume met the same stop and
    ended the same way, so a scheduler resumed a refusal forever. Present through 0.31.0. Now the
    run prints the code of the most stop-worthy coded chunk failure and exits by its class, as the
    same export does without chunk_checkpoint; the message lists every failed chunk under the resume
    remedy, and a --resume that meets the stop again refuses by the same code. The exits that
    change: a refusal in a chunk (e.g. RIVET_SOURCE_VALUE_UNREPRESENTABLE) 1 → 5; an integrity
    stop (e.g. RIVET_DEST_PART_ROWS_MISMATCH) 1 → 3; an internal one 1 → 6; a coded usage or
    environment stop 1 or 2 → its own code's exit. A chunk whose stored window does not parse now
    stops as RIVET_STATE_CHUNK_WINDOWS_BROKEN (exit 5) on the parallel checkpoint runner too. Chunk
    failures that carry no code still exit as RIVET_RUN_UNITS_FAILED. The sequential runner's
    message changed from chunk checkpoint incomplete (N tasks not completed); fix errors and ... to
    N chunk task(s) of export '...' not completed; fix the cause, then ... --resume, or inspect rivet state chunks ..., followed by each failed chunk's cause. What to do: a job that matched
    RIVET_RUN_UNITS_FAILED or exit 1 for these stops matches the stop's own code and exit class
    instead.

  • A stop caused by another live rivet process exited 5, "a human decides", while its remedy was to wait and run again. RIVET_STATE_RUN_IN_PROGRESS (a run, apply, a CDC drain, rivet state reset* or rivet repair beside a live run of the export), RIVET_LOAD_TABLE_BUSY and RIVET_STATE_LEASE_LOST (a rivet load or rivet compact beside another of the same table) exited 5, so a scheduler that does not retry a refusal (the Airflow provider, cron with retries keyed on the exit code) failed the task for good, although the same command succeeds once the other process ends (an overlapping trigger, an orphaned child of a killed parent). Behaviour change: these stops now have the kind busy and exit 2 (retryable); the codes and the texts are unchanged, the stop still writes nothing and never counts as ownership of the table (RIVET_STATE_LEASE_LOST is journaled refused; RIVET_LOAD_TABLE_BUSY stops before a load_run row exists and journals nothing). A rivet run of several exports where one is busy and another fails without a code now exits 2 (the busy stop ranks above an uncoded failure). A stop about what a dead run left behind keeps exit 5. What to do: a scheduler that branches on these codes needs no change; one that branches on exit 5 to catch an overlap matches exit 2 with the code instead.

  • The dest retries line of a --parallel-export-processes run counted a CDC child's retries
    more than once.
    Each child sends its process's running total of destination retries with every
    run it finishes, and a CDC child with initial: snapshot finishes one run per snapshot leg and then
    its own; the parent added every one of those totals, so a run whose leg retried R times and whose
    drain retried D times printed 2R + D, and more with more legs (0.31.0 and earlier with two or more
    legs; with one leg since the snapshot-leg summary change in this release). Now the parent adds
    only what each child's total grew by since that child's previous report. What to do: nothing.

  • rivet run on a PostgreSQL source crashed when the process had no file descriptor left for the
    driver's runtime.
    Under a low RLIMIT_NOFILE the PostgreSQL client's own runtime build hit
    Too many open files and the process ended with a panic (exit 101, no Error: line), while
    MySQL, SQL Server, Oracle and MongoDB failed loudly under the same cap (0.31.0 and earlier).
    Now every PostgreSQL dial (the source, CDC, the Postgres state, init, doctor, check, the
    MCP server) first proves the process can start a runtime and is refused
    RIVET_SOURCE_RUNTIME_UNAVAILABLE (an environment stop) when it cannot. What to do: raise the
    open-files limit of the process that runs rivet.

  • An Azure container removed while rivet run was writing into it was retried for minutes. Each
    chunk's upload was retried as a transient error (Azurite answers its 404 with a keep-alive: timeout=5 header, and the retry classifier read the word "timeout" in the store's answer as a
    timeout), so a run of ten chunks took over nine minutes to fail; a bucket removed on S3 or GCS
    failed in seconds (0.31.0 and earlier). Now a store's own final answer (not found, denied,
    invalid, unsupported) is permanent by its type, whatever words its headers carry, and a container
    that answered once but is gone by the time a write, listing or read fails is asked about again and
    refused RIVET_DEST_CONTAINER_NOT_FOUND. What to do: nothing.

  • The run summary of a CDC export's first run counted the stream only, not its snapshot. A
    mode: cdc export with cdc.initial: snapshot writes each table's baseline through a snapshot leg
    (<export>__snapshot_<table>) and then drains the stream; its per_export[] entry in
    --summary-output, --json and run_aggregate carried the drain's counts alone, so a run that wrote
    the whole table read rows: 0, files: 0 (and so did a scheduler task built on it), and a failed leg
    showed up as an entry named after the leg instead of the CDC export (0.31.0 and earlier). Now the
    CDC export's entry adds what its snapshot legs wrote in the same run to rows, files, bytes and
    bytes_read, whether the exports run in one process or as --parallel-export-processes children,
    and a failed leg fails the entry under the CDC export's name. The legs still have
    their own export_metrics rows, so rivet metrics counts each write once, and duration_ms stays the
    drain's. Under --parallel-export-processes the parent also took each export's entry as the newest
    finished metric row of the export's NAME, so on a shared state database a failed run was reported as
    another config's success of the same export name, with that run's id (0.31.0 and earlier); now
    each child reports its own run id and the parent reads that run's row, and a child with none is
    reported failed. What to do: nothing; summaries written by earlier runs are not rewritten.

  • A coded stop met by one parallel worker lost its code and its exit class. A keyset export
    with parallel: (chunk_by_key), a range-chunked export with parallel: (chunk_column,
    without chunk_checkpoint) and a MongoDB export with parallel: kept each worker's failure as
    text only, so the run always printed RIVET_RUN_UNITS_FAILED and exited 1, or 2 when any
    worker's text read as transient; a scheduler then retried a refusal it should have stopped on.
    Present through 0.31.0. Now the run prints the code of the most stop-worthy coded worker
    failure and exits by its class, as the same export does with one worker; the message still
    lists every failed unit, the deciding one last. The exits that change: a refusal in a worker
    (e.g. RIVET_SOURCE_VALUE_UNREPRESENTABLE) 1 or 2 → 5; an integrity stop (e.g.
    RIVET_DEST_PART_ROWS_MISMATCH) → 3; an internal one → 6; a usage one (e.g.
    RIVET_SOURCE_OVERRIDE_WIRE_MISMATCH, RIVET_SOURCE_VALUE_TOO_LARGE) 2 → 1 when a sibling's
    text read as transient; an environment one keeps its own code and exits 2 only when its own
    text is transient. Worker failures that carry no code still exit as RIVET_RUN_UNITS_FAILED.
    What to do: a job that matched RIVET_RUN_UNITS_FAILED or exit 1/2 for these stops matches
    the stop's own code and exit class instead.

  • A stored incremental cursor moved past rows the export never delivered was continued from.
    With last_cursor_value edited above the last delivered value, the next run exported only the
    rows past it and exited 0, while the destination's own manifest.json ended below it (every
    version through 0.31.0). Now a run that continues from a stored cursor (incremental,
    keyset_incremental) compares it with the cursor_high of the success manifest at its
    destination and refuses a cursor past it with RIVET_STATE_CURSOR_PAST_MANIFEST (exit 5); a
    cursor equal or below, a destination with no success manifest of this export, or one that keeps
    no objects (a warehouse load) proceed as before. What to do: rivet state reset -c <config> --export <name> starts the cursor over with a full pass.

  • rivet could only be installed from a tarball, Homebrew or cargo, which a managed Airflow worker
    (Cloud Composer, MWAA, Astronomer) does not allow.
    Now a release can publish the same
    binaries as rivet-cli wheels on PyPI (manylinux_2_28 x86_64 / aarch64 and macOS arm64 /
    x86_64; rivet and rivet-mcp land in the environment's bin, and rivet_cli.path() returns
    the installed rivet). What to do: nothing; once a release is published there,
    pip install rivet-cli installs the binary.

  • The Linux release binaries did not start on Debian 12, RHEL 8/9 or the official Airflow images.
    They were built on Ubuntu 24.04 and needed GLIBC_2.38 / 2.39: version 'GLIBC_2.39' not found
    (0.31.0 and earlier releases built the same way). Now every release also ships
    rivet-<version>-x86_64-unknown-linux-gnu-glibc2.28.tar.gz and …-aarch64-…-glibc2.28.tar.gz,
    built in manylinux_2_28, which run on any glibc 2.28 or newer; the release checks that they need
    nothing newer and that they start on Rocky Linux 8 and 9, Amazon Linux 2023, Debian 12 and
    Airflow 2.10 / 3.3 before it publishes. What to do: on an older distribution or in an Airflow
    image, take the -glibc2.28 asset.

  • A rivet load stopped while selecting its runs left no ledger row, and a missing load spec hid a state-DB fault. A stop raised before any warehouse write but outside the load's typed refusal seam (a --pool --split prefix holding two split generations, RIVET_LOAD_SPLIT_GENERATIONS_MIXED; a shared prefix; an unreadable loaded set) propagated untyped and unrecorded, so rivet state loads showed no attempt at all, and the one text under RIVET_LOAD_SPEC_MISSING covered both "no column types recorded" and "the state DB could not be read". Since 0.27.0. Now every stop out of run selection is journaled refused (never counted as rivet's own attempt) and the next cycle refuses again by the same code; a state DB that cannot answer for the load spec stops as RIVET_STATE_BACKEND_FAILED, and a missing spec stays RIVET_LOAD_SPEC_MISSING. What to do: nothing.

  • The Airflow provider package moved to its own repository. integrations/airflow (the
    airflow-provider-rivet package: operators for run, load and compact, retry by error class,
    CDC) left the rivet repository for
    panchenkoai/airflow-provider-rivet, with its
    history, so the Rust repository does not publish to PyPI. Now the scheduler contract fixtures
    (tests/fixtures/scheduler/) stay here and the provider's CI reads them from a pinned rivet
    checkout. What to do: install the provider from its own repository; nothing changes in rivet.

  • An empty table under chunk_by_days stopped the run instead of exporting zero rows. The date
    planner read the NULL min of a table with no row as an unreadable bound and stopped with
    min(<column>) returned NULL — table may be empty (since date-native chunking in 0.2.0-beta.3, through
    0.31.0; exit 5 RIVET_SOURCE_PROBE_UNREADABLE once the stop carried a code), while the same table
    range-chunked on an integer key exported zero rows with exit 0. Now both planners read the key's
    span through one function: a table with no row plans one empty window, so the run exits 0 and writes
    _SUCCESS and a manifest of 0 rows, as the range run does; a bound absent over a table that has rows
    is still refused RIVET_SOURCE_PROBE_UNREADABLE, and a NULL key is still refused
    RIVET_PLAN_CHUNK_KEY_NULL. What to do: nothing.

  • A keyset run stopped before its first part blocked the remedy its refusal named. A resumed
    MongoDB keyset run refused for an _id a $gt seek cannot page over (mixed BSON types, NaN) had
    already claimed the export's resume anchor, so the full scan the refusal recommends was refused
    RIVET_STATE_INTERRUPTED_RUN_OWNER_MISMATCH (exit 5) on every run, through 0.31.0. The same held
    for any keyset export whose run failed before it recorded a part. Now a run that ends with a
    resume anchor under which no part was recorded releases it, so the next run of any mode starts
    fresh; an anchor with recorded parts is kept for the resume as before. The refusal
    (RIVET_SOURCE_KEYSET_KEY_UNORDERED) now names every key to remove: page_size, resume and
    parallel (resume: true without page_size is itself refused). What to do: nothing for a
    new refusal. An export already stuck on an anchor a 0.31.0 run left: run the rivet state reset
    command the RIVET_STATE_INTERRUPTED_RUN_OWNER_MISMATCH refusal prints (that run recorded no part).

  • Every remaining error rivet raises outside its source readers carried no RIVET_* code. The
    CLI, destination, plan, init, state, preflight, format, manifest, MCP, type-override and TLS
    stops (152 sites) printed prose only, so a scheduler could match on wording alone, since the
    codes were introduced. Now every stop carries a registered code ([CODE] prefix on the text
    line, code in --json-errors), with 28 new codes in docs/reference/errors.md; message text
    is unchanged. A stop's exit follows its code's class, which moves these stops from exit 1 to the
    class exit: rivet init -o over an existing config (RIVET_INIT_CONFIG_EXISTS, 5); an export
    whose destination prefix holds a manifest of the other mode (RIVET_DEST_PREFIX_MODE_MISMATCH,
    5); a Full load over a --pool --split prefix spanning two split generations
    (RIVET_LOAD_SPLIT_GENERATIONS_MIXED, 5); rivet load for an export with no recorded column
    types (RIVET_LOAD_SPEC_MISSING, 5); a state migration that did not reach the binary's schema
    version, an empty state path, a run aggregate that does not serialise
    (RIVET_INTERNAL_STATE_INVARIANT, 6); a destination backend method the backend does not
    implement, a destination key with no file name, a CSV cell type the writer never rejected
    (RIVET_INTERNAL_DEST_INVARIANT, 6); a row_hash column gone from the batch or read as the
    wrong array kind, a worker panic, a COUNT(*) that is not a number, a committed cursor with no
    column to record it under (RIVET_INTERNAL_RUN_INVARIANT, 6); an Oracle current schema the
    server did not return, a MySQL MCP query with no result set (RIVET_INTERNAL_SOURCE_INVARIANT,
    6). What to do: a script that keyed on exit 1 for one of these stops treats 5 as "a human
    decides, do not retry unchanged" and 6 as "a bug, report it"; the text is unchanged, so a script
    matching text keeps working. Other stops keep their exit: usage codes exit 1 and environment
    codes exit 2 when transient, else 1, as before.

  • Most errors raised while reading a source carried no RIVET_* code. A malformed source
    URL, a CDC table: that names no single base table, a value the column type cannot hold, a
    SQL Server statement the server refused, a MySQL server with no binlog, a spill log that
    would not flush, a pgoutput message in a shape the reader never produces, and the other stops
    raised under src/source/ (PostgreSQL, MySQL, SQL Server, Oracle, MongoDB and the shared CDC,
    spill, checksum and validate modules) printed only their text, and most exited 1 whatever
    their kind (0.31.0 and earlier). Now every error raised there carries a registered code,
    with the message text unchanged: RIVET_CONFIG_SOURCE_URL_INVALID,
    RIVET_CONFIG_SOURCE_OPTION_UNSUPPORTED, RIVET_CONFIG_SOURCE_FEATURE_MISSING,
    RIVET_SOURCE_CDC_TABLE_UNRESOLVED, RIVET_SOURCE_CDC_TABLE_UNSUPPORTED,
    RIVET_SOURCE_CDC_TRANSACTION_TOO_LARGE, RIVET_SOURCE_QUERY_COLUMN_UNREFERENCEABLE (usage,
    exit 1); RIVET_SOURCE_KEYSET_KEY_UNORDERED (refusal, exit 5); RIVET_SOURCE_RUNTIME_UNAVAILABLE,
    RIVET_SOURCE_CONNECT_FAILED, RIVET_SOURCE_QUERY_FAILED, RIVET_SOURCE_CDC_SPILL_UNLOCKABLE,
    RIVET_SOURCE_CDC_SPILL_UNWRITABLE (environment, exit 2 when transient, else 1);
    RIVET_INTERNAL_SOURCE_INVARIANT (internal, exit 6). Existing codes now also cover source
    sites (RIVET_TYPE_UNSAFE_MAPPING, RIVET_SOURCE_VALUE_UNREPRESENTABLE,
    RIVET_SOURCE_OVERRIDE_WIRE_MISMATCH, RIVET_SOURCE_VALUE_TOO_LARGE,
    RIVET_SOURCE_COLUMNS_CHANGED_UNDER_READ, RIVET_SOURCE_CDC_PREREQUISITE,
    RIVET_SOURCE_CDC_CELL_UNSUPPORTED, RIVET_SOURCE_CDC_UNDECODABLE, RIVET_SOURCE_CDC_TRUNCATED,
    RIVET_SOURCE_CDC_CHECKPOINT_INVALID, RIVET_CONFIG_CDC_CAPTURE_INSTANCE_MISMATCH,
    RIVET_CONFIG_MODE_SETTING_INVALID, RIVET_CONFIG_EXPORT_SOURCE_INVALID,
    RIVET_CONFIG_VALUE_OUT_OF_RANGE, RIVET_LOAD_RESERVED_COLUMN, RIVET_STATE_CURSOR_INCOMPARABLE,
    RIVET_DEST_CONTROL_ARTIFACT_INVALID, RIVET_VALIDATE_UNVERIFIED, RIVET_INTERNAL_SPILL,
    RIVET_INTERNAL_TYPE_BUILDER, RIVET_INTERNAL_VALUE_CONVERTER). An Oracle driver error is
    coded by phase (connect against statement) with the server's words kept; a MySQL anchor refused
    because the identity query failed keeps that query's own code; a pgoutput transaction poisoned
    by a refusal re-raises the original cause with its code. See docs/reference/errors.md.
    What to do: nothing; a script matching error text keeps working. A scheduler that keys on
    the exit code sees 5 where these stops used to exit 1: a batch value the column's type cannot
    hold (a PostgreSQL multi-dimensional array or an array of temporal/uuid/bytea elements, a
    NUMERIC NaN or infinity under a decimal target, a DECIMAL that does not fit the declared
    scale on PostgreSQL or MySQL, a SQL Server MONEY that is non-finite or past 2^53, a SQL Server
    decimal that overflows on rescale, an Oracle date, time or NUMBER outside its column type); a
    CDC cell the decoder has no faithful reading for (a deferred PostgreSQL infinity or
    unchanged-TOAST cell, a pgoutput column type with no binary decoder or no known OID, an
    Oracle value the miner cannot decode); a MySQL XA PREPARE or statement-logged DML on a
    captured table, an Oracle row written without ALL COLUMNS supplemental logging, and a MongoDB
    event past 16 MB (RIVET_SOURCE_CDC_UNDECODABLE); a MongoDB change stream invalidated
    (RIVET_SOURCE_CDC_TRUNCATED); a malformed SQL Server resume LSN or an unrecognised MongoDB
    resume-token checkpoint (RIVET_SOURCE_CDC_CHECKPOINT_INVALID); a MongoDB _id span a seek
    cannot page over, and a stored MongoDB keyset cursor token that no longer decodes
    (RIVET_STATE_CURSOR_INCOMPARABLE); a manifest column checksum that is not a number
    (RIVET_DEST_CONTROL_ARTIFACT_INVALID, under rivet validate). It sees 6 where a source
    reader invariant broke: a truncated or unexpected pgoutput message, a row before its Relation,
    a row image whose arity differs from its relation, a text-form cell on a binary slot, a
    spilled row that decodes to no change, a malformed slot LSN or position, a SQL Server open-time
    LSN or sys.databases answer in a shape the server never gives, a MySQL query with no result
    set, an Oracle SCN the server did not return or a mined row of an unexpected table, a MongoDB
    resume token that is not a document or a non-row operation reaching the row decoder, a column
    builder given a type it cannot build (RIVET_INTERNAL_TYPE_BUILDER), a value checksum that
    changed between read and Arrow build or a MONEY render that did not parse
    (RIVET_INTERNAL_VALUE_CONVERTER), and a spill frame that is truncated, oversized, nested too
    deep, not UTF-8, not a date-time, not JSON or followed by trailing bytes
    (RIVET_INTERNAL_SPILL). Environment stops keep their exit: 2 when the failure is transient,
    else 1, as before.

  • Most errors raised while running, planning, applying, reconciling, repairing and validating
    an export carried no RIVET_* code.
    An unknown --export, a stale plan artifact, a NULL
    chunk key, an unfinished chunk checkpoint run, a committed part gone from the destination, a
    damaged manifest.json, failed chunks or keyset ranges, and the other stops raised under
    src/pipeline/ printed only their text, and most exited 1 whatever their kind (0.31.0 and
    earlier). Now every error raised there carries a registered code, with the message text
    unchanged: RIVET_CLI_EXPORT_UNKNOWN, RIVET_CLI_RUN_ID_EXPORT_MISMATCH,
    RIVET_CLI_OPTION_UNSUPPORTED, RIVET_CLI_REPORT_INVALID, RIVET_PLAN_NOTHING_TO_PLAN,
    RIVET_PLAN_ARTIFACT_NO_CREDENTIALS, RIVET_PLAN_PARTITION_TOKEN_UNEXPANDED,
    RIVET_PLAN_REJECTED, RIVET_PLAN_CHUNK_KEY_SPARSE, RIVET_CONFIG_CHUNK_COLUMN_NOT_INTEGER,
    RIVET_CONFIG_SETTLE_COLUMN_NOT_TEMPORAL, RIVET_CONFIG_SPLIT_PREFIX_UNSTABLE,
    RIVET_CONFIG_EXPORT_NAME_RESERVED, RIVET_CONFIG_BATCH_MEMORY_EXCEEDED,
    RIVET_CONFIG_CDC_CAPTURE_INSTANCE_MISMATCH, RIVET_SOURCE_VALUE_TOO_LARGE,
    RIVET_STATE_CHUNK_RUN_MISSING, RIVET_DEST_MANIFEST_MISSING (usage, exit 1);
    RIVET_PLAN_ARTIFACT_STALE, RIVET_PLAN_CHUNK_KEY_NULL, RIVET_SOURCE_KEYSET_KEY_COARSE,
    RIVET_STATE_CHUNK_RUN_UNFINISHED, RIVET_DEST_COMMITTED_PART_GONE,
    RIVET_DEST_CONTROL_ARTIFACT_INVALID (refusals, exit 5); RIVET_DEST_PART_ROWS_MISMATCH
    (integrity, exit 3); RIVET_RUN_UNITS_FAILED, RIVET_RUN_RECONCILE_UNVERIFIED,
    RIVET_DEST_MANIFEST_NOT_WRITTEN, RIVET_DEST_OBJECT_NOT_FOUND, RIVET_DEST_UPLOAD_CORRUPTED,
    RIVET_DEST_PREFIX_UNLISTABLE, RIVET_CLI_OUTPUT_UNWRITABLE, RIVET_CLI_INPUT_UNREADABLE,
    RIVET_CLI_EXECUTABLE_UNRESOLVED, RIVET_CLI_TEST_FAULT_INJECTED (environment, exit 2 when
    transient, else 1); RIVET_INTERNAL_RUN_INVARIANT (internal, exit 6). Some existing codes now
    also cover pipeline sites (RIVET_SOURCE_PROBE_UNREADABLE, RIVET_STATE_CHUNK_WINDOWS_BROKEN,
    RIVET_SOURCE_CDC_PREREQUISITE, RIVET_CONFIG_MODE_SETTING_INVALID,
    RIVET_CONFIG_CDC_SHAPE_INVALID, RIVET_LOAD_KEY_COLUMN_INVALID, RIVET_CONFIG_NO_EXPORTS,
    RIVET_CONFIG_FILE_UNREADABLE, RIVET_VALIDATE_UNVERIFIED, RIVET_STATE_NOT_WRITABLE). An
    error rebuilt around another (the chunked planner's probe around resolve_url, a worker's
    state-database open, state reset / state accept around the source) keeps the inner error's
    code instead of dropping it. See docs/reference/errors.md. What to do: nothing; a script
    matching error text keeps working. A scheduler that keys on the exit code sees 5 where these
    stops used to exit 1: a plan artifact older than 24 h or with a drifted cursor applied without
    --force; NULL values in a range, date or --pool --split chunk key; a keyset page that ends
    on the key it started after; a plain run meeting an unfinished chunk checkpoint run; a
    committed part gone from the destination on resume; a manifest.json that does not parse or
    exceeds the read cap; a min/max/partition bound the planner cannot read, or a keyset key value
    it cannot read from a page's last row (RIVET_SOURCE_PROBE_UNREADABLE); a stored chunk window
    bound that is not an integer (RIVET_STATE_CHUNK_WINDOWS_BROKEN). It sees 3 where a part
    written under validate: true holds fewer rows than counted, and 6 where a run invariant
    broke (a COUNT(*) with no row, a chunk run recorded with no tasks, a reconcile over a
    non-chunked plan, an encode worker that stopped or panicked, a retry loop with no error to
    report).

  • Range chunking refused an Oracle NUMBER(19) key. A NUMBER(19) primary key (the type a
    64-bit integer key usually has on Oracle) was read as not integer-family, so mode: chunked over it
    exited 1 (0.31.0 and earlier). Now NUMBER(p, 0) up to precision 19 is an integer range key for
    the planner and rivet check; a value past the 64-bit range is refused when the bounds are read,
    with a message that says so. rivet init still proposes keyset for such a key, which takes any
    value. What to do: nothing.

  • Most rivet load and rivet compact errors carried no RIVET_* code. A refused warehouse
    object, an unsafe identifier, a missing key, an incomplete run under the prefix, a failed
    BigQuery job and the other stops raised under src/load/ printed only their text, and most exited
    1 whatever their kind (0.31.0 and earlier). Now every error raised there carries a registered
    code, with the message text unchanged: RIVET_LOAD_IDENTIFIER_UNSAFE,
    RIVET_LOAD_KEY_COLUMN_INVALID, RIVET_LOAD_PRIMARY_KEY_MISSING, RIVET_LOAD_RESERVED_COLUMN,
    RIVET_LOAD_LOOKALIKE_COLUMN, RIVET_LOAD_TARGET_COLLISION, RIVET_LOAD_STAGING_PREFIX_INVALID,
    RIVET_LOAD_DESTINATION_TYPE_UNSUPPORTED, RIVET_LOAD_PARTITION_BUDGET_EXCEEDED,
    RIVET_LOAD_FILE_LIMIT_EXCEEDED, RIVET_LOAD_COLUMN_TYPE_UNSUPPORTED, RIVET_LOAD_TLS_CA_INVALID
    (exit 1); RIVET_LOAD_EMPTY_RUN_UNSUPPORTED, RIVET_LOAD_RUN_NOT_LOADABLE,
    RIVET_LOAD_STAGING_OBJECT_UNSAFE, RIVET_LOAD_SOURCES_SHARED, RIVET_LOAD_COLUMN_RESPELLED,
    RIVET_LOAD_TARGET_OBJECT_CONFLICT, RIVET_LOAD_COMPACT_BASE_MISSING,
    RIVET_LOAD_CHANGELOG_PARTITION_CHANGED, RIVET_LOAD_BUFFER_NOT_COMPACTED,
    RIVET_LOAD_REBASELINE_OVER_CHANGES, RIVET_LOAD_TABLE_BUSY (refusals, exit 5);
    RIVET_LOAD_MANIFEST_INCONSISTENT, RIVET_LOAD_SOURCE_FILE_MISMATCH,
    RIVET_LOAD_COPY_COUNT_MISMATCH (integrity, exit 3); RIVET_LOAD_LEDGER_UNREADABLE,
    RIVET_LOAD_RUN_LANDED_DURING_LOAD, RIVET_LOAD_WAREHOUSE_AUTH_FAILED,
    RIVET_LOAD_WAREHOUSE_ACCESS_DENIED, RIVET_LOAD_WAREHOUSE_OBJECT_NOT_FOUND,
    RIVET_LOAD_WAREHOUSE_REJECTED, RIVET_LOAD_WAREHOUSE_UNAVAILABLE (environment, exit 2 when
    transient, else 1); RIVET_LOAD_WAREHOUSE_ANSWER_UNREADABLE and RIVET_INTERNAL_LOAD_INVARIANT
    (internal, exit 6). Some existing codes now also cover load sites (RIVET_CONFIG_NO_EXPORTS,
    RIVET_CONFIG_MODE_SETTING_INVALID, RIVET_CONFIG_SOURCE_MODE_UNSUPPORTED,
    RIVET_CONFIG_LOAD_PARTITION_UNSUPPORTED, RIVET_LOAD_TARGET_NOT_RIVETS). A Snowflake or
    ClickHouse stop before the first statement (an empty newest run, more than 1,000 files, an unsafe
    cluster_by, a CDC log without a key) is now journaled refused, not failed, so it no longer
    counts as rivet having loaded the table. See docs/reference/errors.md. What to do: nothing;
    a script matching error text keeps working. A scheduler that keys on the exit code sees 5, 3 or 6
    where these stops used to exit 1.

  • A float chunk_column under query: lost its fractional keys with exit 0. With table: the
    planner refused a non-integer range key; with query: it could not read a catalog, only warned,
    and integer BETWEEN windows skipped every key between two whole numbers (a double precision key
    delivered 40 of 43 rows) (0.31.0 and earlier). Now the run reads the query's result type before
    it plans the windows and refuses a float or scaled-decimal key with
    RIVET_CONFIG_CHUNK_COLUMN_NOT_INTEGER (exit 1), the same code the table: refusal now carries.
    What to do: use chunk_by_key: (keyset) or an integer column for such a key.

  • Most config errors carried no RIVET_* code. A missing config file, broken YAML, an
    incomplete source: block, an unset url_env, a mode without its setting and the other
    refusals raised while a config is read printed only their text, so a scheduler or a script could
    match them by wording alone (0.31.0 and earlier). Now every error raised under src/config/
    carries a registered code, with the message text unchanged: RIVET_CONFIG_FILE_UNREADABLE,
    RIVET_CONFIG_YAML_INVALID, RIVET_CONFIG_SOURCE_CONNECTION_INVALID, RIVET_CONFIG_SECRET_UNSET,
    RIVET_CONFIG_EXPORT_SOURCE_INVALID, RIVET_CONFIG_MODE_SETTING_INVALID,
    RIVET_CONFIG_DESTINATION_AUTH_INVALID, RIVET_CONFIG_DESTINATION_ENDPOINT_UNSAFE,
    RIVET_CONFIG_VALUE_OUT_OF_RANGE, RIVET_CONFIG_CDC_SHAPE_INVALID,
    RIVET_CONFIG_CSV_COMPRESSION_UNSUPPORTED, RIVET_CONFIG_TLS_VERIFICATION_DOWNGRADE,
    RIVET_CONFIG_LOAD_BLOCK_INVALID, RIVET_CONFIG_PATH_UNSAFE, RIVET_CONFIG_ROW_HASH_INVALID and
    RIVET_CONFIG_COLUMN_OVERRIDE_TABLE_UNKNOWN (see docs/reference/errors.md). The exit code stays

    1. What to do: nothing; a script matching config error text keeps working.
  • rivet run into a bucket that does not exist failed with the store's raw error. The upload
    ended with NotFound (persistent) at write, context: { uri: ... } and no RIVET_* code (exit 1),
    while rivet validate and every read already named the missing bucket (0.31.0 and earlier).
    Now a write the store answers "not found" asks whether the bucket exists, and a missing one is
    refused RIVET_DEST_CONTAINER_NOT_FOUND naming it, on S3, GCS and Azure. What to do: nothing.

  • Two exports of one name on a shared state refused each other even when they read different
    sources.
    Since every run took the export's run lease (keyed by the export name), the
    shared-state deployment the state docs describe (several configs, one export name, one PostgreSQL
    state, different sources) ran one export at a time and refused the rest with
    RIVET_STATE_RUN_IN_PROGRESS. Now the run lease is keyed by the export and its source, as the
    cursor is: runs of one name on different sources run side by side, a second run on the same source
    is still refused. A run under its lease closes only its own source's dead running rows; state
    schema v36 records the source on each run_status row (rows from before v36 have none and are
    closed by any run of the export). What to do: nothing.

  • rivet init could not scaffold a Snowflake load: block. It had flags for BigQuery and
    ClickHouse only, so a Snowflake config needed a hand-written load: block (0.31.0 and earlier).
    Now --snowflake-connection, --snowflake-warehouse, --snowflake-database,
    --snowflake-schema and --snowflake-storage-integration (all five, with --gcs-bucket) write
    a target: snowflake block with pk: auto, cluster_by: auto, cleanup_source: true and a
    per-table partition: guess at day (a leading DATE_TRUNC clustering key on Snowflake). A
    missing companion flag, --s3-bucket, no bucket, or a second warehouse's flags are refused at the
    flag. What to do: nothing.

  • The end-of-run summary of a failed export re-read its error text from every ; . The step
    that strips the inline rivet … recovery hint was quadratic in the message: a 512 KiB message
    made of ; took 21 s (×4 per doubling) before the summary printed (0.31.0 and earlier).
    Now the cut is found in one pass each way; the summary of the same message prints in
    milliseconds. What to do: nothing.

  • rivet compact on a BigQuery base partitioned by hour planned one MERGE job per 166 days of
    the key's whole value span.
    A row holding the end-of-time sentinel 9999-12-31 (or any date
    outside BigQuery's partitioned days, 1960-01-01 to 2159-12-31) stretched the span to the sentinel:
    17,544 serial MERGE jobs for two changed rows (0.27.0 through 0.31.0; monthly and yearly keys planned
    fewer). Now the days outside the partitioned range, which BigQuery keeps in one unpartitioned
    partition, take one MERGE per side, and only the span of values inside the range is stepped.
    What to do: nothing.

  • rivet init --mode cdc on SQL Server named a capture instance that did not exist. It wrote
    capture_instance: <schema>_<table> by convention, so a table whose CDC was enabled under another
    instance name got a config that failed on its first run (0.31.0 and earlier). Now init reads
    cdc.change_tables and writes the instance it holds (the newest of two); a table without CDC gets
    the conventional name with the sp_cdc_enable_table call that creates it. What to do: nothing.

  • A SQL Server password holding @ reached the state database. In the keyword/value URL form
    (sqlserver://host;user=sa;password=p@ss;…) the @ inside the password was read as the end of a
    userinfo, so the source key rivet stores beside every cursor kept the password's tail (0.31.0 and
    earlier). Now keyword passwords are masked before anything is cut from the URL. What to do:
    if such a password was in use, the progress stored under the old key is not found and the next run
    of an incremental export reads its table again; rotate that password, since the state database
    held part of it.

  • A MySQL CDC stream refused a TIME past 24 hours with the batch remedy, "cast it in a query",
    which a stream has no query for.
    The check also ran by the column's native type, so a
    columns: { col: string } override could not carry the duration as text (0.31.0 and earlier).
    Now the refusal (RIVET_SOURCE_VALUE_UNREPRESENTABLE, exit 5) names the two remedies a stream
    has: the column declared string in columns:, which carries the value as text
    (800:00:00.000000), or the column left out of the capture followed by the re-baseline recipe.
    What to do: nothing.

  • mode: cdc on a standalone MongoDB failed with the driver's raw dump. The run exited 1 with
    Error code 40573 … RawDocumentBuf { data: "7d00…" } and no RIVET_* code (0.31.0 and earlier).
    Now it is refused RIVET_SOURCE_CDC_PREREQUISITE, naming the standalone and the ways out (a
    replica-set member, a one-member replica set, or mode: full). What to do: nothing.

  • rivet run refused a MongoDB collection that does not exist without its code. rivet check
    refused it RIVET_SOURCE_COLLECTION_NOT_FOUND; rivet run printed the same sentence with no
    RIVET_* code (exit 1) (0.31.0 and earlier). Now both raise it through one function and run
    carries the code. What to do: nothing.

  • A cursor_column spelled in another case than its column failed with the raw database error on
    PostgreSQL and Oracle.
    MySQL and SQL Server refused it with RIVET_CONFIG_CURSOR_COLUMN_CASE; on
    PostgreSQL and Oracle the quoted name failed the query first, so the run exited 1 with
    column "UPDATED_AT" does not exist / ORA-00904 and no code (0.31.0 and earlier). Now a failed
    incremental query reads the column names of the export's query, and a cursor that matches one only
    when case is ignored is refused with RIVET_CONFIG_CURSOR_COLUMN_CASE on every engine, naming the
    spelling to use. What to do: nothing.

  • Two rivet runs of one mode: cdc export at once both delivered the stream. A batch run
    holds the export's run lease and a second run beside it is refused; a CDC run took no lease, so
    the second read the same changes again (MySQL, SQL Server, Oracle and MongoDB delivered duplicate
    change rows; on PostgreSQL the first run died on the busy slot with an uncoded error) (0.31.0 and
    earlier). Now a CDC run holds the run lease for its whole stream and a second one is refused
    with RIVET_STATE_RUN_IN_PROGRESS (exit 5) before it opens the source. What to do: nothing.

  • rivet apply --pool wrote its own lines into a destination: stdout export. The predicted
    and actual makespan lines (and the wave headers of a multi-wave run) went to stdout, mixed into the
    exported CSV or JSON bytes (0.31.0 and earlier). Now they go to stderr, like every other run
    line. What to do: a script that read the Pool: lines from stdout reads them from stderr.

  • A run that died (killed, out of memory, out of disk) stayed running in the state forever.
    Its run_status and export_metrics rows were closed only by the run itself, or by hand with
    rivet state finish-run; rivet state runs and rivet metrics kept showing it in progress
    (0.31.0 and earlier; a chunk-checkpoint resume closed the rows of the run it adopted, nothing
    else did). Now a run that takes the export's run lease stamps that export's earlier running
    rows interrupted in the same transaction as its own start: while it holds the lease no other
    run of the export can be alive. What to do: nothing; state finish-run still closes a row by
    hand.

  • An Oracle CDC checkpoint the retained redo no longer reaches was refused without a code.
    When the archived logs between the checkpoint and the oldest available log are gone (or a
    redo sequence is missing), the run stopped with the data-loss text and the re-baseline recipe
    but as a plain error: exit 1, no RIVET_* code, so a scheduler could not tell it from a
    transient failure and retried it forever (0.31.0 and earlier). Now it is refused as
    RIVET_SOURCE_CDC_LOG_GAP (exit 5), like the same loss on PostgreSQL, MySQL and SQL Server.
    What to do: nothing; the recipe in the text re-baselines the stream.

  • partition_by refused a PostgreSQL table with a NUMERIC(p,s) column. Each partition is read
    through a window that wraps the query, and the type hints PostgreSQL needs for NUMERIC precision
    were looked up in that wrapper, which names no table: the run failed with precision/scale unavailable from query metadata and catalog lookup, although the same export without
    partition_by ran (0.31.0 and earlier; a --split unit wraps the query the same way). Now
    the plan keeps the query before the window (catalog_query) and every reader resolves type hints
    from it. What to do: nothing.

  • A stored incremental cursor of another type stopped every run with a raw driver error.
    When the stored cursor is not a value the cursor column can be compared with (the state was
    edited, or the column changed type), PostgreSQL, SQL Server and Oracle failed every run with
    their own conversion error (invalid input syntax for type bigint, Error converting data type varchar to bigint), exit 1, no code and no remedy (0.31.0 and earlier; MySQL coerces the text
    and delivers). Now the run is refused with RIVET_STATE_CURSOR_INCOMPARABLE (exit 5), which
    names the cursor, the column and rivet state reset. What to do: nothing; an export stuck on
    this error is led out by rivet state reset -c <config> --export <name>.

  • A MySQL table without a primary key could be paged on a different unique key from one
    run to the next.
    With no chunk_by_key: the planner pages such a table by keyset on "the
    first" single-column unique NOT NULL key, and the list came back in whatever order the
    server's catalog returned it; a table with two such keys (id, ext_id) was paged on id
    by one run and on ext_id by the next, so a run that resumed or continued stored progress
    was refused (RIVET_STATE_CURSOR_OWNER_MISMATCH, exit 5) though nothing had changed. Seen
    on main CI on 2026-10-14 in a crash-then-resume cell. Now the keys are ordered by rivet:
    the primary key first, then by column name, on every engine. What to do: nothing; an
    export refused that way runs after rivet state reset, as the text says. Pin the key with
    chunk_by_key: where the choice matters.

  • A load could be refused as "another rivet load ... is writing" right after its own
    process released that table.
    SQLite state only. The load lease is an flock, and it was
    released by closing the file; a process spawned on another thread at that moment kept a copy
    of the descriptor until its exec, and with it the lock. The next load of the same table in
    this process was refused (exit 5) though nothing was writing it. Now the lease is unlocked
    explicitly before the file is closed. What to do: nothing; a refused load can be run again.

  • A resumed range-chunk run trusted the checkpoint and the parts it continued. A
    checkpoint or a committed part changed between the crash and the resume was continued
    as it stood, and the run exited 0 over wrong data. Measured on 0.31.0 with 40 source rows,
    each case through both rivet run and rivet run --resume:

    • a pending chunk task marked completed: 35 rows delivered;
    • a task whose upper bound was lowered, leaving a hole between two windows: 38 rows;
    • two stored windows that share keys: 43 rows, 3 of them twice;
    • a committed part replaced by its sibling's bytes: 35 distinct ids, one chunk's rows
      twice and another's missing; a truncated part was declared as it was;
    • a run planned under another chunk_size was refused with no code (exit 1), after
      rewriting manifest.json.
    • Now a committed part that is gone, or is not the size the state DB logged, has its
      chunk read again, and so does a chunk marked completed with no part. Keys between the
      stored windows are planned again. Windows that share keys are refused
      (RIVET_STATE_CHUNK_WINDOWS_BROKEN, exit 5), and so is a changed chunk plan
      (RIVET_STATE_CHUNK_PLAN_CHANGED, exit 5); both texts name rivet state reset-chunks,
      and the plan refusal also names restoring the settings. Under verify: content each
      committed part is read back too: a row count other than the logged one or a key outside
      its chunk's window has the chunk read again, which catches a part replaced at its own
      size.
    • What to do: nothing for runs that were not edited by hand. An export resumed by 0.31.0
      or earlier after its state DB or destination was edited may hold the wrong rows; run it
      again whole.
  • rivet load into Snowflake stored a timestamp column as a wrong date, and several
    loads failed outright.
    Snowflake target only; BigQuery and ClickHouse loads are not
    touched.

    • Wrong values with exit 0 (measured on 0.24.2; the statements are the same through
      0.31.0). A column the source types as a timestamp without a time zone (PostgreSQL
      timestamp, MySQL datetime) was read from the Parquet part as its raw microsecond
      integer and stored as that many seconds: 2026-10-13 21:25:02.145628 landed as year
      56785866. The load printed LOAD OK and the row count matched. A timestamptz column
      was stored correctly.
    • Loads that failed in 0.31.0 and earlier: a table with a TIME column (Failed to cast variant value ... to TIME); a table with a binary column, bytea or VARBINARY
      (expecting BINARY ... but got VARIANT); any table whose destination prefix is written
      without a trailing / (Remote file ... was not found: the stage URL and the file name
      were joined without a slash).
    • Loads that failed from 0.26.0 through 0.31.0: every load onto a free table name
      (object-kind probe ran but KIND_ was not in snow's output), and the cycle of an
      incremental or CDC table whose export gained a column (a syntax error at ADD COLUMN).
      A first CDC cycle whose table name is 21 to 30 characters long could also stop on a
      missing change log, because the "does not exist" answer was searched in a line snow
      wraps at 80 columns.
    • Now each part is read through a Parquet format rivet spells inline, with logical
      types on: a timestamp, a timestamptz and a TIME land with their microseconds, a
      binary column lands as its bytes, and a prefix with or without the trailing / loads.
      Whether a table or a change log exists is asked of the schema catalog, never read from
      an error text.
    • What to do: a Snowflake table loaded by 0.31.0 or earlier holds wrong values in
      every timestamp-without-zone column. A mode: full export replaces its table on the next
      rivet run and rivet load. An incremental or CDC table keeps the rows already in its
      <table>__changes log; decide per table whether to rebuild it from a whole pass.
    • Also changed: rivet no longer creates or reads the named file format rivet_pq. One
      left in the schema by an earlier release is unused and can be dropped; the loading role
      no longer needs CREATE FILE FORMAT.
    • rivet compact on a Snowflake target is unchanged: it exits 0 with skipped — this warehouse keeps a change log behind a view and never compacts.
  • Breaking: a second run of an export while one is alive is refused, whatever the mode.
    Every source; a SQLite or PostgreSQL state database. Reproduced on rivet 0.31.0 with
    PostgreSQL.

    • What was wrong in 0.31.0 and earlier. Only a run with chunk_checkpoint (or a MongoDB
      resume) held its export's run lease. Two runs of one export started together in any
      other mode both exited 0 and both wrote: the destination held every row twice, and both
      manifests said success. Measured with mode: full over 200 rows: 400 rows delivered.
      rivet repair --execute beside a live checkpointed run exported that run's pending chunks
      a second time. rivet state reset was accepted while an incremental run was reading, and
      that run's manifest then described its delta as a first pass.
    • Now every run takes the lease. One of the two runs exits 5 with
      RIVET_STATE_RUN_IN_PROGRESS and writes nothing; the other delivers (200 rows in the
      measurement above). It is the later one, except that a first run still creating its
      state database may reach the lease no earlier than a run started beside it. rivet repair --execute and rivet state reset get the same refusal beside a live run.
    • Also changed: the refusal a second run got beside a live checkpointed run was exit 1
      with no code; it is exit 5 with RIVET_STATE_RUN_IN_PROGRESS. The text still starts
      chunk checkpoint run ... still in progress in another live rivet process.
    • What to do if a run now refuses: wait for the live run of that export, or stop it.
      A scheduler that may start an export while its previous run is still going should treat
      exit 5 with this code as "skipped, a run is in progress". Two exports with different names
      never block each other; a --split unit and a partition_by child have their own names.
    • On a PostgreSQL state database every run now opens the lease keeper's connection (one
      more connection per rivet process), where only checkpointed runs did.
    • The lease is released by the operating system (SQLite state) or expires by its keeper
      (PostgreSQL state) when the process dies, kill -9 included; nothing has to be cleaned up.
    • New file beside a SQLite state database: every run leaves an empty
      .rivet_state.db.lease-chunk-run-<export> next to .rivet_state.db, one per export. Only
      checkpointed runs and loads left such files before. They hold no data and may be deleted
      while no rivet process runs; the next run creates its own again.
    • A state database or state directory that is read-only still refuses the run with
      RIVET_STATE_NOT_WRITABLE (exit 1), now also when it is the lease file that cannot be
      created.
  • Fixed: a mode: chunked config the planner refuses on a large table is refused on a small
    one too.
    PostgreSQL, MySQL, SQL Server and Oracle; exports with the table: shortcut.
    Reproduced on rivet 0.31.0 with PostgreSQL.

    • What was wrong in 0.31.0 and earlier. While the source's row estimate was at most
      chunk_size, the planner returned a single pass before it checked the shape. So a config
      with a DECIMAL chunk_column, or with no chunk_column over a table that has no key to
      page by, ran with exit 0 on a small table and was refused once the table outgrew one chunk.
      Measured on a 3-row table with chunk_column: amount (DECIMAL(10,2)), chunk_size: 4:
      0.31.0 exits 0 and writes one file.
    • Now the shape is checked first: the same run exits 1 with chunk_column 'amount' ... is not an integer-family column, twice in a row, and writes nothing. A keyless table gets
      chunked mode found no safe shape. On MySQL an auto-selected unique key under a uuid
      override is refused on a small table as on a large one
      (RIVET_CONFIG_KEYSET_KEY_UUID_OVERRIDE).
    • What to do if a run now refuses: follow the message: name an integer chunk_column,
      set chunk_by_key:, or use mode: full.
    • Unchanged: a config whose shape is valid still runs as one pass on a small table.
  • Fixed: a mode: chunked export whose table fits in one chunk is refused as what it is
    beside an unfinished checkpoint run.
    Applies to PostgreSQL, MySQL, SQL Server and Oracle.
    A mode: chunked export that sets none of chunk_checkpoint, chunk_count, chunk_by_days
    and chunk_by_key runs as one pass while the source's row estimate is at most chunk_size.

    • Before: beside an unfinished checkpoint run of that export, such a run was told this export now runs as full`` and restore the chunked` settings`, which the config still had; a table
      near `chunk_size` got that message or the checkpoint one as the estimate moved.
    • Now it is told what any run of its mode without the checkpoint is told: this export now runs without the checkpoint that run was opened with, with the remedies restore the checkpoint setting (chunk_checkpoint: true) and rivet state reset-chunks -c <config> --export <name>. Both were run on PostgreSQL, MySQL and SQL Server.
    • Unchanged: the code (RIVET_STATE_INTERRUPTED_RUN_OWNER_MISMATCH), exit 5, nothing read or
      written; what such an export runs as (one pass, mode full in rivet plan, the run
      summary and export_metrics); the message a mode: full export gets.
    • Also changed: beside an unfinished run of the other paging mode (a chunk_by_key run, then
      chunk_column with no checkpoint on a small table) the message says now runs as chunked``, where it said full.
    • rivet plan --format json writes "single_pass_of": "Chunked" (or "Keyset") in
      resolved_plan for such an export; the field is not part of the plan's checksum.
  • Fixed: rivet validate no longer passes a prefix whose manifest is gone while _SUCCESS
    is there.
    Every source and destination. Reproduced on rivet 0.31.0 with PostgreSQL and a
    local destination.

    • What was wrong in 0.31.0 and earlier. With manifest.json removed from a complete
      export, validate printed status: legacy_run (no manifest at this prefix — a pre-0.7.0 export ...) and exited 0, so rivet validate && deploy went on over parts nothing had
      verified. _SUCCESS is written only after the manifest, so a marker without a manifest
      is a prefix that lost its manifest, not an old export.
    • Now the same command exits 3 with RIVET_VERIFY_MANIFEST_MISSING (a new code), on
      every pass, and names the two ways out: restore manifest.json, or run the export again.
      A run of the export writes the prefix anew and validate passes after it.
    • If _SUCCESS itself cannot be read (an I/O error, not absence), validate reports
      RIVET_VERIFY_SUCCESS_READ_ERROR and exits 1, where it exited 0.
    • Unchanged: a prefix with neither a manifest nor _SUCCESS is still legacy_run, exit 0.
  • Fix: a FROM inside an SQL comment is no longer read as the table a query: reads.
    PostgreSQL and SQL Server exports with a hand-written query:; the preflight probe of
    rivet check on every SQL engine. Reproduced on rivet 0.31.0 with PostgreSQL and SQL Server.

    • What was wrong in 0.31.0 and earlier. To give a NUMERIC / DECIMAL column its
      declared precision and scale, rivet reads the table name out of the query text and asks
      the catalog. The reader did not skip comments, so a -- ... line or a /* ... */ block
      holding the word from before the query's own FROM named the table. Measured on a
      table with a numeric(10,2) column:
      • -- rows from the last week (no table of that name): the export failed, exit 1,
        1 column(s) have no safe Rivet mapping, no file written. The same query without
        the comment exported decimal(10,2).
      • -- taken from legacy, where legacy has that column as numeric(20,6): exit 0, and
        the column was written as decimal(20,6), the other table's type. The values were
        right.
      • the same with legacy holding numeric(6,0): the export failed, exit 1,
        cannot parse DECIMAL "12345.67" as decimal(scale=0).
      • A comment AFTER the table name (FROM t -- from b) made the reader give up, with the
        first outcome.
      • SQL Server, the same table as decimal(10,2): every one of these exported, exit 0,
        with another type for the column. With no table of the commented name, or with the
        comment after the table name, the column was written as decimal(38,2); with
        legacy holding decimal(20,6) it was written as decimal(20,6).
        No wrong value was delivered in any of these; the column's declared type was.
    • Now one reader finds the outermost FROM for every caller, and a comment counts as
      whitespace wherever it stands. Every query above exports decimal(10,2) on both engines.
    • Also read now, where the reader gave up before: a query ending in ;, a table name
      with a non-ASCII letter (FROM café), and on SQL Server a double-quoted name
      (FROM "s"."t"). The preflight probe no longer takes from x inside a string literal
      for the table, and reads s . t as s.t.
    • Unchanged: a join, a comma list, a subquery or a function call in FROM still
      gives no table, and the column needs a columns: override as before. A MySQL #
      comment is not skipped: # is an operator in PostgreSQL, and the reader is told no
      engine.
    • Stored progress (unreleased builds only): the stream of a query's stored progress is
      read by the same reader, so a query with such a comment now names its real table there.
      No released version stores a stream.
  • Fix: an Oracle export whose table loses a column while it is being read stops with an
    error instead of a panic.
    Oracle only; every batch mode. Seen once in CI on main
    (07e3e9b7..97d3f57b) as exit 101 of a range-chunked run; not reported from the field.

    • What was wrong in 0.30.0 and 0.31.0. Each Oracle read describes its query's columns,
      then runs the query. When the query needs no column rewritten (no LOB, no type read
      through a conversion) it is sent as written, so a column dropped between the two
      statements came back as a narrower result, and the reader indexed that result by the
      wider description: index out of bounds, exit 101 (an abort in a release build). The run
      ended without recording its failure: its run_status and export_metrics rows stayed
      running.
    • Now that read stops with RIVET_SOURCE_COLUMNS_CHANGED_UNDER_READ (kind:
      environment), the run is recorded as failed, and the next run reads the table as it is.
    • Proof and its limit. A unit test pins the column layout on a result narrower than
      its probe; it panicked before the change. The window between the two statements is a
      few milliseconds, so no live cell hits it on demand: the existing cell
      a_column_dropped_under_a_live_run_delivers_or_fails_loudly_oracle reaches it by chance.
    • Unchanged: a query whose columns are rewritten names each column, and Oracle itself
      refuses a dropped one (ORA-00904). A column ADDED between the two statements is still
      delivered as an extra column of that read.
  • Fix: masking a URL password in a log line or an error text takes one pass over the
    text.
    Applies to every command and every engine: each emitted log line and each error
    text goes through this scan.

    • What was wrong in 0.31.0 and earlier. The scan for scheme://user:password@ started
      a new read at every byte: from each letter to the end of the run of letters, digits and
      + . - it stands in, and from each :// to the next whitespace. The time grew with the
      square of the longest such run. Measured on that function compiled alone with
      optimisations: a run of 16 KiB took 0.1 s, 64 KiB 1.1 s, 256 KiB 18.6 s. The text
      came out correct; only the time was wrong. Whether a real driver message holds a run
      that long (a hex or base64 value quoted in an error) was not observed in the field: the
      defect was found by reading the code under mutants that hung its tests.
    • Now the same 256 KiB take 0.7 ms and 16 MiB take 34 ms. What is masked is
      unchanged: the new scan and the previous one give byte-identical output on 400,000
      generated texts of URL punctuation, whitespace and multi-byte characters, and that
      comparison stays in the test suite.
    • Also fixed: a secret query parameter with a name longer than 64 characters was not
      masked
      (?<65+ characters>_token=..., since 0.25.0). The length limit is removed;
      a name of any length ending in a secret word is masked.
    • Removed, no behaviour change: two config checks that no config could reach, because
      an earlier check refuses the same input first — the .. check of a local destination
      path/prefix (every destination's .. segment is refused before it), and "cdc mode
      reads the transaction log, not a query" (a query: beside table: is refused as two
      sources, and a query: alone as a missing table:). The messages an operator sees are
      the ones they saw before.
  • Fix: a PostgreSQL run through a transaction-mode pooler no longer leaves session settings
    on the pooled server connection.
    PostgreSQL batch connections and mode: cdc. Reproduced
    on rivet 0.31.0 behind pgBouncer 1.26.0 in transaction mode, with pool sizes 1 and 5.

    • What was wrong in 0.30.0 and 0.31.0. Each connection rivet opened for run, check,
      plan, doctor, init, a PostgreSQL state database and rivet-mcp sent
      SET TimeZone = 'UTC'; SET DateStyle = 'ISO, MDY'; SET IntervalStyle = 'postgres'; SET bytea_output = 'hex' as session settings, unless it detected a transaction pooler.
      The detection compares the backend pid of two statements, and an idle pooler answers both
      from one server connection, so the settings were sent and stayed on the connection the
      pooler then gave to its other clients.
      • Who was affected: other clients of the same pooler, not rivet. pgBouncer puts back the
        parameters it tracks for each client (TimeZone, DateStyle and IntervalStyle in
        1.26), so the setting another client read differently is bytea_output. Measured on a
        database whose default is escape: after one rivet run, another client's
        bytea::text came back in hex. A pooler that tracks fewer parameters hands on more
        of the four.
      • mode: cdc had no pooler detection at all: every connection its test_decoding reader
        opened sent the same four and SET extra_float_digits = 3, a refused run included (the
        reader has set session formats since 0.19.1, by git log; measured on 0.31.0). Through
        pgBouncer on a database whose defaults are escape and extra_float_digits = 0, one CDC
        run left bytea_output = hex and extra_float_digits = 3 on the pooled connection for
        the pooler's other clients.
      • rivet's own output was correct.
    • Now a batch connection sends no session-level SET. The export, the bound and count
      probes of a run, the range and plan probes of check and plan, and the table
      statistics of rivet-mcp set the four formats with SET LOCAL inside the transaction
      that reads, on a direct connection and behind a pooler alike, and nothing outlives the
      transaction. On a direct connection the delivered files and the check output are the
      same as in 0.31.0 (compared under a role whose defaults are Asia/Tokyo, German, DMY,
      sql_standard and escape).
      • Behind a pooler busy enough for 0.31.0 to detect it, the probes and check ran in the
        role's zone and formats while the export ran in UTC. They now read in UTC as well.
      • mode: cdc sends no session-level SET either. Each read of the slot runs in a
        transaction of its own that sets the four formats and extra_float_digits = 3 with
        SET LOCAL (two round trips more per read of the slot, which holds up to one part's
        changes). On a direct connection the delivered changes are the same as in 0.31.0
        (one window of inserts, updates and deletes over float8, float4, timestamptz,
        timestamp, date, bytea, interval and numeric, read under a role whose defaults
        are Asia/Tokyo, German, DMY, sql_standard, escape and extra_float_digits = 0).
        • Checked by hand through pgBouncer 1.26.0 in transaction mode with one server
          connection, in front of a wal_level = logical server: a refused run, a run that
          read changes and a run that found none each left the five settings and their
          pg_settings.source as they were, on the same backend. The test suite covers a
          refused capture through the pooler and, offline, the statements the reader sends;
          it has no pooler in front of a logical server.
      • A setting an earlier release left behind stays on the pooled server connection until
        the pooler closes it (pgBouncer: server_lifetime, one hour by default) or is restarted.
  • Fix: a query: holding a non-ASCII character no longer ends the run in a panic.
    Applies to every source engine. Reproduced on rivet 0.31.0 with PostgreSQL, MySQL,
    SQL Server and MongoDB.

    • What was wrong in 0.31.0 and earlier. Two readers of the query text cut it at a
      byte offset, and a cut inside a multi-byte character stopped the process: exit 101,
      byte index N is not a char boundary, on every run.
      • Every export, in every mode: a multi-byte character across byte 6 of the query (a
        comment or a CTE name that opens it: /* <non-ASCII comment> */ SELECT ...,
        WITH é AS (...) SELECT ...), or across byte 4 of what follows SELECT *
        (SELECT *, 'été' AS label FROM t). On MongoDB the same text crashed in place of the
        refusal a hand-written query: gets there.
      • Range-chunked exports (chunk_column), and the value-partition and split planners
        that probe a query's bounds the same way: any non-ASCII character before the
        query's first FROM (SELECT id, 'café' AS label FROM t).
      • A non-ASCII character anywhere else (a WHERE literal, a table name after FROM)
        was read correctly.
      • Nothing was written to the destination: no part, no manifest, no _SUCCESS, no run
        summary. The state database kept one run_status row in running for each crashed
        run and the export's schema baseline.
      • The cut at the query's start dates from 0.6.0 and every export's query has gone
        through it since 0.18.0; the scan for FROM dates from 0.16.0.
    • Now both readers compare bytes and never cut the text inside a character. Such a
      query runs and delivers what its ASCII spelling delivers; on MongoDB it is refused by
      name like any query:. An export that crashed needs nothing but the upgrade: run after
      two crashes of 0.31.0 on the same state, this build exited 0 and delivered every row.
      The running rows the crashes left stay in run_status, behind the newer run.
    • Unchanged: what is recognised in an ASCII query. The SELECT * FROM t and plain
      column-list shortcuts, the position of the outer FROM and every SQL text rivet builds
      from a query are byte-identical for 575,270 query texts that did not crash before
      (568,212 of them ASCII), compared between the previous code and this one. A query with a
      non-ASCII identifier (SELECT id, café FROM t) is not taken for the bare-table
      shortcut: its bounds are read from the query, wrapped as FROM (<query>) AS _rivet.
  • rivet load --pool N keeps at most 16 storage requests in flight for the whole
    process.
    Every table loaded at once used to send its storage requests (list, read,
    delete) with no shared limit, each request in flight on a connection of its own: a load
    of 16 tables with --pool 16 held 46 connections to Cloud Storage and 145 open files,
    measured on macOS with a release build. It now holds 16 and 117, in the same time
    (15.0 s before, 14.5 s after, one run each). The budget counts requests, not
    connections: a connection can outlive its request for a moment, and a debug build
    sampled every 50 ms held up to 22. A pool larger than the budget waits for a free
    request instead of opening more connections. The export side (rivet run writing to a
    bucket) is unchanged. No config key: the budget is a constant.

  • Fix (behaviour change): on a PostgreSQL state, rivet load and rivet compact stop
    instead of writing a table under a lease nothing renews.
    Applies only with
    RIVET_STATE_URL=postgresql://...; a SQLite state takes its lease as a file lock and is
    unchanged. No state table, config key or TTL changed, and 0.31.0 and this release
    exclude each other on the same lease rows.

    • What could happen in 0.31.0 and earlier. Every lease (one per table being loaded
      or compacted, one more per table with cleanup_source, one per checkpointed export
      run) was renewed by a thread of its own on a connection of its own, opened after the
      lease was granted. When that connection could not be opened the load went on, printed
      LOAD OK and exited 0; the lease lapsed after RIVET_STATE_LEASE_TTL_S (30 s by
      default), and a second rivet load or rivet compact of the same table could then
      run beside the first. It took three conditions together: a PostgreSQL state; the
      renewing connection failing to open (the process short of file descriptors, where the
      PostgreSQL client crashed its thread, or the state server out of connections for the
      role); and a load of that table still running after the TTL while another one started.
      A lease another process had taken after a stall was logged and not acted on.
    • Now one connection and one thread per process renew every lease the process
      holds, and they are started before the first lease is granted.
      • If that connection cannot be opened, the table is not loaded: the command fails with
        RIVET_STATE_LEASE_KEEPER_UNAVAILABLE, having written nothing to the warehouse and
        nothing to the load ledger, on every run until a connection is free. Free a
        connection on the state database, raise the open-file limit, or lower --pool.
      • A load or compact whose lease is no longer its own (not renewed within the TTL, or
        taken by another process) stops before the warehouse write with exit 5
        RIVET_STATE_LEASE_LOST, recorded refused in the load ledger; the next
        rivet load loads the table. When the loss is found only after the write, the
        command fails with exit 3 RIVET_LOAD_LEASE_LOST_DURING_WRITE and the ledger row
        is failed, not success: another load or compact may have written the table at
        the same time, and the next rivet load loads the same runs again.
      • A checkpointed rivet run takes its run lease through the same connection and is
        refused with RIVET_STATE_LEASE_KEEPER_UNAVAILABLE when it cannot be opened. A
        PostgreSQL state that cannot be reached for want of file descriptors is an Error:
        line and exit 1, where 0.31.0 crashed with exit 101.
    • Connections. A pooled load or compact holds one state connection per worker plus
      one, so --pool 16 is at most 17 connections to the PostgreSQL state where 0.31.0
      held 32 (48 with cleanup_source). Count N + 1 against the role's and the server's
      connection limits.
  • Fix: rivet load into BigQuery opens fewer files per table loaded at once. Each
    table of a --pool N load built its own HTTP client for the BigQuery API (a thread, an
    event queue and its connections apiece); one client now serves every table of the
    process. Measured on macOS, 16 tables from GCS at --pool 16: the peak of open files
    goes from 204 to 139 (event queues 51 to 6), the load takes the same 14 s, and
    --pool 1 and --pool 4 are unchanged at 18 and 30. The rest of the peak is one state
    connection and the storage connections of each table, which this change does not touch.
    No config, state or file format change.

  • Fix: a PostgreSQL export through a transaction-mode pgBouncer that keeps prepared
    statements is delivered under its own columns.
    Applies to every batch mode (full,
    chunked by key or by range, incremental) when the source URL points at pgBouncer in
    transaction pooling mode with max_prepared_statements above zero (pgBouncer's
    documented default is 200). Reproduced on pgBouncer 1.26.0 with rivet 0.31.0; a direct
    connection was never affected, and neither was MySQL through ProxySQL.

    • What was wrong in 0.31.0 and earlier. rivet read every table through a statement
      of one text, FETCH n FROM _rivet. Such a pooler keeps a prepared statement by its text
      and reuses it for every client, and PostgreSQL keeps the columns it described when that
      text was first parsed. A table read after another one through the same pooled server
      connection was decoded as the earlier table:
      • with the same number of columns the run exited 0 with a success manifest, and the
        part carried the earlier table's column names and types (a double precision 1.5
        delivered as the BIGINT 4609434218613702656);
      • one table whose column type was altered between two runs was delivered under the old
        type, exit 0;
      • with another number of columns, or after ALTER TABLE .. ADD COLUMN, the run failed
        with DataRow field count does not match the number of columns, on every run until
        the pooler dropped its server connection;
      • after a chunk key was widened the run failed with cached plan must not change result type or sent a malformed wire payload, and a chunked run skipped its schema-drift
        check with could not resolve schema for drift check (skipping).
    • Now every statement whose columns follow a table (the cursor FETCH, the bound and
      count probes, the type probe) is prepared under a text of its own, so the pooler never
      answers it with a statement another read left. The first run of this release through a
      pooler that still holds statements of an older rivet delivers the table as the source
      holds it.
    • If you ran 0.31.0 or earlier through such a pooler: compare each delivered part's
      column names and types with the source table (DESCRIBE SELECT * FROM read_parquet('<part>') in DuckDB against \d <table> in psql). A part whose columns
      are the table's own was decoded as that table. A part with another table's columns, or
      a column under a type the table does not have, holds wrong values: remove it and
      extract the export again with this release (rivet state reset -c <config> --export <name> first for an incremental or checkpointed export, so the pass is a full one).
  • Breaking: a destination or format edited under stored progress is refused, and a run
    resumed after a crash delivers the rows that arrived since.
    Applies to incremental,
    keyset_incremental, MongoDB resume and chunk_checkpoint exports on every source
    engine.

    • A moved destination or another format under a stored cursor is refused. Before, an
      export that continues from a cursor kept it when its destination (bucket, path,
      prefix or store) or its format was edited with no reset: exit 0, and the new
      location held only the rows past the cursor, or one prefix held parts of two formats.
      Now the run is refused with exit 5 RIVET_STATE_CURSOR_DESTINATION_MISMATCH, naming
      where the cursor was delivered and where the config points, on every run. Restore the
      destination and format to continue, or run
      rivet state reset -c <config> --export <name> for a full pass delivered where the
      config points now. The parts already delivered are not moved or removed: empty that
      location first if the full pass lands in it too. rivet state accept does not lift
      this refusal. The destination is compared as written, so a {date}, {export},
      {table} or {run_id} placeholder is one destination on every day, and a rivet plan
      artifact sealed before this release still applies. A destination that comes from
      ${VAR} is compared after substitution: a value that moves the path between runs is
      refused, so spell a per-run prefix with {date}.
    • An unfinished checkpointed run is not finished in another format. Before, a
      chunk_checkpoint run interrupted while writing parquet and continued after
      format: csv exited 0 over one success manifest naming parts of both formats. Now the
      run, and rivet run --resume, is refused with exit 5
      RIVET_STATE_INTERRUPTED_RUN_OWNER_MISMATCH, naming both formats. Restore the format
      and run once to finish it, or abandon it (rivet state reset-chunks for a range-chunk
      run, rivet state reset for a keyset run) and the next run starts a full pass in the
      new format. This holds for a run 0.31 or earlier left unfinished too.
    • A resumed chunk-checkpoint run reads the source's key span again. Before, a plain
      run or --resume that continued a crashed chunk_column + chunk_checkpoint run
      finished the plan the crashed run had stored: rows that reached the source past it
      were in no part, under exit 0 and _SUCCESS, and an empty table that filled after the
      crash was delivered as 0 rows. Now the resumed run detects the key span again, adds
      tasks for the keys outside the stored plan, and plans a run that committed no part
      again whole. The windows the crashed run committed are not read again. The units of
      rivet apply --pool --split --resume resume the same way, so rows gained past the top
      key land in the last unit (before, the resumed split delivered the keys of the crashed
      plan only). rivet apply of a sealed plan artifact replays the ranges its plan
      sealed, as before. A resume now runs the min/max and NULL-key probes a fresh run
      does and fails where a fresh run would (a NULL key, a key span past the sparse-plan
      limit).
    • A resumed parallel keyset run reads past its last range. Before, a chunk_by_key +
      parallel + chunk_checkpoint run whose last key range had committed before the crash
      read no key above it on resume. Now it reads the keys past the highest one that range
      delivered. A range committed by 0.31 or earlier recorded no such key: it is read again
      whole, and none of its parts is declared twice.
    • A chunk run whose task rows are gone is planned again. Before, a resume over a
      chunk_run with no chunk_task rows wrote a success manifest with 0 rows.
    • Upgrading: nothing to migrate, and the state schema version does not change. A cursor
      row written by 0.31 or earlier holds no destination: the first run of this release
      records where the export delivers, logs one predates destination tracking warning
      and continues, so an edit of the destination or format made in the same step as the
      upgrade is not seen; make the upgrade and the edit two separate runs. A run that 0.31
      left unfinished is resumed with the behaviour above.
  • Behaviour change: a table that is empty at its CDC baseline gets an empty warehouse
    table.
    Applies to a mode: cdc export with cdc.backfill: loaded under
    load.layout: base_buffer (BigQuery), every source engine.

    • Before, the load consumed the empty baseline as "nothing to load" and created no
      <table>. rivet compact passed the table by until its first change was buffered, and
      from then on refused it on every cycle (the base table does not exist; 0.30 and older
      passed BigQuery's Not found: Table through), so the command exited non-zero although
      every other table was merged. The buffered changes stayed in <table>__changes.
    • Now the load of an empty baseline creates <table> empty, with the spec's columns,
      __is_deleted, partition and clustering, and the first change merges into it. A new
      baseline that holds no row empties an existing base the same way, in every load; before,
      it did so only when the same load also carried change files.
    • Upgrading: nothing to do. The first rivet load of this version creates the missing
      table and prints note: ... does not exist although its baseline was loaded, and the next
      rivet compact merges every change buffered since the baseline. It does so only when the
      state DB beside the config recorded exactly one baseline run for the table, with 0 rows,
      and no compaction ever merged a row into it. A table whose base was dropped by hand after
      it held rows is still refused by rivet compact, as before, and so is one whose state DB
      does not hold the extract's run records (a load that runs on another host).
    • rivet compact over several tables exits 1 when at least one table failed, names each
      failed table, and still merges and reports (COMPACT OK) the others.
  • Breaking (file names): a single-runner part is named after its run, with the process id
    and a random nonce appended.
    Applies to mode: full, mode: incremental and time-window
    exports that are not chunked, on every source engine, and to MongoDB parallel exports.

    • Before: orders_20261008_174122_776.parquet (the UTC millisecond the read ended). Two
      rivet run of one export that ended their read in the same millisecond wrote the same
      name; the second file replaced the first, both runs exited 0, and both manifests,
      file_log and export_metrics counted the one file (two runs of a 200-row table:
      400 rows recorded over one 200-row file).
    • Now: orders_20261008_174117_653_33972_9f3a1c0b5d7e2a41.parquet, that is
      <export>_<YYYYMMDD>_<HHMMSS>_<mmm>_<pid>_<nonce>[_partN].<format>: the UTC millisecond
      the run started and its process id (both from the run's id), then 16 random hex digits,
      so no two runs write one name, two containers with the same process id included. A
      MongoDB parallel part is
      <export>_<YYYYMMDD>_<HHMMSS>_<mmm>_<pid>_<nonce>_w<worker>_keyset<page>.<format>.
    • What to change: a consumer that matches part names with a pattern anchored after the
      millisecond field (orders_\d{8}_\d{6}_\d{3}\.parquet) must allow the _<pid>_<nonce>
      fields. orders_*.parquet and a date prefix (orders_20261008_*) match as before, and
      names still sort by time across the upgrade. Take part names from manifest.json rather
      than from a pattern.
    • Nothing else moves: no state or manifest format change. Parts written by earlier
      releases keep their names and are read, validated and loaded as before, and a prefix may
      hold both shapes. Chunked, keyset and CDC part names are unchanged.
  • Breaking: stored progress belongs to one source, one table and one set of rows, and a run
    that reads another is refused.
    Applies to incremental, keyset (chunk_by_key) and
    MongoDB resume exports on every source engine, and to rivet state reset.

    • An edited query: filter is refused. Before, an export whose query: kept its FROM
      table but changed its filter (a WHERE, a join, a CTE) continued from the old filter's
      cursor: exit 0 and none of the rows the new filter admits below it. Now the run is refused
      with exit 5 RIVET_STATE_CURSOR_STREAM_MISMATCH, naming both queries, on every run.
      Restore the previous query to continue, or run
      rivet state reset -c <config> --export <name> to deliver the new filter's rows in full,
      or run rivet state accept -c <config> --export <name> to keep the cursor under the new
      query (see below). A changed column list, letter case or layout is not an edit and
      continues; any other change of the text after FROM <table> is one, a renamed alias or
      reordered predicates included.
    • A ${param} value that changes between runs is not an edit. The query is compared
      as written, before --param and environment values are substituted, so
      WHERE region = '${region}' run with another region continues from the stored cursor
      with no refusal and no warning, as it did before this release. An edit of the text
      around the placeholder is refused like any other. A rivet plan artifact sealed before
      this release still applies, before or after a run of this release: it carries the
      substituted query only, and a substituted query that fills the placeholders of the
      stored one is the same rows.
    • New command rivet state accept --config <PATH> --export <NAME> [--param KEY=VALUE].
      It answers the refusal above with "the edit is meant, go on from where the export was":
      the stored progress of that one export is recorded as belonging to what the config reads
      now, the cursor value is not touched, and the next run continues from it (rows of the
      new query below the cursor are not delivered; state reset delivers them). It is given
      per export and per edit; no setting accepts later edits. A cursor written for another
      cursor_column is not accepted (RIVET_STATE_CURSOR_OWNER_MISMATCH). Each acceptance
      is listed by rivet journal with status accepted and a line naming both queries.
    • The same table under another PostgreSQL search_path is refused. Before, a source
      URL that set options=-c search_path=<schema> moved an unqualified table to another
      schema under the cursor of the first: exit 0, zero rows. Now it is the same refusal with
      the same three remedies. Qualifying the table by hand with the schema it was read through
      continues.
    • rivet state reset clears one source. Before, state reset --export X deleted the
      cursor of every export named X in the state database, so a same-named export of another
      source re-delivered its whole table on its next run. Now it clears the progress X holds
      on the source its config names, and the config's source URL must resolve (a missing
      url_env variable stops the reset with nothing removed).
    • A source URL without its default port is the same source. Before, dropping or adding
      :5432 / :3306 / :1433 / :1521 / :27017 in the URL started a whole-table pass
      under a second state row. Now both spellings are one source and the run continues.
    • A resumed parallel keyset run stores the run's highest key. Before, a
      chunk_by_key + parallel + chunk_checkpoint run resumed after a crash stored the
      highest key of the ranges it re-ran (or none), so a later incremental run on the key
      delivered rows again. Now each committed range records its highest key and the resumed
      run stores the highest of all.
    • Upgrading: nothing to do. State written by 0.31 or earlier is read as follows. A cursor
      row stored under a URL without its default port is moved onto the spelled source on the
      first run or state reset of that export and continues (if both spellings hold a row,
      the one under the spelled port is kept). A cursor row holds no query and no
      search_path yet: the first run records the ones the export has now, logs one
      predates stream tracking or predates query tracking warning, and continues; edits
      are compared from the second run on, so make the upgrade and a filter edit two separate
      runs. A parallel keyset run interrupted before the upgrade and resumed after it stores
      the highest key of the ranges that commit after the upgrade, as 0.31 did. The state
      schema moves to v35 (keyset_range.max_key), so an older rivet no longer opens the
      state database.
  • Breaking: a CDC stream that cannot continue exits 5 with a RIVET_* code, where it exited 1
    with none.
    Applies to mode: cdc runs and rivet cdc on PostgreSQL, MySQL, SQL Server,
    MongoDB and Oracle.

    • RIVET_SOURCE_CDC_CHECKPOINT_INVALID: a checkpoint file that is not JSON (every engine),
      and a checkpoint file that is gone while the export has a baseline (MySQL, SQL Server,
      MongoDB, Oracle).
    • RIVET_SOURCE_CDC_TRUNCATED: a TRUNCATE of a captured table (PostgreSQL, MySQL; Oracle
      gave it already), a DROP of one (MySQL), a dropped captured collection or database
      (MongoDB).
    • RIVET_SOURCE_CDC_LOG_GAP: a PostgreSQL slot missing under a checkpoint or a baseline,
      MySQL binlogs purged past the checkpoint (ERROR 1236), a MongoDB oplog rolled past the
      resume token (error 286). The MongoDB refusal used to end with "change streams require a
      replica set", which was not the cause; it now names the oplog and the re-baseline.
    • The corrupt-checkpoint message said "delete it to accept a new anchor from a fresh
      snapshot". That was wrong on every engine but PostgreSQL: with a baseline the next run
      refused the missing file, and without one it anchored at the current position and skipped
      every change since the checkpoint. The message now ends "Restore the file, or:" and the
      re-baseline steps, the same ones every other CDC data-loss message prints.
    • What is refused has not changed, and nothing stored changes: a checkpoint and a state
      database written by 0.31.0 resume as before.
    • Upgrading: a script or scheduler that matched exit 1 for these cases now sees exit 5
      (refusal: do not retry, follow the remedy in the message). rivet schema errors lists
      the three codes.
  • Breaking: a run that fails before its first write leaves the destination as it was.
    Applies to every batch export (full, incremental, chunked, keyset, the snapshot leg
    of a CDC export) on every source engine and on every destination that keeps objects at
    rest (local, S3, GCS, Azure).

    • Before, a run that could not reach the source (exit 2), was given a wrong password, or
      failed in any other way before it wrote a part, still rewrote manifest.json to
      status: failed with 0 rows, added a manifest-<run id>.json copy and deleted
      _SUCCESS. After a network failure rivet validate exited 1 and a loader or sensor
      keyed on _SUCCESS stopped seeing an export that was whole.
    • Now _SUCCESS and a success manifest mean "this prefix holds a whole export", not
      "the last run succeeded". Over a prefix that already holds a manifest such a run writes
      nothing to the destination; the failure is told by the exit code and the run journal
      (rivet metrics, export_metrics). On a prefix with no manifest yet (the export's
      first run) it records its failure as before: a failed manifest with 0 parts and no
      _SUCCESS, so rivet validate exits 1 on a prefix whose only run failed. A run that
      put a part at the destination and then failed still deletes _SUCCESS and writes a
      failed manifest naming those parts.
    • What a scheduler or loader that reads the destination sees differently: after a run
      that failed before its first write, _SUCCESS is still there, manifest.json still
      says success with the previous run's run_id and finished_at, and no
      manifest-<run id>.json of the failed run is added. A consumer that used "the marker is
      gone" or "the manifest says failed" as its signal that the last run failed must read
      the run's exit code, or compare the manifest's run_id with the run it started.
    • --resume reads the run journal beside the marker, so its behaviour after such a
      failure is what it was: rivet apply <config> --resume runs again an export whose last
      journaled run failed, even though an earlier run's _SUCCESS is still at its prefix, and
      rivet run --resume is not refused over that marker. Both read export_metrics in the
      state database the config names; with a state database that has no row for the export
      the marker alone decides, as before.
    • Upgrading: nothing to migrate. A prefix that 0.31 or earlier already re-marked as
      failed stays so until the next successful run writes its manifest and marker.
  • Breaking: --resume over a complete prefix is refused by code, and --resume --force
    runs on every strategy.
    Applies to rivet run --resume on every source engine.

    • Before, --resume into a prefix that holds _SUCCESS exited 1 with no code and said
      "Re-running would overwrite a verified dataset. Pass --force to override". On a
      chunk_checkpoint: true export --resume --force then exited 1 with "nothing to
      continue" and re-marked the complete prefix as failed; on the other strategies it
      exported again. Nothing was ever overwritten.
    • Now the refusal is RIVET_DEST_ALREADY_COMPLETE (exit 5, was 1) and says what
      --force does: it continues an interrupted run of the export if one is recorded, else
      the run goes on as a plain run does (a full export is exported again, a delta export
      continues past its cursor); new parts land beside the old ones and manifest.json then
      describes only the new run. --resume --force does that on a
      checkpointed chunked export too. --resume without --force and with no run to
      continue is refused as before ("nothing to continue; run without --resume").
    • A script that matched exit 1 or the old text of this refusal must match exit 5 or the
      code.
  • Behaviour change: a CDC run that refuses a prerequisite leaves nothing behind, and
    rivet doctor agrees with rivet run.
    Applies to mode: cdc and rivet cdc on
    PostgreSQL and MySQL.

    • Before, on PostgreSQL a run refused for a table that does not exist had already created
      its replication slot (inactive, pinning WAL); a bounded run on a standby was refused only
      after the slot, the baseline and snapshot/_SUCCESS were written. On MySQL a first run
      refused for binlog compression, a replica that does not re-log, or an unroutable table had
      already written its checkpoint. rivet doctor reported All checks passed for a
      PostgreSQL server with wal_level below logical and for a MySQL replica with
      log_replica_updates = OFF.
    • Now every prerequisite is asked before the first durable write, and these refusals carry
      RIVET_SOURCE_CDC_PREREQUISITE (exit 1, as before): no slot, checkpoint, baseline or state
      row is left. A PostgreSQL table that does not exist and wal_level below logical are
      named by rivet instead of by the server's error. Doctor asks the same function the run
      refuses by and fails a CDC run prerequisites check with the run's own message.
    • rivet init --mode cdc against a PostgreSQL standby writes until_current: false, the
      mode a standby runs; it wrote until_current: true, which the run refuses there.
    • Upgrading: nothing to do for a healthy stream. A slot or checkpoint left by a run 0.31
      refused is reused as it is once the prerequisite is met; if the export was abandoned
      instead, drop the slot (SELECT pg_drop_replication_slot('<slot>')) or delete the
      checkpoint file.
  • Behaviour change: rivet cdc to stdout emits each change once on PostgreSQL. Before,
    a drain to stdout never advanced the slot, so every run printed every change since the slot
    was created, with or without --checkpoint, and the slot pinned WAL without bound. Now a
    drain acknowledges the last commit it printed when it ends. Upgrading: the first drain with
    this version prints the backlog the slot still holds once more, then the slot advances. A
    consumer that relied on each run re-printing the history must keep its own copy.

  • Fix: PostgreSQL CDC with rollover: 1 (--rollover 1) delivers the pending changes.
    A bounded run leaves its drain barrier (a logical message) in the WAL; at rollover: 1 the
    next run's one-change read window held only that message, and the run exited 0 with a
    Success manifest of 0 rows while commits were pending, on every run. With rollover: N the
    same happened after N idle bounded runs in a row. Messages outside a transaction are now
    stepped over, up to the run's open-time bound. Upgrading: the first run with this version
    delivers what such a slot was holding.

  • Breaking: SQL Server CDC refuses a log gap from the first run after the baseline.
    Applies to a mode: cdc SQL Server export with cdc.initial: snapshot.

    • Before, the checkpoint the baseline wrote ("pinned": true) could sit below the capture
      instance's start, and a run moved such a position up to the start. If the change table lost
      rows before the first run that read a change (retention cleanup past them, or the capture
      instance disabled and re-enabled, which is also the remedy the ADD COLUMN refusal names),
      the next runs ended with exit 0 and those changes were never delivered.
    • Now the baseline's anchor is always inside the capture instance's lifetime, and any resume
      position below the instance's start_lsn is refused: exit 5 RIVET_SOURCE_CDC_LOG_GAP
      with the re-baseline remedy, on every run until the stream is re-baselined. An idle run now
      moves a baseline-only checkpoint forward like any other, so a quiet table is not refused
      when the cleanup job passes its anchor.
    • Upgrading: a checkpoint that already read a change needs nothing. A baseline-only
      checkpoint written by 0.31 or earlier that is below the instance's start is refused on
      the first run of the new version, with the same exit 5 RIVET_SOURCE_CDC_LOG_GAP: the
      file cannot tell a harmless anchor from a gap that already happened, so rivet does not
      guess. This affects a table that had no captured change since its baseline while the
      cleanup job ran at least once, or whose capture instance was enabled just before the
      baseline. Re-baseline such a stream once, as the refusal says.
  • Breaking: rivet run --validate exits non-zero when its validation fails. Applies to
    every source engine and every destination that keeps objects at rest.

    • Before, a run whose end-of-run verification failed printed validated: FAIL and exited 0,
      while rivet validate on the same prefix exited 3. A script or scheduler keyed on the
      run's exit code learned nothing.
    • Now the run exits as rivet validate does on that prefix: RIVET_VALIDATE_FAILED
      (exit 3) when a check found the destination wrong (a missing part, a size or checksum
      mismatch, verify: content not met), RIVET_VALIDATE_UNVERIFIED (exit 1) when the
      destination could not be read back. The error line names each failure. The export itself
      is not undone: its parts, manifest, _SUCCESS, metrics row and cursor are as before.
    • Upgrading: a job that ran rivet run --validate && next now stops before next when
      the verdict is FAIL. The case to look for is verify: content on a destination that
      exposes no content checksum, such as a local path: those runs print validated: FAIL
      today and exit 0. Use verify: size there, or drop --validate.
  • Breaking: a run over a state database it cannot write is refused, and a cursor that could
    not be stored fails the run.
    Applies to rivet run and rivet apply, every batch mode
    and source engine.

    • Before, with a read-only .rivet_state.db (file permissions, a read-only volume, a
      read-only role) a run logged warnings, exported, and exited 0. An incremental run could
      not advance its cursor, so every later run delivered the same rows again; no run left a
      metrics row.
    • Now the run is refused before it exports anything (RIVET_STATE_NOT_WRITABLE, exit 1),
      on every run, until the state takes the write. Make the state database writable and run
      again: the run then delivers from the stored cursor.
    • A read-only DIRECTORY around a writable state file (a read-only volume mount) gets the
      same refusal, from every command that opens the state. Before, it failed with exit 2 and
      could not acquire the migration lock ... Another rivet process is migrating this state database; wait for it to finish and retry, although no other process existed.
    • If the state fails only at the end, when the rows and the manifest are already written,
      the run exits non-zero with RIVET_STATE_CURSOR_NOT_STORED instead of logging an error
      and exiting 0. The rows are delivered and the next run delivers them again from the
      previous cursor, as before; the exit code now says so.
    • Not changed: mode: cdc runs, a chunk_checkpoint run (it already failed, with
      Permission denied), and commands that only read the state (state show, metrics).
    • Upgrading: nothing to do for a writable state. A deployment that ran on a read-only
      state and relied on exit 0 now sees exit 1.
  • Breaking: a RIVET_STATE_URL that is not a PostgreSQL URL is refused. Applies to every
    command that opens the state (run, plan, apply, load, state, metrics, ...).

    • Before, any value that did not start with postgres (a typo such as postgre://,
      another engine's URL) was ignored without a message: the command kept its state in a
      SQLite file beside the config. In a container that file is lost with the container, so
      every run started from no cursor.
    • Now it is refused with RIVET_STATE_URL_SCHEME_UNSUPPORTED (exit 1); no SQLite file is
      created. Set RIVET_STATE_URL to a postgres:// or postgresql:// URL, or unset it to
      keep the state in .rivet_state.db beside the config. An empty value still selects
      SQLite, as an unset one does.
    • Upgrading: a deployment with such a value has been running on SQLite. After correcting
      the URL the PostgreSQL state is empty, so each incremental export starts with a full
      pass; to keep the stored cursors, unset the variable instead and keep the SQLite file.
  • Breaking: rivet plan fails over a source it cannot read, and writes no plan.
    Applies to every engine.

    • Before, when the preflight failed (no connection, a login the source rejects, a table that
      does not exist), rivet plan printed a WARN, exited 0 and wrote a plan whose diagnostics
      said unknown (preflight failed); rivet apply then failed on it. On MongoDB a collection
      that does not exist was planned with no warning at all.
    • Now the command ends with RIVET_PLAN_SOURCE_UNREADABLE and the source's own error, and
      writes no plan file for any export of the config: exit 1, or exit 2 when the cause is
      transient (a refused or dropped connection). A script that relied on exit 0 here now sees
      a failure; fix the connection, the credentials or the table name and plan again.
    • Upgrading: nothing to migrate. A plan file written by 0.31 or earlier over an unreadable
      source is still accepted by rivet apply, which fails on it as it did before.
  • Breaking: rivet validate exits 1 on a bucket or container that does not exist.
    Applies to s3, gcs and azure destinations.

    • Before, a destination whose bucket does not exist (a typo, another account) was reported
      as status: legacy_run, exit 0: the same answer as a prefix that was never written.
    • Now the export's verdict is NO MANIFEST with RIVET_VERIFY_MANIFEST_READ_ERROR naming
      the bucket, and the command exits 1 (could not be verified). An empty prefix of a bucket
      that exists is still legacy_run, exit 0, and so is a local path that does not exist.
    • The same question is asked wherever rivet looks for an object and the store answers 404
      (run --resume, repair, reconcile): a missing bucket is now
      RIVET_DEST_CONTAINER_NOT_FOUND there instead of "the object is absent". It costs one
      extra listing request (one result) per destination per process.
  • Breaking: rivet check describes the plan rivet run builds.
    Applies to mode: chunked on PostgreSQL, MySQL, SQL Server and Oracle, and to MongoDB.

    • Before, the Strategy: and Mode: lines were derived from the config alone. A
      mode: chunked export with no chunk_column on a table that fits one chunk printed
      Strategy: chunked(id, size=100000) and Mode: chunked (column: ?, ...), while the run
      exported it as one unchunked pass.
    • Now both lines (and strategy / mode under --json) come from the planner run uses:
      Strategy: full-scan and Mode: full (mode: chunked runs as one pass: ...) for that
      table, the resolved primary key instead of ? when the run does range-chunk, and the
      resolved chunk_size under chunk_size_memory_mb. The verdict, warnings and
      recommendations are computed for the same resolved plan. rivet plan records its
      diagnostics for the resolved plan as well.
    • On MongoDB, rivet check now fails on a collection that does not exist
      (RIVET_SOURCE_COLLECTION_NOT_FOUND, exit 1), with the refusal rivet run gives; before
      it printed Looks good and exited 0.
    • Exit codes of rivet check are otherwise unchanged; a tool that parses the strategy or
      mode field sees the new values for the exports above.
  • Breaking: a CDC table put back into tables: is refused until it is re-baselined.
    Applies to mode: cdc exports with cdc.initial: snapshot or backfill:, on every engine.

    • Before, a table removed from tables: and added again later kept its old baseline: the run
      exited 0 and every insert, update and delete made while the table was out was missing from
      the destination. The same happened when table: was pointed at another table and the
      destination already held the first table's baseline: the new table got no baseline at all.
    • Now each run records the tables the stream captures. A table that the last run did not
      capture, whose baseline is already recorded complete, is refused
      (RIVET_STATE_CDC_TABLE_REJOINED, exit 5, nothing read or written), on every run, until
      it is re-baselined: move every file out of the table's directory in the destination,
      delete its cdc_snapshot row from the state DB, and truncate its __changes table if a
      warehouse load consumes the stream. The next run reads the table in full and captures it
      from there. A table that was never captured is still baselined when it is added, as before.
    • Upgrading: nothing to do and no state migration. The first run on the new version records
      the current table set without checking it, so a table that was removed and put back
      under 0.31 or earlier is not detected: re-baseline it as above if that happened.
  • Breaking: a keyset run that a gate refused is read again, not adopted, by the next run.
    Applies to chunk_by_key with parallel: N + chunk_checkpoint: true, and to
    keyset_incremental: true (sequential or parallel), when a quality: content rule
    (unique_columns, null_ratio_max) or on_schema_drift: fail refuses the run after its rows
    were written.

    • Before, the refused run kept its checkpoint. The next run adopted its pages, read nothing,
      graded nothing, and ended with exit 0, _SUCCESS and "quality: pass" over the refused
      data; the refused run's run_status row was rewritten to success.
    • Now every re-run reads the same rows again and is graded again: it keeps exiting 3
      (quality) or 4 (schema drift) until the source data or the rule is fixed, and the refused
      runs stay failed. Each refused run leaves its parts on the destination under a Failed
      manifest, as a refused run without a checkpoint always did.
    • A sequential keyset_incremental run no longer moves the stored cursor page by page: the
      cursor moves once, after the manifest is written, so a refused or interrupted run leaves it
      where the run began. A run that breaks off mid-way is still resumed from its last
      committed page.
    • Upgrading: an export whose last run finished needs nothing. An export whose last run
      under 0.31 or earlier was refused by such a gate still gets the old behaviour once: the
      first run on the new version adopts the pages that run left. The same holds for a
      sequential keyset_incremental run interrupted under 0.31 or earlier (its cursor had
      already moved): if the new version resumes it and a gate refuses it, the run warns that
      it "cannot be read again", names the command below, and the next run adopts its pages. To
      have those rows read and graded again, run rivet state reset -c <config> --export <name>
      before the next run.
      The export then starts from the beginning: a keyset_incremental export reads the whole
      table again.
  • Breaking: a run of another mode refuses to continue past an unfinished checkpoint run
    (RIVET_STATE_INTERRUPTED_RUN_OWNER_MISMATCH, exit 5, nothing read or written). The stored
    record of an export now says whether it is a committed high-water or an interrupted run, and
    which mode (keyset, chunked) opened that run.

    • Before (0.31.0 and earlier): a keyset run with chunk_checkpoint: true crashed after its
      first page, the export was switched to mode: incremental without a reset, and the
      incremental run continued from the crashed run's page cursor: exit 0, _SUCCESS, a manifest
      listing 1900 of 2000 rows.
    • Now any other mode that meets the interrupted run is refused, including mode: full and a
      switch away from an interrupted range-chunk run (which lost nothing before; it is refused so
      the interrupted run is finished or abandoned on purpose). Remedy, either one: restore the
      previous mode's settings and run once to finish the run, then switch; or abandon it with
      rivet state reset -c <config> --export <name> (keyset run) or rivet state reset-chunks -c <config> --export <name> (range-chunk run), after which the next run is a full pass.
    • A switch after a run that FINISHED is unchanged: keyset then incremental on the same key
      still continues from the high-water.
  • Breaking: a run without the checkpoint is refused beside an unfinished checkpoint run
    (RIVET_STATE_INTERRUPTED_RUN_OWNER_MISMATCH, exit 5, nothing read or written), the same way a
    run of another mode is.

    • Before (0.31.0 and earlier): a range-chunk run with chunk_checkpoint: true was killed after
      6 of 20 chunks; the export ran once with chunk_checkpoint removed and finished (exit 0); 20
      source rows changed, one per chunk; with chunk_checkpoint: true back, the next run resumed
      the killed run: exit 0, manifest success, 120000 rows, and 14 of the 20 changed rows. The
      other six chunks were the parts written before the kill (PostgreSQL, MySQL, SQL Server).
    • Now the run without the checkpoint is refused while the interrupted run exists: range chunks,
      keyset, parallel keyset, and a MongoDB page_size export with resume removed. Remedy, either
      one: restore the checkpoint setting and run once to finish the run, then remove it; or abandon
      it with rivet state reset-chunks -c <config> --export <name> (range-chunk run) or rivet state reset -c <config> --export <name> (keyset run, MongoDB), after which the next run is a
      full pass.
    • Changed for keyset: a keyset run without chunk_checkpoint used to read the whole table
      beside an interrupted checkpointed keyset run and release it on success. It is refused now.
    • An interrupted range-chunk run counts even when the export's stored row no longer points at
      it (after rivet state reset, or a run left by 0.31.0): another mode and a run without the
      checkpoint are refused there too. rivet state reset-chunks is the command that abandons it.
    • Upgrade, state written by 0.31.0 and earlier: an interrupted chunk run that nothing finished
      beside is refused without the checkpoint and resumed with it, as above. One that a later run
      of the export already finished beside (those versions allowed it) is refused for every config
      (RIVET_STATE_CHUNK_RUN_SUPERSEDED, exit 5) until rivet state reset-chunks -c <config> --export <name>; the next run starts a fresh pass. Both paths were run with the 0.31.0 binary
      first and this version second.
  • Fixed: an interrupted range-chunk run is resumed only by the source that opened it. Two
    configs with one export name, one state database and different sources: when the first crashed
    after a chunk, the second resumed the first one's chunk windows over its own table (exit 0,
    1000 of 5000 rows). Each source now has its own in-progress run; state schema v34 records the
    source on chunk_run and keyset_range.

    • Upgrade: a chunk run left in progress by 0.31.0 recorded no source. It is resumed, with one
      warning, by the next run of that export name when the name has progress under no other source;
      when it has, the run is refused (RIVET_STATE_CHUNK_RUN_OWNER_UNKNOWN, exit 5) until
      rivet state reset-chunks -c <config> --export <name>, and the next run starts a fresh pass.
      Finished cursors and interrupted keyset runs carry over unchanged. Older binaries refuse a
      v34 state (RIVET_STATE_SCHEMA_NEWER).
  • CDC memory: the events of one transaction share its commit position. Closing a
    transaction copied its resume position, a JSON object, into every event (about 475 to 730
    bytes each), which was the largest single item on the heap of a large transaction. The
    position is now one shared value. Measured on MySQL, one transaction: 500 000 changes
    940 MB -> 427 MB peak and 1.90 s -> 1.39 s; 20 000 changes 77 MB -> 51 MB. Peak memory still
    grows with the largest transaction. The transaction-buffer and rollover byte budgets charge a
    position once per run of events that share it, so they keep tracking real memory. Output,
    checkpoint files and the NDJSON stream are unchanged.

  • Breaking: an incremental or CDC load refuses a view or a <table>__changes it has no record
    of loading
    , on BigQuery, Snowflake and ClickHouse. Before, only a foreign TABLE at <table>
    was refused: a view there was replaced by rivet's current-state view, a foreign
    <table>__changes was appended to, and the load exited 0. A base-and-buffer load appended its
    buffer beside a base it had never loaded, and that success made the base rivet's own for
    rivet compact.

    • The load now stops before any warehouse write, with exit 5 and
      [RIVET_LOAD_TARGET_NOT_RIVETS], and names the object. The whole-table refusal
      (refusing to overwrite ...) carries the same code and exits 5; it exited 1.
    • What "no record" means: the state DB this rivet load opened has no load of that target.
      Tables, views and logs loaded under the same state DB are unaffected, and so is a load
      without a state DB. A load pointed at a NEW state DB is now refused where it used to
      append: a lost state DB, another host, or a container whose state DB does not outlive the
      run (refused from its second cycle). Keep the state DB between runs (a volume, or a
      PostgreSQL state), restore it, or drop or rename the view and the log and load again.
    • A state DB that is configured and cannot be read refuses the same objects, as it already
      did for a table.
    • A stop before the write exits as what stopped it. RIVET_LOAD_ADOPTION_COLUMN_MISMATCH
      now reaches the error line and exits 5 (it exited 1 with no code).
  • A load that stops before writing never becomes the reason the next load overwrites. The
    stale-buffer stop of a base-and-buffer load, and a staged part that vanished between the
    ledger read and the write, were recorded failed; the next rivet load then treated a table
    it had never loaded as its own and replaced it. They are recorded refused. The ownership
    question is also asked before the writing ledger row, so a load killed while refusing, or
    one whose state DB failed its closing write, leaves no row that claims the table.

  • PostgreSQL CDC: a failed run no longer acknowledges the slot past what it wrote. When a
    part rolled in the middle of a read window (most often through rollover_memory_mb) and a
    later part of the same run then failed to write, the run still advanced the slot's
    confirmed_flush_lsn to the last transaction it had READ. The