Skip to content

flusso-source-postgres-v0.18.0

Choose a tag to compare

@flusso-release-plz flusso-release-plz released this 19 Nov 10:38
· 83 commits to main since this release
3ed4982

Added

  • (engine,daemon,source-postgres) [breaking] a stop drains what is in flight before the run ends
  • (bench) review-ab --digest and --plans over the cases roots (#347)
  • (bench) review-ab shows each side's range; a wide composite-key root and a never-analysed root
  • (bench) just review-ab compares a branch with its base on the Docker-backed benches
  • (source) warn on a timestamp format that refuses the ISO form
  • (design) author array columns (#276)
  • (source) [breaking] validate array columns against the catalog (#276)
  • (source-postgres) clean up an array column's value as the document is decoded (#276)
  • (source-postgres) per-element transforms on an array column (#276)
  • (kernel) a column may hold an array of its type (#276)

Fixed

  • (engine) a snapshot row is its own document, not a change to resolve
  • (engine,source-postgres) a stop that can't confirm its final position fails; a deferred sink refreshes past skipped batches
  • (source-postgres) a stop right after the start doesn't wait for a status update
  • (cli) bound the stop, honour a second signal and SIGHUP, follow the parent
  • (script,source-postgres) a declared float in --input is a float; jsonb zero padding is not digits
  • (source-postgres) a float reads as Postgres casts to float4; exact reads pinned in flusso-query
  • (source-postgres) an untyped number is a decimal to a script at every magnitude
  • (source-postgres) an ids of float keys reads each key as an f32
  • (source-postgres) read and write every number exactly
  • (source-postgres) date a snapshot by its read, not by when the stream took it
  • (source-postgres) a snapshot task that can't miss a commit; bounded reads; one hold clock
  • (source-postgres) [breaking] emit a change only once its transaction is visible
  • (bench) a plan check that cannot read its input is a setup error (#347)
  • (bench) the review harness is a bench target of its own (#347)
  • (test) the guard rejects the postgres import, not call shapes
  • (test) every test and bench starts Postgres through dev/test-postgres
  • (source-postgres) [breaking] a filter value is never cut or rounded to its column's size
  • (bench) review-ab only adopts a pre-marker cache with a real stamp and nothing foreign; --clean-results apart from --clean
  • (bench) review-ab's spread narrows with more rounds; --clean deletes only its own cache; owner-labelled containers; cargo config in the stamp
  • (bench) review-ab re-runs a flagged case before calling it slower, alternates sides, and runs one at a time
  • (source-postgres) a batch whose prepared statement changed result type reruns once
  • (source-postgres) write a batch's keys as an array constructor, padded to a power of two
  • (source-postgres) a shared key whose document reads only its key orders nothing
  • (source-postgres) choose a shared key's row by its primary key or the columns the document reads
  • (source-postgres) a batch by key reruns by position when its cached key type is stale
  • (source-postgres) a name key pairs by position, since its cast cuts a requested key
  • (source-postgres) pair a batch's positions in a full join, which never runs as a nested loop
  • (source-postgres) the source's own sessions run without JIT
  • (source-postgres) build the batch at the root's own level and pair positions one level out
  • (source-postgres) name the batched build's key list so no table's column meets it
  • (source-postgres) a batched build's root probe returns one row, so it is never JIT-compiled
  • (source) say what an unreadable root key type costs now
  • (source-postgres) read a key type's name as the qualified identifier it is
  • (source-postgres) a role that may not run has_schema_privilege still reads its key types
  • (source-postgres) Postgres decides which row answers which requested key
  • (source-postgres) mirror real, boolean, time and uuid casts in the batched build's matching
  • (source-postgres) a key of a domain over a type the role may not use keeps indexing
  • (source-postgres) the batched build serves the ids a single build serves, across key kinds
  • (source-postgres) a numeric key equal to its row's is built, not tombstoned
  • (source-postgres) a key is never cast to a size, and a resize refreshes the cached types
  • (source-postgres) statements survive a column type change
  • (source-postgres) only a declared decimal changes what the sink receives
  • (source-postgres) [breaking] write every numeric digit as a JSON number
  • (source-postgres) a refused link probe is cached like a verdict (#324)
  • (source) a check the database would not let run is reported, not dropped (#324)
  • (source) [breaking] a schema that can never index a document stops the start (#316, #309)
  • (source-postgres) a Relation that omits a column is no evidence its type changed (#298)
  • (source-postgres) a table's Relation message drops its cached column types (#298)
  • (source-postgres) a failed document query forgets its tables' column types (#298)
  • (source-postgres) wrap an array of a bytea domain; keep the session SET as a backstop (#298)
  • (source-postgres) a binary field holds the column's bytes whatever bytea_output is (#298)
  • (source-postgres) say a lossy ids key may be stored altered; check prints a skipped probe (#299, #313)
  • (source-postgres) probe a type pair once, in one round trip; pin exact ids pairs (#299, #313)
  • (source-postgres) check reports a link or ids element the key can't take (#299, #312, #313)
  • (query-derive) refuse by name only what can't be a string type of your own
  • (query-derive) say only what the derive promises; make the boundary test cheap
  • (source-postgres) resolve each change as it is buffered again (#295)
  • (source-postgres) compare a resolved value as the join compares its column (#295)
  • (source-postgres) parse years past 9999, so their ids keep main's form (#295)
  • (source-postgres) read a batch's root keys back in one query per index (#295)
  • (source-postgres) set no startup options, so PgBouncer passes (#295)
  • (source-postgres) a link names the root's own document, however it spells the key (#295)
  • (source-postgres) read a key back as the text the WAL sends (#295)
  • (source-postgres) refuse a jsonb or json root key (#295)
  • (source-postgres) read every key type the same way in backfill, WAL and build
  • (source-postgres) read bytea as hex whatever the database's bytea_output
  • (source-postgres) a constant builds the value the schema holds
  • (source-postgres) index a binary column as its bytes, not Postgres hex text
  • (cli) script run reports a document that fails to build as the failure
  • (source) a domain over an array or jsonb holds an array
  • review findings on array columns (#276)
  • (source-postgres) a transform probe the role cannot run is not a verdict
  • (source) [breaking] settle the root-key kinds; probe transforms without a table
  • (source) report four schemas check accepted that cannot work

Other

  • comments say what the code does, not which review or stage changed it (review L4)
  • a mark records the comparison operator the walk used, so one taken with the bare < is never resumed with the qualified one; compare_keys' doc says an unplaced lane starts from the first key
  • the source marks a walk's point and resumes after it in its own key order (walk_mark, resume_after, compare_keys)
  • changes are resolved on a connection of their own (step 2)
  • review follow-ups: no watermark left, wal_idle's half named, the argument per lane
  • a walk ends at the greatest key it found, read through the key index's own operators
  • READMEs and a doc link no longer name the stream's watermark
  • rustfmt the review harness
  • wal_idle confirms Idle once a second, as the engine's tick does
  • the source confirms and fences from Landed; the Postgres capture keeps no map
  • Position carries its seq and the source's stamps (touches all constructors)
  • a backfill pages each index's root keys on the builder pool
  • (source-postgres) an untyped whole-number double is the integer it reads as
  • (source-postgres) name the untyped number rule the reader follows
  • (source-postgres) a commit arriving as the snapshot task stops is still checked
  • (source-postgres) keep a snapshot fresh while transactions await a check
  • (source-postgres) read the visibility snapshot on its own task
  • (source-postgres) measure a small batch's plan on one connection
  • (source-postgres) a small batch's plan test refuses a server that counts no plans
  • (source-postgres) a small batch's cached plan is measured the same way on every major
  • trailing spaces on varchar(n), an array of a sized domain, rewrapped lines
  • (source-postgres) a fitting filter value still matches its row for every sized type
  • (source-postgres) a filter value a domain's CHECK refuses matches nothing
  • (source-postgres) a domain filter compares as written for a role that may not probe schemas
  • (source-postgres) a filter value that does not fit its column is compared as written
  • (source-postgres) is_partitioned takes the name in the form each caller holds
  • (source-postgres) the differential stores three uuid rows, so a key mix-up shows there too
  • (source-postgres) a batched statement takes its shape whole, and the proptests draw both key forms
  • (source-postgres) one query answers whether a table is partitioned
  • (source-postgres) pruning is checked past the small-batch size, and plans are read once logged
  • (source-postgres) the differential reaches both ways a batch sends its keys, and partitioned roots
  • (source-postgres) a small batch's plan count is bounded per connection, not per call
  • (source-postgres) large batches on an unpartitioned root pass their keys as array parameters
  • (source-postgres) a batch survives its key column changing from text to name
  • (source-postgres) a small batch reuses its plan, and a generic plan still prunes partitions
  • (source-postgres) a shared key whose document reads only its key still builds
  • (source-postgres) which connections set jit = off, and what it costs and gains
  • (source-postgres) pass a batch's keys as one array parameter per kind
  • (source-postgres) choosing a shared key's row reads no column the document does not read
  • (source-postgres) a batch by key with a stale type cache still finds its rows
  • (source-postgres) the differential covers name keys and keys past 63 bytes
  • (source-postgres) pair a batch by key where the key's = is identity
  • (source-postgres) a batched build pairs positions without a nested loop
  • (source-postgres) the batched build's scope, the builder's jit setting and duplicate keys
  • (source-postgres) the batched build's chunk and key check no longer describe an IN list
  • (source-postgres) a raw filter reads the root row's columns in a batched build
  • (source-postgres) a batched build on a partitioned root is not JIT-compiled
  • (source-postgres) number the batched build's keys with unnest WITH ORDINALITY
  • (source-postgres) a generated build/build_many differential, a revoked privilege probe, offset-less timestamptz text, quoted key type names
  • (source-postgres) both build paths agree for real, uuid, bool and time keys too
  • (source-postgres) both build paths agree across key kinds; a domain over a type the role may not use
  • (source-postgres) a BigDecimal key binds as text and matches its row by its text
  • (source-postgres) keep resolve's happy path as on main; one mismatch message for both query kinds
  • (source-postgres) pin resized keys, numeric keys of another scale and the resolve mismatch message
  • (source-postgres) match a batched build's rows to ids in one function
  • (source-postgres) widen the differential corpus to 7605 cases
  • (source-postgres) a wide document's first failure is the first declared
  • only a declared decimal may change what the sink receives
  • a numeric keeps every digit as a JSON number; a non-finite one fails the document
  • the resolve bench's walked path, the per-path root-link answer and the generation
  • (source-postgres) look up an already-warned unread table without a clone or an exclusive lock
  • (source-postgres) a change on an unread table warns once per table
  • (source-postgres) find each path once per builder and keep its root-link answer
  • (source-postgres) count the catalog lookups and path searches resolve makes per change
  • (source-postgres) bench a path walked from the key, and an unread table after its warning
  • (source-postgres) seed the self-ancestor set only when the root table has paths
  • (source-postgres) read root keys back only for a change that has links
  • (source-postgres) bench resolving the changes that need no query, in process
  • (source-postgres) a refused link probe is asked once per type pair (#324)
  • (source) a check the database would not let run is reported (#324)
  • (source-postgres) a Relation that omits a cached column keeps the cached types (#298)
  • (source-postgres) a link or ids element of a type the key can't take (#299, #312, #313)
  • warn on a timestamp format that refuses the ISO form; pin tuned mappings
  • (query-derive) typed-map shapes the review found still compile and fail (#294)
  • (source-postgres) restore the recorded pgoutput fixture (#295)
  • (source-postgres) a jsonb or "char" root key is refused (#295)
  • (source-postgres) every key type in every relation position (#295)
  • (source-postgres) array columns against a live Postgres (#276)