Skip to content

Mason v0.17.0

Choose a tag to compare

@github-actions github-actions released this 12 Nov 01:06
· 127 commits to main since this release
3142780

Two fixes a real deployment met come first. Several first syncs on a small machine could make the synced data
unusable, so that every table's sync failed and every restart failed again: a first sync no longer can, and a run
already stuck that way recovers when it starts on this release, with nothing lost. A run whose synced data met a fatal
error (a full disk, an I/O error) stayed "running" while it answered nothing, until it was restarted by hand: it now
ends as a crash does, its cause said, and is started again. The web page's Tables now choose how each table is kept
current: Off, a poll, or Live (logical replication) where your database is ready for it. mason wait waits for a
table's catch-up poll, mason share opens with the URL to paste, and the card of a user just added has a Copy on every
field.
Every project has something to do first: its pin moves. The first fix has a cost on disk until a future release
compacts tables: watch the disk under .mason/ ("Before you upgrade").

Before you upgrade

  • Move the pin: version: 0.17.0 in mason/mason.yml. A project pinned to 0.16.0 is refused by the 0.17.0 command
    and image, with the way out named: before 1.0 they run a project pinned to their own minor version only, at their own
    patch or an older one. Set the pin and start 0.17.0 in the same step.
  • On a hosted replica, upgrade the replica before you deploy a project pinned to 0.17.0. mason deploy asks the
    replica's Mason version first and refuses a project whose pin the replica would not run, naming both versions and the
    ways out; mason replica upgrade <org>/<name> is one of them, where the service offers it.
  • Watch the disk under .mason/. The synced data now grows on disk where it used to be compacted, and tables kept
    current by the poll or the stream get slower to read on a busy server ("What was wrong and is now right" gives the
    numbers). To compact a table, sync it again whole. With Mason running, run mason remove <table>. Wait until mason logs shows dropped <table>'s synced data, the table named with its schema (dropped public.orders's synced data), then run mason add <table>. The table is not served in between. If you
    add it back while Mason is stopped, or before that line appears, the add cancels the remove and nothing is compacted.
    The file does not shrink, but later writes reuse the space. mason sync does not compact a table.
  • If anything reads tables[].sync.state from --json (mason status, mason wait), handle the new state
    catching-up: a table served and being caught up by one poll, which was syncing before ("What changes").
  • mason add <table> --sync replication while Mason is up now reads your database's checks for logical replication
    first
    (as the web page's Live does, below), in a run that already replicates other tables too, and refuses the
    change, writing nothing, when one fails, naming the failing check (replication role, for instance). The web page's
    Sync choice shows each failing check with the SQL or setting that fixes it. A script
    that switches tables to replication may now be refused at the add, where the edit used to be written and the table
    then failed in the run.

New

  • The web page's Tables choose how each table is kept current: every added table says its sync on its line (Off,
    Every 5 minutes, Live (replication)), and where the page may edit the sources file it opens a choice — Off (the
    default), a poll every 5 minutes, 15 minutes, hour or day, each saying it scans the whole table on your database at
    that interval (a table over a million rows says what each poll costs and asks before saving), or Live, logical
    replication. Live can be chosen only when your database is ready for it: otherwise it is shown greyed out with each
    check that fails — no primary key, wal_level, a user without REPLICATION, no free replication slot or WAL sender, a
    read replica before Postgres 16, the table missing from your publication — and the SQL or setting that fixes it. The
    checks read your database's settings and catalog only, never a row, when the choice opens and again when it is saved,
    so a database that changed in between is refused. Saving writes the table's sync: line in the sources file, as
    mason add <table> --sync … does. Where the project's files are read-only, the page shows the sync without a choice.

What changes

  • mason wait now waits for a table's catch-up poll (after its first sync, a sync of it again whole, or a change
    of its sync: to replication). A first sync reads the table as of its start, then catches it up by one poll for
    what changed during the read, a sync: off table too; on a large table that poll is a scan of its own. mason wait
    said Ready while it ran, so a tool started on Ready read data as of the read's start, and the table's line said "NOT
    kept current" with a yellow bar. Now the table's line says it is being caught up by one poll, its bar at work, and
    mason wait is ready once that poll is over (a script waiting on it may wait that much longer); mason status, the
    web page and EXPLAIN say the same. In --json a table in its catch-up has sync.state catching-up (it was
    syncing), and readiness lists it under waiting.
  • mason share opens with the URL to paste: its first line is now Try mason with your favorite postgres client: postgres://<user>:<password>@<host>:<port>/<database>?sslmode=require, the share's own login and address, so a
    client that takes a connection string connects from that one line. The host, port, database, user and password rows
    below it stay, for a tool that asks for each field, and mason share --json carries the same URL as share.url.
    The password is still made for this run alone and kept nowhere.
  • The web page's card for a user just added has a Copy on every field: URL, User, Password, Database, Host, Port
    and SSL mode, so each can be pasted into a BI tool's connection form one by one. The card no longer shows a psql
    line: paste the URL after psql. mason user add --json keeps its psql field.
  • mason user add, beside a mason start run on this machine's own address, now ends its tips with the way to a
    tool outside this machine
    : mason share opens an address on the internet, with a login and password of its own
    (not the new user's), for as long as it runs. It is not said beside mason serve, nor with --replica, nor while
    Mason is not running. --json is unchanged.

What was wrong and is now right

  • A first sync of a table of wide values on a small machine could stop all of Mason. After the sync read the
    table, the synced data's own checkpoint could fail when it reorganised that table, with failed to offload data block … or failed to pin block … in the log. That made the synced data unusable: every other table's sync failed
    with database has been invalidated, and every restart failed again in the same place. Mason now never reorganises
    a table's storage that way, so the sync finishes and the other tables sync. A run already stuck like this recovers
    when it starts on this release; nothing is lost.

    What this costs until a future release compacts tables: the synced data grows on disk, and tables kept current
    by the poll or the stream get slower to read on a busy server.

    • A table keeps the space of every row the poll or the stream updates or deletes. A table that had every row
      updated three times over took 2.1 times the disk and scanned in 1.7 times the time.
    • Every table written to between two of the synced data's checkpoints keeps a separate block of at least 256 KiB.
      A checkpoint comes about every 16 MiB written and at every stop. In measurements, small appends with 2,000
      checkpoints took a 1 M-row table from 31 MiB to 536 MiB, and its count from 1.2 ms to 2.8 ms. 25 restarts grew a
      project of 8 polled tables by 50 MiB, where it grew by 1 MiB before.

    Meanwhile, watch the disk under .mason/. To compact a table, sync it again whole. With Mason running, run mason remove <table>. Wait until mason logs shows dropped <table>'s synced data, the table named with its schema (dropped public.orders's synced data), then run mason add <table>. The
    table is not served in between. If you add it back while Mason is stopped, or before that line appears, the add
    cancels the remove and nothing is compacted. The file does not shrink, but later writes reuse the space. mason sync
    does not compact a table. EXPLAIN's "about N rows" also counts rows updated or deleted since the table's last sync
    whole.

  • A run whose synced data met a fatal error stayed "running" and answered nothing. When a write to the synced data
    fails in a way it cannot recover from inside the run (a checkpoint on a full disk, an I/O error), every statement
    was refused with XX000 … database has been invalidated, mason status still said running, the request log
    stopped, and nothing restarted it. Now the run ends as a crash does, with the cause and the way out in its log:
    sessions are told 57P02 and to reconnect, mason start's supervisor starts it again (mason status shows the
    restart and its reason), and mason serve exits 1, /healthz saying failed until it does, for the platform to
    start it again. Starting again opens the synced data again; nothing synced is lost. When the restart meets the same
    cause (the disk still full), the supervisor says it is the same as the last restart, and gives up after five
    restarts in ten minutes as for any crash; a full disk that stops the synced data from opening is said as the disk's,
    never with mason destroy as the way out.

  • A table of large values could fail its first sync under a small MASON_DUCKDB_MEMORY, now and then, with
    Failed to commit: could not allocate block of size … in mason status and the log, and wait for its retry (at
    least 5 minutes later). When a chunk of the first sync runs out of memory as it commits, it is now rolled back and
    the sync reads on one chunk at a time, smaller, as it already did when the out-of-memory came in other words.

  • mason add <table> --sync … refused a table added with key: (one with no primary key) as "cannot be added: no
    primary key". It now changes the table's sync: and keeps its key; the web page's sync choice does the same.

Known

  • The synced data is not compacted on its own yet: a table kept current by the poll or the stream keeps the space
    of the rows it updated or deleted until it is synced again whole ("Before you upgrade" says how). Compaction is the
    next release's work.
  • A generated table can't yet hold a numeric that divides (an average is one); is a variance or a standard deviation, which Postgres computes exactly at decimals it picks per value, rounded, cast to numeric(p,s) or not; is a sqrt, exp, ln, log or power, whose decimals Postgres picks per value, rounded, cast to numeric(p,s) or not, whatever the decimals of what it is computed from; is a JSON number's text cast to numeric with no scale (cast to numeric(p,s), it is stored); has more than 18 decimals, at the end or on the way; sums values whose decimals the rows decide; is a CASE, GREATEST, LEAST or COALESCE of values with differing decimals; or is cast from a double; nor a domain or a composite, and kind: live is refused. Each is refused by file and key before anything is built, said as Postgres types the value (a sqrt or an average of a double is a double, so cast to numeric it is a cast from a double), wherever it sits in the column (under an operator, a coalesce, a round, a cast or an aggregate, or in a scalar subquery, a WITH query, a subquery in FROM or a UNION's arm), with the way out your own SQL with the cast moved and the aggregate kept (sum(sqrt(x)::double precision)), or the WITH query, subquery or UNION the cast belongs in, given whole. Each of these keeps its reason read back from its text (::text, ::varchar, || '', a jsonb's ->>, a WITH column), cast to numeric(p,s) or not, so a typmod over a variance's text never stores an approximation; only a stored column's or a JSON number's text read as a numeric is said as a text: a JSON number's, a stored double's or any other. A run that fails (a value past 38 digits, bigint out of range, division by zero) records Postgres's error, SQLSTATE and DETAIL (numeric field overflow says the precision and scale it must fit, on every path), and is tried again after its back-off.
  • mason status shows generated tables; the web page doesn't yet.
  • Pages of every LIMIT with no ORDER BY over a join may overlap: LIMIT n OFFSET m with no ORDER BY over a join has
    no stable order, so a tool paging one can see a row twice and miss another. Add an ORDER BY where the pages matter.
  • A replicated table whose re-read never completes can rejoin the stream with no catch-up, and miss what the stream
    passed over, as in 0.8.0 to 0.16.0. It was found by reading the code and has not been reproduced.