Skip to content

Releases

Mason v0.23.0

Choose a tag to compare

@github-actions github-actions released this 16 Nov 20:53
fbf6a67

A column your sources file leaves out is no longer served under another name, after a rename or a swap on your
database. A table synced whole is no longer served with rows missing until its catch-up. A sync that waits for an
open transaction on your database now says Waiting, not Failed. A WITH query named like the table it reads now gives
Postgres's answer, and TABLESAMPLE is refused by name. A grouped min() or max() of a text over few groups, a
sorted SELECT DISTINCT and mode() are many times faster under a libc collation, and so is to_char with a quarter
or a padded name. The web page shows how to connect an MCP client.
This is a new release line, so every project's
pin moves. No table is read again. Syncing a column the sources file left out now takes mason add. A script that
reads --json or the events for a sync that waits may need to change ("Before you upgrade").

Before you upgrade

  • No table is read again. At the first start, Mason records which column is which on your database. It reads no
    table's data again.
  • Move the pin: version: 0.23.0 in mason/mason.yml. The 0.23.0 command and image refuse a project pinned to
    0.22.x, and say how to fix it. A 0.22.x command or image refuses a project pinned to 0.23.0. So set the pin and
    start 0.23.0 in the same step, once everyone and everything that runs the project has 0.23.0.
  • The 0.22 line still gets security fixes. They are released as 0.22.x patches, from the release/0.22 branch. A
    project pinned to 0.22.0 can stay on that line until you are ready to move.
  • To sync a column the sources file left out, run mason add. Mason now knows each column by its identity on your
    database, not by its name alone ("What was wrong and is now right"). A column the file left out is synced again
    only once you confirm it. This covers a column that exclude: names, one a columns: entry does not list, and a
    secret-looking column mason add left out.
    • Before, you could take a name out of exclude: or add it to columns: by hand, and Mason synced the column. Now
      Mason holds that table instead. It stops syncing the table, and keeps serving the data it holds.
      The table's line in mason status and on the web page says why.
    • To sync the column, run the mason add <table> --columns … line that the table's line gives. It names every
      column to sync. A plain mason add <table> would leave secret-looking columns out again. You can also remove the
      table's entry and add it back.
    • A script or CI job that edits the sources file to sync such a column must run that command instead.
    • In a project deployed read-only (MASON_PROJECT_READONLY=1), the committed file still syncs a column as it says,
      if the column did not move on your database. A column that moved is held, such as one that was renamed. To sync
      it, take the entry out of the committed file and start Mason. Then put it back and start Mason again. The table is
      read again whole, and is not served from the first start until that sync ends.
    • Nothing is held at the first start. Mason records which column is which as it takes the columns by name.
  • Check any script or alert that reads mason status --json, mason wait --json or the events for a sync that
    waits for an open transaction on your database ("What changes" has each):
    • The wait is no longer the table's error. Its sync.state is waiting, not failed, and a new syncWaits
      gives why and checkedAt.
    • Its event is now sync_waits, which is not a failure. Before, it was full_sync_failed. full_sync_failed is
      still sent when a whole sync fails.
    • mason wait keeps waiting for a first sync that waits, within its --timeout. Before, it exited 1.

What changes

  • A sync that waits for an open transaction on your database now reads as waiting, never as failed. Mason waits
    when a table was rewritten after a transaction that is still open began. It checks again every 30 seconds and goes
    on by itself.
    • mason status and the web page now say Waiting, with the transaction the sync waits for. Before, they said
      Failed.
    • mason wait keeps waiting for a first sync that waits, within its --timeout. Before, it exited 1, so
      mason add <table> && mason wait read the wait as a failure. A table synced before is served meanwhile, so
      mason wait is ready for it.
    • In mason status --json and mason wait --json, the wait is no longer the table's error, and its sync.state
      is waiting. A new syncWaits says why, and when Mason last checked. If a script or an alert looks for failed
      to catch this case, it no longer matches. Another error on the table, such as its stream's, is still said.
    • The event is now sync_waits, and it is not a failure. Before, it was full_sync_failed.
    • The message names the transaction and its pid only while that transaction is still open. Before, it could name
      one that had already ended, whose pid may belong to another session by then.
  • You can see, before you upgrade, which synced tables the new release reads again whole. Run
    npx @mason-data/cli@<version> upgrade --check in the project's folder while Mason runs, with the version you are
    upgrading to. It lists each table that release's first start reads again from your database, why, how large it is,
    and whether it keeps serving its synced data until its new sync is done. It changes nothing. --json gives the same
    as data.
    • It works only for an upgrade from this release or a later one. An older Mason does not say what its synced data
      holds, and the command says so. For an upgrade from 0.22.0 or before, read each release's "Before you upgrade".
  • A grouped min() or max() of a text or varchar, and a SELECT DISTINCT sorted by one, are faster under a
    libc collation
    , such as a database whose collation is en_US.utf8. Over 4.5 M rows, the min and max of names by
    region took 29 s and now take about 3 s, against Postgres's 0.4 s. The distinct values of a code column, sorted, took
    13 s and now take 0.2 s, about Postgres's time. The answers are Postgres's, as before. Sixteen such queries at once
    under a 1 GB memory limit now finish in a tenth of the time, and Mason's peak memory was about 8 % higher.
    • A query with many groups (more than a sixteenth of its rows, such as a min and max per name) is not faster. A min
      and a max of the same column over many groups is slower: over 3.9 M groups, it took 60 s, where 0.22.0 took 43 s.
    • These are still slow: a grouped min or max over many groups, FETCH FIRST … WITH TIES sorted by a text, a min or
      max with FILTER or under ROLLUP, and a SELECT DISTINCT with no LIMIT over a column whose values are nearly
      all different.
  • mode() WITHIN GROUP (ORDER BY …) of a text or varchar is faster under a libc collation, such as a database
    whose collation is en_US.utf8. Over a million rows, the most common status per tenant took 2.2 s and now takes
    under 0.1 s, about a fifth of Postgres's time. The answers are Postgres's, as before: of values equally common, the
    first in the collation's order wins. Under a locale other than English, a value holding a character Mason cannot order
    as the collation does now refuses the query only when it is among the most common values of its group. Before, any
    such value it read refused it. These are still slow: a mode() of a char(n) or a citext, one ordered with
    USING, and one of a value that calls random() or reads a subquery or an outer query's column.
  • to_char of a date or a time is faster with a quarter, a padded name and most other patterns. Over a million
    rows, a count by to_char(created_at, 'YYYY "Q"Q') or by to_char(created_at, 'Month YYYY') took about 4 s and
    now takes 0.1 to 0.15 s, where Postgres takes about 0.4 s. This now holds for most patterns, padded or with FM,
    with TH after a number. These are still slow: TM, FX, a time zone's TZ, TZH, TZM and OF, SP, FM
    before literal text, a backslash, a format passed as a parameter, a year before 1 or after 9999, and a date formatted
    with a time of day.
  • The web page's Users page now shows how to connect an MCP client. The Connect box is now titled "Connect a
    Postgres client or BI tool". Under it, a new box, "Connect an MCP client", gives the line to run for Claude Code and
    the JSON for Claude Desktop, Cursor and other clients, each with a Copy button. Both run mason mcp through npx,
    pinned to the version of Mason you run, so the client's computer needs Node 22.13 or later. The web versions of
    ChatGPT and claude.ai can't connect this way yet.
    • When you add a user, its dialog has the same two in a closed section, "Connect an MCP client", with the user's
      password in them. Like the password, they are shown only once.
  • On the web page's Users page, "Temporary cloud login" is now "Temporary cloud tunnel". Its button is now "Open a
    cloud tunnel". mason share is unchanged.
    • Its card now shows the Database and the SSL mode, each with a Copy button, in the order of the Connect box: Host,
      Port, Database, SSL mode, Login. A BI tool's form asks for both. The SSL mode is always require, and a tool
      that connects without SSL is refused. Before, they were only in the URL in the dialog shown once.
    • It now counts down to its end: "Online · ends in 59 min, at 18:55". The dialog shown once counts down too. A
      tunnel with no end says "Online until you stop it". Before, it counted up: "Public for 53 s".
    • It says it usually opens in a few seconds. Before, it said that opening one can take up to 30 s. Mason still
      gives up after 30 s, and says so.
  • mason status --json now gives each open share's database and sslmode, beside its host and port.
  • **On the web page's Tables list, an added table's columns and its sync have columns o...
Read more

Mason v0.22.0

Choose a tag to compare

@github-actions github-actions released this 15 Nov 20:24
e5b8058

Adding a table now says up front when it likely won't fit on disk, and you can go ahead anyway. Mason no longer
loses synced rows when your database stops keeping a table's key unique, whether the table is polled or under
sync: replication. A table rewritten on your database while Mason reads it is no longer served empty or partial.
Names that differ only in case, such as "Orders" and orders, are refused by name. Before, such a table failed its
first sync for ever, or two tables were served as one.
This is a new release line, so every project's pin moves.
Mason now refuses to start when the sources file names such a pair. Some tables are read again whole from your
database at the first start. A script that adds tables may need --yes ("Before you upgrade").

Before you upgrade

  • Some tables are read again whole from your database, once, at the first start. Plan for a full read of each.
    These are the tables:
    • A table whose primary key has INCLUDE columns, when its entry's columns: leaves out any of them. It keeps
      serving its old data until the new sync is done, and the log says so.
      • Why: Mason used to count an index's INCLUDE columns as part of the table's key. A table's key is always synced.
        So for PRIMARY KEY (id) INCLUDE (v, w), Mason synced v and w even when columns: left them out. Now the key
        is id alone, so v and w are synced only if the entry names them.
      • To avoid the read, add every INCLUDE column to the entry's columns: before you upgrade.
      • A table whose entry is {} or uses exclude: is not read again.
    • A table whose synced data an older Mason lost. Only a project that removed one of two tables whose names
      differ only in case, such as "Orders" and orders, has one. It is the table that was kept. It is not served
      until its new sync is done. It was not served before the upgrade either. You don't need to remove or add it.
    • No other table is read again.
  • Move the pin: version: 0.22.0 in mason/mason.yml. The 0.22.0 command and image refuse a project pinned to
    0.21.x, and say how to fix it. A 0.21.x command or image refuses a project pinned to 0.22.0. So set the pin and
    start 0.22.0 in the same step, once everyone and everything that runs the project has 0.22.0.
  • The 0.21 line still gets security fixes. They are released as 0.21.x patches, from the release/0.21 branch. A
    project pinned to 0.21.0 can stay on that line until you are ready to move.
  • Mason now refuses to start when the sources file names something it cannot sync because of the case of its
    names.
    Postgres keeps "Status" and status apart. The synced data holds names without regard to case (A to Z),
    so it takes them for one name. The refusal names the entry and the line to write. mason tables does not say so of
    an entry the file already holds, so run mason start to find out. There are two cases:
    • A table with two columns whose names differ only in case, such as "Status" and status. Such a table never
      synced: its first sync failed again and again. Leave one of the two columns out. The refusal says how, such as
      exclude: [status], or the names to take out of columns:. If both columns are in the table's primary key, Mason
      cannot sync the table, and the way out is to remove its entry.
    • Two tables whose names differ only in case, such as "Orders" and orders, or "Billing".invoices and
      billing.invoices. Before, Mason served one of the two tables under both names. Remove one of the two entries from
      the sources file, then start Mason. The table you kept is read again whole from your database, once. It is not
      served until that sync is done, because the synced data may hold the other table's rows. Every other table is
      served throughout.
  • Check for missing rows in a table whose primary key has an INCLUDE column that can be NULL. Before this
    release, a poll of such a table could delete synced rows whose INCLUDE column is NULL, while your database still
    held them. Upgrading stops further loss. It does not bring the lost rows back. A lost row comes back only when it
    changes on your database.
    • To tell if a table is affected, run \d <table> in psql on your database. Look for a primary key with
      INCLUDE (…), where an included column is not not null. Then compare SELECT count(*) FROM <table> on your
      database and on Mason.
    • To get its rows back after upgrading, sync it again whole. Run mason remove <table>, then mason add <table>,
      then mason wait <table>. The table is not served from the remove until that sync ends, so plan it for a quiet
      time.
    • Where the project's files are read-only, take the table's entry out of the sources file and start Mason with it.
      Then put the entry back and start Mason again. The table is read again whole, and is not served until that sync
      ends.
  • A script that adds tables may now be refused for disk. mason add now refuses a table that likely won't fit on
    disk, with exit 1 and the code disk_short. mason deploy asks at a terminal, and refuses the same way elsewhere.
    Where a script or CI must go ahead regardless, add --yes. A script that calls POST /api/pick itself now gets a
    409 for such a table, with the code disk_short and the numbers in disk. Send "yes": true in the body to add it
    anyway. "What changes" says how the check works.

What changes

  • Adding a table now says up front when it likely won't fit on disk. Before, its first sync started, and stopped
    only once it was under way and the disk ran short.
    • Mason estimates how much disk the table's synced data needs, for the columns it syncs. The estimate reads your
      database's catalog only, never the table. It is rough, and it errs toward warning.
    • The estimate is held to the free disk. Mason first takes off the room it keeps for queries that spill to disk
      (half of what is free, or MASON_SPILL_LIMIT), and what other first syncs under way still need.
    • A table whose columns you change is synced again whole. Its synced data stays on disk until the new sync is done,
      so the new sync must fit beside it.
    • When it likely won't fit, the table is not added, and you are told the numbers. Choose fewer columns, give Mason
      more disk, or go ahead anyway: Add anyway on the web page, or --yes on mason add and mason deploy.
    • mason deploy judges a table by an estimate of all its columns, or by its synced size now. So choosing fewer
      columns does not change its answer.
    • The check during a sync is still there. Once a sync has read about 1% of the table, Mason projects what the rest
      needs, and stops the sync when it won't fit. The rows synced so far are kept. This projection is not exact either,
      and on a small disk the disk can fill before it runs.
    • The Tables page shows each table's estimated size before you add it, and mason tables says it too.
    • When a sync runs short of disk, its failure event and mason status --json now carry the code disk_short and
      the numbers, so a script need not read the message.
  • The web page links to how to change a deployed project. When a deployed project's Tables page says it is
    read-only, its link now goes to the README's new "Change a deployed project" section. A host that runs Mason for
    others can link to its own steps instead. Set MASON_UI_DEPLOY_DOCS_URL on mason serve to a path on the page's
    host, or to an https:// or http:// URL. It follows MASON_UI_BACK_URL's rules. A value Mason cannot use stops
    the start with a message that says what is wrong.
  • Every list on the web page now behaves the same in a narrow window. Where a list is wider than the window, it
    scrolls sideways inside its box, and each row's buttons stay in view at its right. The page itself never scrolls
    sideways.

What was wrong and is now right

  • A table synced by a key: no longer loses synced rows when your database stops keeping that key unique. Mason
    checks a key: against your database's unique indexes when it reads the sources file. Before, if that index was
    dropped while Mason ran, two rows could share a key. A poll then deleted both of them from the synced data, and put
    back only the one that changed.
    • Now Mason asks your database whether a unique index still covers the key. It asks before each poll, at
      mason sync, and before it syncs a table whole. The check reads only your database's catalog, in under a
      millisecond.
    • If none does, Mason does not sync that table, and its synced data stays as it was. mason status, mason sync
      and the web page say so. They name the sources file and the key, and the unique index to name instead, if the
      table has one.
    • A polled table whose primary key is dropped is held to the same check.
    • Mason asks your database again every minute, and syncs the table once the key holds again. The log says once
      that the table waits for its key, and again only if the reason changes.
  • A table under sync: replication no longer loses a synced row when its primary key is dropped on your
    database.
    Before, a row inserted with the same key as a synced row replaced that row in the synced data, and
    nothing reported it. This happened with every replica identity, and when the key was dropped in the same
    transaction as the insert.
    • Now the stream checks the table's key after any change to the table's definition, before it applies the table's
      next change. Most tables need no extra query. A table with REPLICA IDENTITY FULL, or one whose key is gone,
      costs one catalog query on a short connection.
    • If the key no longer keeps the rows apart, Mason stops applying that table's changes, and keeps serving its synced
      data. The other tables keep streaming. mason status and the log say why, and what to do: restore the primary
      key,...
Read more

Mason v0.21.0

Choose a tag to compare

@github-actions github-actions released this 15 Nov 10:17
083f879

A column added to, renamed in or dropped from a synced table on your database now reaches Mason without a restart.
A numeric column with no precision, such as Rails makes from t.decimal, prints each value as your database does:
12.5, not 12.50. A table under sync: replication no longer blocks ALTER TABLE … DROP COLUMN on your database.
mason stop ends within seconds when a read waits on a lock there, and removing a table stops its sync at once.
A sync says the rows it added, updated and deleted. Listing a JSON column's keys, and the smallest and largest value
of a text column, are many times faster. node-pg's array parameters work.
This is a new release line. Every project
has something to do first: its pin moves. A generated table with a jsonb column makes the first start longer, and
nothing is served until it is done. A script that reads mason sync --json or its exit code may need to change
("Before you upgrade").

Before you upgrade

  • Move the pin: version: 0.21.0 in mason/mason.yml. The 0.21.0 command and image refuse a project pinned to
    0.20.x, and say how to fix it. A 0.20.x command or image refuses a project pinned to 0.21.0. Before 1.0 they only
    run a project pinned to their own minor version, at their own patch or an older one. So set the pin and start 0.21.0
    in the same step, once everyone and everything that runs the project has 0.21.0.
  • The 0.20 line still gets security fixes. They are released as 0.20.x patches, from the release/0.20 branch.
    A project pinned to 0.20.0 can stay on that line until you are ready to move.
  • On a hosted replica, upgrade the replica first, then deploy the project pinned to 0.21.0. mason replica upgrade <org>/<name> installs 0.21.0 beside the version the replica runs. The replica keeps running your project as
    it is until you deploy. Then set version: 0.21.0, make the changes these notes ask for, and run mason deploy.
    0.21.0 starts with that deploy. mason deploy refuses a pin the replica would not run, before it sends anything,
    and names both versions.
  • Allow for a longer first start if a generated table has a jsonb column. At Mason's first start on 0.21.0,
    Mason rewrites each such table's jsonb values once, in the form Postgres stores them ("What was wrong and is now
    right").
    • Nothing is served until it is done. /healthz answers 503 meanwhile.
    • It takes about 30 to 45 seconds a million rows on a 4-core machine. The log names each table rewritten.
    • If your platform restarts a container that stays unhealthy, give the first start room in its health check for
      your largest generated tables. The image's own check gives up after about 2 minutes 45 seconds. The rest of the
      start counts against it too, so it covers about 3 million rows.
  • Some synced tables are synced again once, at the first start. Each such table keeps serving its old data until
    its new sync is done. A table takes about as long as its first sync did, and is read from your database again.
    mason status says so on the table's line while it waits. You don't need to do anything. These are the tables:
    • A table with a numeric column of no precision. Mason now keeps each value's own decimals, which the data
      synced before does not hold. Until the new sync is done, its values print the old way.
    • A table that an older Mason left without one of its columns. This is a table with a column whose name begins
      with __mason_json_, which that Mason dropped ("What was wrong and is now right").
    • A table with a column whose name begins with a byte-order mark (U+FEFF), when that column is not of a text
      type.
      An older Mason stored such a column as text. A column of type text, varchar, char, name, inet,
      citext, json or jsonb was not affected, and its table is not synced again. Nor is a column of a domain.
  • Over replication, an older Mason removed some invisible spaces from array elements. This hit the elements of a
    text[], a varchar[] or an enum array. An element that began or ended with a byte-order mark (U+FEFF), a
    non-breaking space (U+00A0), an ideographic space (U+3000) or a similar space lost it in the synced data. 0.21.0
    keeps these spaces in every new change. A row changed before the upgrade stays as it is until it changes again, or
    until its table is synced again. If this matters to you, sync again each replicated table with such a column:
    mason remove <table>, then mason add <table>. The table is not served until its new first sync ends.
  • Check any script that reads mason sync, its --json or POST /api/sync ("What changes" has each):
    • mason sync --json gives added, updated and deleted for each table, in place of changed.
    • A sync of a table that is already syncing is refused with the new code busy, and exits 1. Its --json has a
      syncing key beside the failure. Where a table's first sync, whole sync again or catch-up is under way, the code
      was nothing_yet and is now busy. nothing_yet stays for its other cases, such as a first sync that failed.
    • POST /api/sync answers 202 as soon as the sync starts. Add "wait": true to the body to get the old answer.
    • A script that matched the words of a synced, polled, streamed or full_sync event must match the new words.
    • A script that matched the text of sourcesReadOnly in mason status --json must match its new text.

What changes

  • A sync now says how many rows it added, updated and deleted, such as "1,200 added, ~282,770 updated, 0 deleted".
    Before, it said "283,970 changed", which lumped new rows and edited rows together. The new words are in mason sync,
    the web page's Sync now and Sync events, and mason logs --events. A first sync, or a table synced again whole, adds
    every row.
    • A poll's updated count is an estimate, shown as ~N updated. A poll counts each changed row once, by its key. So
      it can count rows that it read again while a transaction was open on your database, which makes it too high. It
      also counts a row updated several times as one, and a row whose key changed as one added and one deleted. A count
      of 0 is shown without ~.
    • mason sync --json keeps the plain number in updated.
    • Over replication, the counts follow your database's own inserts, updates and deletes, and are exact. mason status, the web page and the run's log say them the same way, such as "1,000 added, 200 updated, 4 deleted".
      Before, they said "1,204 changes (1,000 inserts, 200 updates, 4 deletes)".
    • A poll's own lines in the run's log still say "changed".
  • A sync of a table that is already syncing, or waiting to start, is refused at once. This holds for the web
    page's Sync now, for mason sync and for POST /api/sync. It is never queued for another read of the table.
    • It is said as progress, not as a failure. mason sync prints, on stdout, "public.orders is already syncing. It
      started 2 min ago. Mason never starts a second sync beside it." Then it says how to follow the sync. The page says
      it beside Sync now, and that the table's row shows when it ends.
    • The refusal has a new code, busy, and exits 1, since the sync you asked for did not run.
    • With --json, it is the failure shape with code busy, and syncing beside it. syncing is { tables, startedAt } of the sync running, or null when it is Mason's own sync of the table.
    • mason sync names mason status when a sync by hand or a scheduled poll is running. When the table's first sync,
      whole sync again, catch-up or column change is under way, it names mason wait <table> first.
  • POST /api/sync answers as soon as the sync has started. It answers 202 with { sync: { id, tables, state, startedAt, endedAt } }. Each table in /api/state then has syncByHand. It says syncing until the sync ends.
    Then it says synced with what the sync did, or failed with why. A script that calls the endpoint and needs the
    old answer adds "wait": true to its body. mason sync does this, so it still waits and prints the same report. A
    refusal is answered at once, as before.
  • A sync you start by hand that fails is now listed under Sync events, as "Manual sync failed" (sync_failed in
    mason logs --events).
  • The web page's Sync events list shows each sync while it runs. A sync you start by hand, a first sync, a table
    synced again whole and the catch-up after it each get a row at the top of the list. The row says "Syncing…", and
    Took counts how long it has run so far. When the sync ends, its event takes the row's place. Scheduled syncs are not
    shown while they run, as each takes a moment. mason status --json gives each table's own sync running now as a new
    field, rebuilding, with kind (copy or catch-up) and startedAt.
  • The Sync events list shows how long each sync took in a column of its own. The Took column holds it, such as
    570 ms or 7 min 57 s, and you can sort the list by it. mason logs --events --json gives it as a new field,
    tookMs. A first sync's event now ends with its time as a poll's does, such as synced: 1,000 row(s) · 2.0 s.
    Before, it said in 2.00 s. Events recorded before the upgrade keep their time in the message.
    • In a narrow window, the list scrolls sideways inside its box, as the Tables and Users lists do. A very long table
      name is cut short. Hover over it to read it whole.
  • Listing a JSON column's keys is now about as fast as on Postgres. jsonb_object_keys and json_object_keys of a
    column took 16 to 70 times as long as on Postgres. This was so grouped as Metabase sends it, with DISTINCT, and in
    a LEFT JOIN LATERAL. Over about 5 million rows they timed out, and a small query in another session waited up to 8
    seconds meanwhile. Over 3 million rows they now take 3 to 5 seconds, a little less than P...
Read more

Mason v0.20.0

Choose a tag to compare

@github-actions github-actions released this 14 Nov 12:30
f1e9e15

Mason no longer leaves a time limit behind on your database's pooled connections. Behind a pooler in transaction
mode, your application's queries could be cancelled after 10 seconds, for up to an hour. After a fatal error in the
synced data, Mason now exits cleanly, and the cause reaches mason logs. A failed sync event now says its code and
where it happened, so you can group failures and report them. Metabase's previews of a join to a table with JSON
fields are faster. A host can show its own icon and who is signed in on the web page.
Every project has something
to do first: its pin moves. A script or an alert that filters sync events by kind must change ("Before you upgrade").

Before you upgrade

  • Move the pin: version: 0.20.0 in mason/mason.yml. The 0.20.0 command and image refuse a project pinned to
    0.19.x, and say how to fix it. Before 1.0 they only run a project pinned to their own minor version, at their own
    patch or an older one. Set the pin and start 0.20.0 in the same step.

  • On a hosted replica, upgrade the replica before you deploy a project pinned to 0.20.0. mason deploy refuses a
    pin the replica would not run, before it sends anything, and names both versions. mason replica upgrade <org>/<name> is one way out, where the service offers it.

  • Two sync event kinds have new names. copied is now full_sync, and copy_failed is now full_sync_failed.
    You see them in these places:

    • mason logs --events and its --json;
    • mason status's Sync errors section, and recentErrors in its --json;
    • the Syncs filter on the web page.

    If a script or an alert filters on an old name, change it in the same step. Events recorded before the upgrade keep
    their old names. They still show in the full list, but an old copy_failed no longer counts as a failure.

What changes

  • A failed sync event says its code and where it happened. mason logs --events --json and /api/events give
    each failure a code and an origin.

    • The code is the error's SQLSTATE when your database failed it. It is the system's error when the network or the
      disk failed it (ECONNREFUSED, ENOSPC). Otherwise it is the error's class (conversion_error), or unknown.
    • The origin is a file and a line in Mason's code, such as src/duck/sync.ts:3637.

    Group failures by the code. Give the origin when you report one. An event that is not a failure has neither. The
    Sync events page shows both under a failure's message.

  • A query that failed inside Mason says where. A request-log row answered with XX000 has an origin in mason logs --requests --json and /api/requests. Every other row has none.

  • Both keys are new. Events and requests recorded before you upgrade are kept, and read with both keys null.

  • Metabase's preview of a join to a table with JSON fields is faster. This is the preview you see when you join a
    table in Metabase's query builder and show fields it unfolds from a JSON column. Over a 3 M-row table, the 2,000-row
    preview took about 5 s. It now takes about 0.6 s. Postgres still answers it faster, so there is more to do. The
    20-row preview is faster too, when the table you start from is the bigger one. Mason no longer reads the other
    table's whole join column before it returns the first rows. Both previews still return any rows of the join, as Postgres allows for a query
    with no ORDER BY.

  • Three shapes that Metabase and other BI tools send are faster:

    • A count grouped by to_char(created_at, 'YYYY-MM') over 4.5 M rows took 23 to 29 s. It now takes under a second,
      about a quarter of Postgres's time. A format with a quarter (Q), a padded month name or an ordinal (DDth) is
      still slow.
    • Metabase's week of the year over 12 M rows now takes about 1 s. Postgres takes about 4.5 s.
    • A query that returns many timestamptz values in a session in UTC sends them about three times as fast. 300,000
      rows of a DISTINCT ON took 10 s, and now take under 3 s, about Postgres's time. In a session in another time
      zone, these values are still slow to send.
  • The SQL support list says plainly how Mason prints a numeric column with no precision. Rails makes one from
    t.decimal. Mason prints every value of such a column with the most decimals any of its values has. So 12.5
    reads 12.50, and 0 reads 0.00. The values are right. Only their trailing zeros differ from Postgres's. To get
    the same text from Mason and Postgres, cast the column to numeric(p, s) or use round(x, s). A fix is planned.

  • For a host that serves the web page (mason serve):

    • The page can show your own icon in the browser's tab. Set MASON_UI_ICON to a data: URL of a PNG or SVG
      image in base64, up to 4096 bytes. The icon is part of the page, so the browser makes no extra request for it.
    • The page can show who is signed in. Set MASON_UI_ACCOUNT to JSON with a label and a url, such as
      {"label":"ada@example.com","url":"/account"}. The label shows at the foot of the sidebar, as a link to the URL.
      On a narrow window it is in the menu, with the nav.
    • A host view's link can be marked as a destructive action. Give it "danger": true, and the page draws it in
      red, like the Remove button on the Tables page. A link that is both primary and danger is refused, and the
      view says why. So is a danger that is not true or false.

    A value of MASON_UI_ICON or MASON_UI_ACCOUNT that Mason cannot use stops the start. The message says what is
    wrong and what a valid one looks like. Unset, the page is as before.

What was wrong and is now right

  • Mason no longer leaves a time limit on your database's pooled connections. When the web page checked whether a
    table could be kept current by replication, Mason set a 10-second time limit on its connection and never took it off.
    Behind a pooler in transaction mode (PgBouncer, or Supabase's default pooler URL), that connection is handed on to
    your application next. Your application's queries could then be cancelled after 10 seconds, for up to an hour.
    mason mcp did the same on Mason's own Postgres API, with each tool call's time limit. Now each limit ends with
    Mason's own query, whatever pooler sits in between.
  • After a fatal error in the synced data, Mason no longer aborts on its way out, and the cause reaches mason logs.
    A checkpoint that fails on a full disk is one such error. Mason then ends the run, so that it can start again.
    Sometimes the run aborted instead, with exit code 134 and terminate called after throwing an instance of 'Napi::Error'. The cause could then be missing from mason logs. Now the run waits up to 5 seconds for the synced
    data to close. Then it writes its log to the end and exits with code 1. If the synced data is still closing after 5
    seconds, the log says so, and the run exits anyway.
  • A polled table no longer costs an extra connection to your database now and then. On some machines, Mason's
    timer for the next poll woke a moment early. Mason then connected to your database, found no table due yet, and
    connected again a moment later to poll it. Now every timer waits until its time has come.
  • A table's columns dialog on the web page names the same choice as its row. On the Tables page, a table that
    leaves out columns which look like secrets says "All but 7 columns". Its columns dialog said "All columns, … but
    password, …", so it read like a different choice. Now the dialog's choice starts with the same words, "All but 7
    columns", and names the columns left out.

Known

  • A generated table can't yet hold some kinds of computed numeric column. These are refused:

    • a division, or an average;
    • a variance or a standard deviation;
    • sqrt, exp, ln, log and power;
    • a JSON number cast to numeric with no scale;
    • a value with more than 18 decimals;
    • a sum, a CASE, a GREATEST, a LEAST or a COALESCE of values with differing decimals;
    • a cast from a double.

    A domain and a composite type are refused too, and so is kind: live. Each refusal names the file and the key, and
    shows the SQL to write instead. It comes before anything is built. A run that fails, such as one with a value too
    big for its column, records Postgres's error and is tried again later.

  • mason status shows generated tables; the web page doesn't yet.

  • Pages of every LIMIT with no ORDER BY over a join may overlap: LIMIT n OFFSET m with no ORDER BY over a join has
    no stable order, so a tool paging one can see a row twice and miss another. Add an ORDER BY where the pages matter.

  • A replicated table whose re-read never completes can rejoin the stream with no catch-up, and miss what the stream
    passed over, as in every release since 0.8.0. It was found by reading the code and has not been reproduced.

  • The web page's errors are not recorded under MASON_UI_READ_ONLY=1. A read-only page cannot send them.

  • A table under sync: replication blocks ALTER TABLE … DROP COLUMN on your database. This happens on Postgres
    15 and later, when Mason made its own publication and the table's replica identity is its primary key (the
    default). That publication lists the table's columns, so Postgres refuses
    the drop with "publication … depends on column". To drop a column today, run these with Mason running:

    1. mason add <table> --sync "poll 5m". Mason takes the table out of its publication, and keeps its synced data.
    2. Drop the column on your database.
    3. mason add <table> --sync replication. Mason reads the table's columns again and syncs the table again, whole.

    Or run mason remove <table>, drop the column, then mason add <table>. This drops the table's synced data until
    it is synced again. A fix is planned. This is so on 0.19.0 too.

  • **A column added to a table on your database isn't synced until Mason resta...

Read more

Mason v0.19.0

Choose a tag to compare

@github-actions github-actions released this 13 Nov 22:56
9481bf3

Two fixes for tables kept current by replication come first. If the replication slot was dropped while Mason ran,
the stream could stay stopped for the rest of the run, and the tables served old rows with no error. Now the stream
always starts again by itself. A table being compacted is no longer called current while its newest changes wait.
A running Mason no longer connects to your database every 10 seconds while idle, so a serverless database can
suspend. Metabase's cards that group by a JSON field or bin a number are many times faster. The web page's Overview,
Tables, Users and Queries pages are simpler, and every time range is live.
Every project has something to do first:
its pin moves. Scripts that pass --remote or read mason status's human output must change ("Before you upgrade").

Before you upgrade

  • Move the pin: version: 0.19.0 in mason/mason.yml. The 0.19.0 command and image refuse a project pinned to
    0.18.x, and say how to fix it. Before 1.0 they only run a project pinned to their own minor version, at their own
    patch or an older one. Set the pin and start 0.19.0 in the same step.

  • On a hosted replica, upgrade the replica before you deploy a project pinned to 0.19.0. mason deploy refuses a
    pin the replica would not run, before it sends anything, and names both versions. mason replica upgrade <org>/<name> is one way out, where the service offers it.

  • Drop --remote from your scripts. mason init --remote now fails as an unknown option, with exit 2. A database
    on another machine needs nothing extra now ("What changes"). An older .mason/config.json's "remote" key is
    ignored.

  • A script that reads mason status's human output must change. The rows have new labels:

    • Database is now Source;
    • Source (the sources file) is now Sources file;
    • the Changes row is gone;
    • Disk is now Free disk;
    • Sessions is now Connections open;
    • the Mason row is now two rows, Version and Up;
    • a table's line says "not kept current" in lower case, where it said "NOT kept current".

    A script that searches for the upper-case words must change too. mason status --json is unchanged, except for a
    new users field. Read --json where you can.

  • mason init --json: the information lines moved from warnings to a new notes list. These are the line about
    a database on another machine, the line about your main database, and the tip about a password in the URL. mason status --json's warnings no longer has the first two either.

  • Handle the new state compacting if you read a replicated table's state from --json (mason status, mason wait). It is a table that is being compacted while its newest changes wait ("What was wrong and is now right").

  • mason logs --events says "Mason" where it said "(the sync)" for an event of no one table. --json still has
    table: null there. There is a new event kind, ui_error, which is a failure, so --failed lists it too. The new
    kinds polled and streamed appear only with the new --scheduled.

  • GET /api/tables/<table>/sync: polls[].cost is now an object, not a sentence. It is { rows, scanSeconds, everySeconds, busyPercent, basis, rowsPerSecond }, or null for a table that is not big. This is the endpoint the web
    page reads. No mason command prints it.

What changes

  • A database on another machine works with no extra step. mason init and mason start no longer stop for it.
    They say once: "Your database is on another machine. Mason will sync the tables you choose to this machine."
  • Your main database is information, not a warning. mason init, and mason start until the first sync is done,
    say that Mason only reads from it and keeps its load small, and that you can point Mason at a read replica instead.
    mason status's Source line ends with "your main database" or "a read replica".
  • Calmer words. A password in mason init's URL is now a tip: your shell's history holds it, so next time put
    the URL in .env. mason doctor no longer warns about CREATE in schema public when that is only the default of
    Postgres before 15. A grant someone made is still a warning. The note for a serverless database names everything
    that keeps it awake.
  • Mason records the web page's own errors. If a bug in the page stops it from working, you see a ui_error event
    in mason logs --events, on Sync events, and among mason status's recent errors. /metrics counts them in the new
    mason_ui_errors_total. The event says which page and file hit the error, never the error's message, which could
    quote your data. At most 10 are recorded a minute.
  • Scheduled syncs can be listed. Run mason logs --events --scheduled, or tick Scheduled syncs on the Sync events
    page. Each scheduled poll is a row. A replicated table gets at most one row a minute, only when changes arrived. They
    are kept for MASON_HISTORY_DAYS, like every event. At poll 1s that is up to 86,400 rows a day for one table.
  • The web page:
    • The Overview is shorter. A "mason status" box at the top says Ready, Under way or Failed, what that means for
      the synced data, and the Source. Getting started has three steps: Mason is running, Sync a table, and Create a
      user. It is done while at least one table is synced and one user exists. If you remove every table or every
      user, it comes back. Until it is done, it is all the Overview shows, under the "mason status" box once a table is
      added.
    • Two new cards: Errors and Query errors, for the last day. Each links to its list. They replace the lists of
      recent errors. The table of the last day's and week's queries is gone, and so is the Overview's Connect panel. The Postgres
      URL is on the Users page and in mason status.
    • Tables has two lists, Added and then Not added, each with its count. The Show filter is now one box, Failed
      only. A table's State says Syncing while you sync it from the page. Its column form offers All columns or Only
      these columns.
    • A big table's Sync choice says what a scheduled sync costs once: the rows, how long a scan takes, and for each
      interval how busy it keeps one connection to your database. A share of 10 % or more is shown as a warning.
    • Users lists the users first, in Active and Removed tabs. Then comes the box that adds one, then Connect. A new
      user's URL and password open in a dialog that closes only when you click "I have copied it", since the password is
      not shown again. The dialog says when a tool first connects with it. A line under the form says where roles are
      set.
    • Connect from anywhere is now Temporary cloud login, on the Users page. It opens what mason share opens, for
      a tool in the cloud that cannot reach your machine. It is not offered under mason serve.
    • Every time range on Queries, Usage and Sync events is live, and updates by itself. Now is renamed Last 15
      mins, and Refresh is gone.
    • Queries is shorter. The time range is its only window, and the User filter is a list of your users.
    • Smaller changes. Each box that opens beside a button has an X to close it. An empty list is one quiet line.
      Sync events says each kind in words, such as Scheduled sync or Live sync. Connects from is a dropdown again.
  • Faster queries:
    • A JSON field whose path your tool sends as a parameter is read as fast as one written in the query. Metabase
      sends JSON paths this way. A card grouped by a JSON field took 11 to 16 times as long as on Postgres. It now takes
      two to three times Postgres's time.
    • A chart that bins a number is now faster than on Postgres. Metabase bins a number column for its X-rays and
      for binned fields. Over millions of rows, such a card took 16 to 25 times as long as on Postgres. Now it takes
      about half Postgres's time. This holds for columns of numeric(p, s) and the integer types. The answers are the same as
      before, to the last digit.

What was wrong and is now right

  • A running Mason no longer connects to your database every 10 seconds while idle. It read the database's list of
    tables on a timer, whatever each table's sync. So a serverless database (Neon, Aurora Serverless) never suspended,
    and every database logged about 8,600 connections a day. Now Mason reads the list only when you ask:

    • mason status, wait and doctor, and the open web page, reuse a read under a minute old. They never wait more
      than 2 seconds for a new one.
    • mason tables, add and remove, and the page's Columns and Sync choices, read the list as it is now. If they
      cannot (a migration holds a lock, or the connection went silent), they say so and change nothing. A database with
      many tables can take a minute to read, and mason add says it is still going.
    • With every table sync: off, a run nobody asks opens no connection after its first sync.

    The read is one short read-only transaction, so it leaves no setting behind on a pooled connection. It works behind
    a connection pooler in transaction or session mode (PgBouncer, Supavisor, Neon's proxy). It does not work in
    statement mode, where Mason's first sync already fails.

  • Tables under sync: replication no longer stop being kept current after the replication slot is dropped. The
    stream waited a moment before it started again. Another sync asked for in that moment could cancel the restart. The
    tables then served old rows for the rest of the run, with no error. Now the stream always starts again.

  • A replicated table is no longer called current while Mason compacts it. During a compaction, its newest changes
    wait until it ends, and queries read the rows as they were. Before, mason status, mason wait and the web page
    said the table was current. Now its line says it is being compacted and when it was last synced. If the compaction
    takes...

Read more

Mason v0.18.0

Choose a tag to compare

@github-actions github-actions released this 13 Nov 10:37
f5c4721

A replica that switched to another publication could lose rows or stop keeping tables current. When the sources
file's publication: changed between two starts, a write that was open during the switch could be skipped: on
Postgres 18 its rows never reached the synced tables, and on Postgres 16 the stream stopped for good. Mason now makes a
new replication slot at such a switch, so no row is lost and the stream goes on. It costs one full read of each
replicated table on your database, once. If you switched publication with an earlier Mason, check your replicated
tables ("What was wrong and is now right"). The web page now speaks plain English. Its Tables page lets you choose
the columns an added table syncs, keeps table names readable at laptop widths, and says why a table has no size.
mason deploy now works in a folder that holds only mason/, with no mason init. Two metrics and a Helm key were
renamed.
Every project has something to do first: its pin moves ("Before you upgrade").

Before you upgrade

  • Move the pin: version: 0.18.0 in mason/mason.yml. The 0.18.0 command and image refuse a project pinned to
    0.17.x, and say how to fix it. Before 1.0 they only run a project pinned to their own minor version, at their own
    patch or an older one. Set the pin and start 0.18.0 in the same step.
  • On a hosted replica, upgrade the replica before you deploy a project pinned to 0.18.0. mason deploy refuses a
    pin the replica would not run, before it sends anything, and names both versions. mason replica upgrade <org>/<name> is one way out, where the service offers it.
  • Rename two metrics and two phase values in your dashboards and alerts. On GET /metrics, mason_copy_bytes is
    now mason_synced_bytes and mason_copy_rows is now mason_synced_rows. The phase label of mason_tables says
    synced where it said copied, and syncing where it said copying. waiting and failed are unchanged. A query
    or alert on an old name finds no series after the upgrade, so it reads as absent, not as zero. Rename it in the same
    step. The tables counts in /healthz keep their keys.
  • Helm: rename memory.copy to memory.engine in your values file before helm upgrade. A values file that still
    sets memory.copy fails to render, naming memory.engine, and nothing is installed. It is never run with the
    default in its place.
  • EXPLAIN (FORMAT JSON) now says syncing again in full where it said copying again in Kept Current By, for a
    table read again whole at its interval. mason mcp's kept_current_by says the same. A tool that matched the old
    words must be updated to match the new ones.
  • A heavy statement may now fail where it used to finish. The limit on what statements may spill to disk (half of
    the free disk under .mason/, or MASON_SPILL_LIMIT) is now enforced from the start. A statement, or several at
    once, that spilled past it now fails with 53400. To allow more, set MASON_SPILL_LIMIT (for example 40GB), or ask
    for less at once.
  • An application that decodes Mason's rows by type numbers must look up enums, citext and composites on Mason.
    Mason now sends its own number for such a type, not your database's (see "What was wrong and is now right"). A
    number learned on your database no longer matches.
  • If you switched publication with an earlier Mason, check your replicated tables after upgrading ("What was wrong
    and is now right"). Nothing on 0.18.0 repairs a table that already missed rows.

What changes

  • mason deploy works in any folder that holds a project's mason/, with no mason init. When there is no
    .mason/ in the folder or above it, the current folder's mason/ is the project (or the folder that mason -C <folder> deploy names). It needs a mason.yml or a YAML file in mason/sources/. Mason never looks for a mason/
    elsewhere. Only that mason/ is sent, and the output says which folder it was (folder in --json). --dry-run
    finds it the same way. So a clone in CI, or a folder you copied the files into, deploys as it is. A project made with
    mason init deploys as before.

  • The web page speaks plain English. It uses short, whole sentences, and says "query" where it said "statement".
    It tells you how to do things on the page (Sync now, a table's Sync button, Enable, Users). It names a mason
    command or a file only when the page can't do it. mason status and the other commands keep their own words.

  • On the web page's Tables, an added table says which columns it syncs, and you can change them there. Its note
    says "All columns", "5 of 12 columns" or "All but 1 column". Click it to tick the columns to sync, as `mason add

    --columns …` does. Before you save, the form tells you the cost: the table is synced again in full from your database, and you can still query its synced data until that ends. Saving the same columns changes nothing. If the form can't show an entry as ticks (an `exclude:` written by hand), it describes it and says that saving replaces it. A read-only page shows the columns but can't change them. `mason status --json` adds `columnCount`, and for an added table `entry` and `everyColumnLeavesOut`.
  • A synced table with no size says why, instead of "—". On the web page's Tables, a table whose rows are all still
    in the write-ahead log shows < 3 MiB (less than the log's size). A table not measured yet since its sync, since data
    was last written to disk, or since Mason started shows Measuring…. A table whose first sync is not done shows
    nothing. Hover over it to see why. mason usage lists these tables the same way under the sizes, and its --json
    adds sizedAt, walBytes and unsized.

  • "Catching up" is now "Finishing sync". When a table's first sync is done and Mason is getting the changes made
    during it, the Tables page says "Finishing sync", with "Getting the changes made since it was last synced. You can
    query it meanwhile." under it. mason status and mason wait say that sentence too. A streamed table's line says,
    for example, "getting the changes made since 09:00 UTC: reading the changes". The catching-up state in --json is
    unchanged.

  • The Overview's "Mason" card shows only what you can use: Database (its name, host, Postgres version, and whether
    it is your main database or a read replica), Mason (version and uptime), Disk (free space beside the space the synced
    data uses) and Connected tools. Restarts show only when there were some. The sources file, the sync settings and the
    page's socket are gone from it; the Tables page shows each table's sync. A database on another machine is now a
    quiet line, not a warning, and a main database is no longer a warning on the page. How text sorts shows only when it
    differs from your database.

  • The web page no longer shows a psql line. Each place that showed one keeps the URL, with Copy: paste it after
    psql. mason status and the psql field in --json are unchanged.

  • The card for a user just added shows URL, User and Password only, each with Copy. Host, Port, Database and SSL
    mode are on the Users page's Connect card above it. mason user add --json is unchanged.

  • On the web page's Users, Connect comes first, and the cursor starts in the new user's Name (not on a touch
    screen, a read-only page, or when you have already clicked into something else).

  • Every example in a box on the web page starts with "e.g." (e.g. orders, e.g. 2h, e.g. metabase), so an
    example is never taken for a value already filled in.

  • What was wrong and is now right

    • A switch of publication could lose rows or stop the stream. The switch is a change of the sources file's
      publication: (or PG_PUBLICATION) between two starts: from Mason's own publication to one you named, from one you
      named back to Mason's own, or from one named publication to another. Mason kept using its replication slot with the
      new publication. A write transaction that was open while the new publication was created, and committed after, was
      then skipped:

      • on Postgres 18, its rows never reached the synced tables. Your database's log had only a warning: The publication does not exist at this point in the WAL;
      • on Postgres 16, the stream failed at every start (publication "…" does not exist) and never moved again. The
        replicated tables stopped changing, and the slot held your database's WAL.

      Now a start that sees a switch drops the slot (and Mason's own publication, when that was the one read), checks that
      it is gone, and makes a new slot. It then catches each table whose sync is replication up by one poll. That poll
      is one full read of each of those tables on your database, once.
      mason start says so when it makes the switch.

      This does not repair a table that already missed rows. If you switched publication with an earlier Mason while
      your application was writing, find out like this:

      • On Postgres 18, search your database's log, from the time of the switch, for The publication does not exist at this point in the WAL. No such line means no rows were skipped.
      • On Postgres 16, the stream has been failing since the switch. mason status and mason logs show the stream
        failing with publication "…" does not exist, and the replicated tables are behind.

      To fix a table, sync it again whole. With Mason running, run mason remove <table>. Wait until mason logs shows
      dropped <table>'s synced data (the table named with its schema). Then run mason add <table> --sync replication.
      The table is not served in between. If you add it back while Mason is stopped, or before that line appears, the add
      cancels the remove and nothing is synced again. On Postgres 16, do this for every table you replicate: once none is
      left, the remove drops the stuck slot, and the first add makes...

Read more

Mason v0.17.1

Choose a tag to compare

@github-actions github-actions released this 12 Nov 06:55
33769b8

Mason now compacts each synced table by itself, so v0.17.0's disk growth and slower reads are bounded, and the
workaround in v0.17.0's notes (mason remove, wait, mason add) is no longer needed. A fix comes with it: in v0.17.0
and earlier, a table kept current by the stream (sync: replication) could be left with rows twice when it was synced
whole and then caught up while more than about 110,000 of its keys changed: its first sync, the remove-and-add that
v0.17.0's notes gave to compact it, a switch to sync: replication, or the replication slot made again. A plain
restart with the slot kept is not affected. New duplicates no longer happen, but rows already duplicated stay until the table is synced again whole: if
you replicate a busy table, check it ("What was wrong and is now right" has the query and the way out). And a generated
table over a citext or enum column could be refused or given the wrong type.
A project pinned to 0.17.0 runs on
0.17.1 as it is ("Before you upgrade").

Before you upgrade

  • The pin needs no change. Before 1.0 the command and the image run a project pinned to their own minor version, at
    their own patch or an older one, so 0.17.1 runs a project pinned version: 0.17.0. Moving it to version: 0.17.1 in
    mason/mason.yml is optional; once you do, a 0.17.0 command or image refuses the project, so move it only once
    everyone and everything that runs the project has 0.17.1. A project mason init makes on 0.17.1 is pinned 0.17.1.
  • On a hosted replica, upgrade the replica first, then move the pin. A replica on 0.17.0 runs a project pinned
    0.17.0, but not one pinned 0.17.1: mason deploy refuses that one before it sends, naming both versions, with
    mason replica upgrade <org>/<name> as a way out where the service offers it. A project pinned 0.17.0 deploys to a
    replica on either version.
  • If you replicate a table that is busy, check it for duplicate rows after upgrading ("What was wrong and is now
    right"). Nothing on 0.17.1 removes rows already duplicated.
  • Leave room on disk for compaction. A table is compacted only with room on disk for its rewrite and a margin (a
    quarter of the table, at least 64 MiB); without it the table waits, said on its line in mason status.
  • If anything reads mason status --json, each synced table now has tables[].copy.compaction ("What changes"),
    and mason logs --events has a new event, compact_failed.

What changes

  • Mason compacts a synced table by itself. It reads each synced table's space a minute after it starts and every 10
    minutes after, and when a table's deleted rows and small blocks make up half of its space (and at least 4 MiB), it
    rewrites that table from the synced data alone, never reading your database.
    • The table is read and served the whole time. While a polled table is compacted, its polls wait and a mason sync
      of it is refused; a streamed table's mason sync is never refused, and its changes are written to disk beside it, in the synced data, and applied right after (a
      stop or a crash meanwhile catches the table up by one poll at the next start). It takes about as long as reading the
      table: measured, about 2 s for 6 million narrow rows and 5 s for 650,000 rows of 1 KB on a laptop. One table is
      compacted at a time, and never beside a first sync.
    • It works within MASON_DUCKDB_MEMORY, a piece at a time; one that runs out of memory fails alone, the table served
      and kept current as it was, and is tried again later.
    • It starts only with room on disk for the rewrite and a margin (a quarter of the table, at least 64 MiB), checked
      again before each piece; without it the table waits, and its line says how much room it needs and how much is free.
      A disk that runs short partway through fails the compaction (a compact_failed event), which is tried again later.
    • A table with a column named rowid is never compacted, and its line says so.
    • mason status and the web page say, on a table's line, that its compaction failed and when it is tried again; or
      its share of deleted rows, from 10 % up, and why it is not compacted when it waits; or when it was last compacted.
      mason status --json has it under tables[].copy.compaction (deadShare, wasteShare, checkedAt, at,
      seconds, error, retryAt, waiting). A failure is a compact_failed event in mason logs --events.
    • The file under .mason/ does not shrink: later writes reuse the space a compaction gives back.

What was wrong and is now right

  • The synced data no longer grows without bound on a busy server. Since 0.17.0 a table kept current by the poll or
    the stream kept the space of every row it updated or deleted, and every table written to between two of the synced
    data's checkpoints kept a block of at least 256 KiB, so the disk under .mason/ grew and reads slowed. With
    compaction ("What changes"), measured: a table whose rows are updated over and over (5 % of them a round) stays within
    about 2.5 times the size and the read time of a just-compacted one, where without it it kept growing, to 5.4 times
    its size and 4.6 times its read time after 60 rounds; a table appended to a little at every checkpoint stays within
    about twice its size. The workaround in 0.17.0's notes is no longer needed.

  • A streamed table's catch-up could leave rows twice. A table kept current by the stream (sync: replication) is
    caught up by one poll after it is read whole, and the stream's changes held meanwhile are applied after that poll;
    when they changed more than about 110,000 of its keys, the rows already there for those keys were not deleted before
    they were written again, so the table held them twice (seen: a 2 M-row table left with 935,331 keys twice) and counts
    and sums over it were wrong. A table was exposed when it was caught up so while your database wrote to it heavily:

    • its first sync, or a sync of it again whole — including the mason remove / mason add that v0.17.0's notes
      gave to compact a table
      ;
    • a change of its sync: to replication;
    • the replication slot made again, also after max_slot_wal_keep_size invalidated it;
    • the stream no longer carrying one of its columns;
    • a stop or a crash during any of these, which catches it up at the next start.

    A plain restart with the slot kept is not affected: the stream reads what Mason missed from the slot, in batches too
    small for this. Mason being down matters only when it fell during one of the above. Those changes are now
    applied once each, whatever their number.

    This does not repair a table already affected. A duplicated row stays until its key changes again on your
    database (the stream then replaces every row of that key) or the table is synced again whole; a compaction keeps the
    rows as they are, and mason sync does not read a streamed table. To check a table, run over Mason's Postgres API,
    with the table's primary key in place of id:

    SELECT id, count(*) FROM orders GROUP BY id HAVING count(*) > 1 LIMIT 10;

    No row means no duplicate. If there are any, sync the table again whole: with Mason running, run `mason remove

    `, wait until `mason logs` shows `dropped
    's synced data` (the table named with its schema), then run `mason add
    --sync replication`. The table is not served in between. If you add it back while Mason is stopped, or before that line appears, the add cancels the remove and nothing is synced again. If it is the only table you replicate, the remove drops the replication slot on your database and the add makes a new one.
  • A generated table over a citext or enum column could be given the wrong type, and its statements refused.
    Mason named such a column's type by the number your database gives the type, read in Mason's own catalog, where the
    same number can be another type: a synced table's row type. The generated table's column was then declared as that
    row type, and a statement Postgres answers, such as SELECT id, name FROM kinds WHERE name = 'ALICE', was refused
    with 0A000 input of anonymous composite types is not implemented. Whether it happened depended on your database's
    type numbers, so it could appear after any restart. The column is now always declared as your database's type
    (citext, the enum's name), and the table is built again at the next start.

  • Known

    • A generated table can't yet hold a numeric that divides (an average is one); is a variance or a standard deviation, which Postgres computes exactly at decimals it picks per value, rounded, cast to numeric(p,s) or not; is a sqrt, exp, ln, log or power, whose decimals Postgres picks per value, rounded, cast to numeric(p,s) or not, whatever the decimals of what it is computed from; is a JSON number's text cast to numeric with no scale (cast to numeric(p,s), it is stored); has more than 18 decimals, at the end or on the way; sums values whose decimals the rows decide; is a CASE, GREATEST, LEAST or COALESCE of values with differing decimals; or is cast from a double; nor a domain or a composite, and kind: live is refused. Each is refused by file and key before anything is built, said as Postgres types the value (a sqrt or an average of a double is a double, so cast to numeric it is a cast from a double), wherever it sits in the column (under an operator, a coalesce, a round, a cast or an aggregate, or in a scalar subquery, a WITH query, a subquery in FROM or a UNION's arm), with the way out your own SQL with the cast moved and the aggregate kept (sum(sqrt(x)::double precision)), or the WITH query, subquery or UNION the cast belongs in, given whole. Each of these keeps its reason read back from its text (::text, ::varchar, || '', a jsonb's ->>, a WITH column), cast to numeric(p,s) or not, so a typmod over a variance's text never stores an approxim...
Read more

Mason v0.17.0

Choose a tag to compare

@github-actions github-actions released this 12 Nov 01:06
3142780

Two fixes a real deployment met come first. Several first syncs on a small machine could make the synced data
unusable, so that every table's sync failed and every restart failed again: a first sync no longer can, and a run
already stuck that way recovers when it starts on this release, with nothing lost. A run whose synced data met a fatal
error (a full disk, an I/O error) stayed "running" while it answered nothing, until it was restarted by hand: it now
ends as a crash does, its cause said, and is started again. The web page's Tables now choose how each table is kept
current: Off, a poll, or Live (logical replication) where your database is ready for it. mason wait waits for a
table's catch-up poll, mason share opens with the URL to paste, and the card of a user just added has a Copy on every
field.
Every project has something to do first: its pin moves. The first fix has a cost on disk until a future release
compacts tables: watch the disk under .mason/ ("Before you upgrade").

Before you upgrade

  • Move the pin: version: 0.17.0 in mason/mason.yml. A project pinned to 0.16.0 is refused by the 0.17.0 command
    and image, with the way out named: before 1.0 they run a project pinned to their own minor version only, at their own
    patch or an older one. Set the pin and start 0.17.0 in the same step.
  • On a hosted replica, upgrade the replica before you deploy a project pinned to 0.17.0. mason deploy asks the
    replica's Mason version first and refuses a project whose pin the replica would not run, naming both versions and the
    ways out; mason replica upgrade <org>/<name> is one of them, where the service offers it.
  • Watch the disk under .mason/. The synced data now grows on disk where it used to be compacted, and tables kept
    current by the poll or the stream get slower to read on a busy server ("What was wrong and is now right" gives the
    numbers). To compact a table, sync it again whole. With Mason running, run mason remove <table>. Wait until mason logs shows dropped <table>'s synced data, the table named with its schema (dropped public.orders's synced data), then run mason add <table>. The table is not served in between. If you
    add it back while Mason is stopped, or before that line appears, the add cancels the remove and nothing is compacted.
    The file does not shrink, but later writes reuse the space. mason sync does not compact a table.
  • If anything reads tables[].sync.state from --json (mason status, mason wait), handle the new state
    catching-up: a table served and being caught up by one poll, which was syncing before ("What changes").
  • mason add <table> --sync replication while Mason is up now reads your database's checks for logical replication
    first
    (as the web page's Live does, below), in a run that already replicates other tables too, and refuses the
    change, writing nothing, when one fails, naming the failing check (replication role, for instance). The web page's
    Sync choice shows each failing check with the SQL or setting that fixes it. A script
    that switches tables to replication may now be refused at the add, where the edit used to be written and the table
    then failed in the run.

New

  • The web page's Tables choose how each table is kept current: every added table says its sync on its line (Off,
    Every 5 minutes, Live (replication)), and where the page may edit the sources file it opens a choice — Off (the
    default), a poll every 5 minutes, 15 minutes, hour or day, each saying it scans the whole table on your database at
    that interval (a table over a million rows says what each poll costs and asks before saving), or Live, logical
    replication. Live can be chosen only when your database is ready for it: otherwise it is shown greyed out with each
    check that fails — no primary key, wal_level, a user without REPLICATION, no free replication slot or WAL sender, a
    read replica before Postgres 16, the table missing from your publication — and the SQL or setting that fixes it. The
    checks read your database's settings and catalog only, never a row, when the choice opens and again when it is saved,
    so a database that changed in between is refused. Saving writes the table's sync: line in the sources file, as
    mason add <table> --sync … does. Where the project's files are read-only, the page shows the sync without a choice.

What changes

  • mason wait now waits for a table's catch-up poll (after its first sync, a sync of it again whole, or a change
    of its sync: to replication). A first sync reads the table as of its start, then catches it up by one poll for
    what changed during the read, a sync: off table too; on a large table that poll is a scan of its own. mason wait
    said Ready while it ran, so a tool started on Ready read data as of the read's start, and the table's line said "NOT
    kept current" with a yellow bar. Now the table's line says it is being caught up by one poll, its bar at work, and
    mason wait is ready once that poll is over (a script waiting on it may wait that much longer); mason status, the
    web page and EXPLAIN say the same. In --json a table in its catch-up has sync.state catching-up (it was
    syncing), and readiness lists it under waiting.
  • mason share opens with the URL to paste: its first line is now Try mason with your favorite postgres client: postgres://<user>:<password>@<host>:<port>/<database>?sslmode=require, the share's own login and address, so a
    client that takes a connection string connects from that one line. The host, port, database, user and password rows
    below it stay, for a tool that asks for each field, and mason share --json carries the same URL as share.url.
    The password is still made for this run alone and kept nowhere.
  • The web page's card for a user just added has a Copy on every field: URL, User, Password, Database, Host, Port
    and SSL mode, so each can be pasted into a BI tool's connection form one by one. The card no longer shows a psql
    line: paste the URL after psql. mason user add --json keeps its psql field.
  • mason user add, beside a mason start run on this machine's own address, now ends its tips with the way to a
    tool outside this machine
    : mason share opens an address on the internet, with a login and password of its own
    (not the new user's), for as long as it runs. It is not said beside mason serve, nor with --replica, nor while
    Mason is not running. --json is unchanged.

What was wrong and is now right

  • A first sync of a table of wide values on a small machine could stop all of Mason. After the sync read the
    table, the synced data's own checkpoint could fail when it reorganised that table, with failed to offload data block … or failed to pin block … in the log. That made the synced data unusable: every other table's sync failed
    with database has been invalidated, and every restart failed again in the same place. Mason now never reorganises
    a table's storage that way, so the sync finishes and the other tables sync. A run already stuck like this recovers
    when it starts on this release; nothing is lost.

    What this costs until a future release compacts tables: the synced data grows on disk, and tables kept current
    by the poll or the stream get slower to read on a busy server.

    • A table keeps the space of every row the poll or the stream updates or deletes. A table that had every row
      updated three times over took 2.1 times the disk and scanned in 1.7 times the time.
    • Every table written to between two of the synced data's checkpoints keeps a separate block of at least 256 KiB.
      A checkpoint comes about every 16 MiB written and at every stop. In measurements, small appends with 2,000
      checkpoints took a 1 M-row table from 31 MiB to 536 MiB, and its count from 1.2 ms to 2.8 ms. 25 restarts grew a
      project of 8 polled tables by 50 MiB, where it grew by 1 MiB before.

    Meanwhile, watch the disk under .mason/. To compact a table, sync it again whole. With Mason running, run mason remove <table>. Wait until mason logs shows dropped <table>'s synced data, the table named with its schema (dropped public.orders's synced data), then run mason add <table>. The
    table is not served in between. If you add it back while Mason is stopped, or before that line appears, the add
    cancels the remove and nothing is compacted. The file does not shrink, but later writes reuse the space. mason sync
    does not compact a table. EXPLAIN's "about N rows" also counts rows updated or deleted since the table's last sync
    whole.

  • A run whose synced data met a fatal error stayed "running" and answered nothing. When a write to the synced data
    fails in a way it cannot recover from inside the run (a checkpoint on a full disk, an I/O error), every statement
    was refused with XX000 … database has been invalidated, mason status still said running, the request log
    stopped, and nothing restarted it. Now the run ends as a crash does, with the cause and the way out in its log:
    sessions are told 57P02 and to reconnect, mason start's supervisor starts it again (mason status shows the
    restart and its reason), and mason serve exits 1, /healthz saying failed until it does, for the platform to
    start it again. Starting again opens the synced data again; nothing synced is lost. When the restart meets the same
    cause (the disk still full), the supervisor says it is the same as the last restart, and gives up after five
    restarts in ten minutes as for any crash; a full disk that stops the synced data from opening is said as the disk's,
    never with mason destroy as the way out.

  • A table of large values could fail its first sync under a small MASON_DUCKDB_MEMORY, now and then, with
    Failed to commit: could not allocate block of size … in mason status and the log, and wa...

Read more

Mason v0.16.0

Choose a tag to compare

@github-actions github-actions released this 11 Nov 08:47
a7cfb31

A preview of a join (a LIMIT with no ORDER BY) now stops as soon as it has its rows, where it sorted every joined
row: milliseconds where it took seconds to tens of seconds over a table of a hundred million rows. A page of an outer
join sorted by its preserved table's columns sorts that table's first rows alone. A statement that takes a second or
more has its plan recorded with its row of the request log (mason logs --requests --id <request id>). Statement times
are whole milliseconds everywhere, and the synced data's size counts its write-ahead log.
Every project has something
to do first: its pin moves. A tool that pages through a join with no ORDER BY may now see a row twice and miss another.

Before you upgrade

  • Move the pin: version: 0.16.0 in mason/mason.yml. A project pinned to 0.15.0 is refused by the 0.16.0 command
    and image, with the way out named: before 1.0 they run a project pinned to their own minor version only, at their own
    patch or an older one. Set the pin and start 0.16.0 in the same step.
  • On a hosted replica, upgrade the replica before you deploy a project pinned to 0.16.0. mason deploy asks the
    replica's Mason version first and refuses a project whose pin the replica would not run, naming both versions and the
    ways out; mason replica upgrade <org>/<name> is one of them, where the service offers it.
  • If a tool pages through a join with LIMIT n OFFSET m and no ORDER BY, and the pages matter, give it an ORDER BY.
    Every such statement may now give overlapping pages ("What changes").

New

  • A statement that takes a second or more has its plan recorded with its row of the request log, so why it was
    slow can be read afterwards without rebuilding the plan by hand. Once the statement has ended and its answer was
    sent, Mason runs a plain EXPLAIN of it in the background: planning only, nothing run a second time. The plan says
    the tables it read and how fresh each was, the SQL sent to the synced data, and its operators with their estimated
    rows. Read it in mason logs --requests --json (plan, planTruncated), in a row's details on Observability ›
    Queries, or whole with the new mason logs --requests --id <request id>. A plan is at most 8 KiB and says when it
    was cut. One plan is recorded at a time: a slow statement that ends while another's is being recorded gets none, and
    the log says so. A statement bound with parameters has its operators recorded without their details, which could
    show the values it was bound with. A cancelled statement has no plan. Neither does one refused before it reached
    the synced data, or refused as SQL Mason does not support. It is kept as the request log is (MASON_HISTORY_DAYS).

What changes

  • A preview of a join with a LIMIT and no ORDER BY stops as soon as it has its rows
    (SELECT a.*, b.* FROM a LEFT JOIN b ON a.id = b.a_id LIMIT 100, what a BI tool sends to show a table). Before,
    where each join matched on a primary key, Mason sorted every row of the join to keep its pages stable. That took
    seconds to tens of seconds over a table of a hundred million rows; now it takes milliseconds. The pages of every
    LIMIT with no ORDER BY over a join may now overlap
    : LIMIT n OFFSET m can show a row twice and skip another, as
    SQL and Postgres allow. Add an ORDER BY where the pages matter.
  • A page of an outer join sorted by its preserved table's columns limits that table first: a LEFT JOIN sorted by
    the left table (SELECT … FROM a LEFT JOIN b ON a.id = b.a_id ORDER BY a.created_at DESC LIMIT 100, a filter on a
    allowed), and likewise a RIGHT JOIN sorted by its right table or a chain of LEFT JOINs sorted by its first table.
    Mason now sorts only that table's first rows, then joins them, instead of joining and sorting every row of the other
    side. Over 25 thousand × 30 million rows it took about a fifth of the time (0.9 s to 0.2 s, warm). It still reads
    the other table's join column whole, so on a cold disk a table of a hundred million rows still takes seconds (about
    20 s at 150 million rows, an estimate). Unchanged: a sort or a filter that reads another table, an INNER, CROSS or
    FULL join on the way, a DISTINCT ON, a subquery in the WHERE, and a LIMIT or OFFSET bound as a parameter.
  • How long a statement took, and the average, p50 and p95 of those times, are always whole milliseconds with
    thousands separators (19,800 ms, 810 ms, 27 ms), so a column of them can be compared at a glance. Before, the
    unit changed at a second and at a minute (19.8 s, 2 min 5 s). This applies on the web page (Observability ›
    Queries' Duration, the Overview's p95 card, statements table and query errors, Usage's cards, hours, p95 chart
    read-out and slowest statements) and in the text of mason logs --requests, mason status and mason usage.
    --json still gives numbers. A chart's axis, a statement still running, a sync's time, an interval and an age keep
    their form. On Observability › Queries, the Statement cell now has its own + beside the statement. It opens and
    closes the same row as the Query id's +.

What was wrong and is now right

  • The synced data's size left out its write-ahead log: the Overview's "The synced data's file" card, mason status, mason usage and mason_copy_bytes read 12 KiB for a file whose log held 1.3 MB. They now count the log.
  • A small synced table's size could never be shown, and the page said it was coming at the next measurement. A
    table's size is measured once its rows are written from the write-ahead log into the file, which happens on its
    own, for instance when the log passes 16 MiB or Mason stops, so a small table's can wait for the life of a run.
    Until then the Tables view shows —, whose tooltip says why; mason usage lists no size for the table, and says
    why under the sizes when another table has one (and in mason usage --help).

Known

  • A generated table can't yet hold a numeric that divides (an average is one); is a variance or a standard deviation, which Postgres computes exactly at decimals it picks per value, rounded, cast to numeric(p,s) or not; is a sqrt, exp, ln, log or power, whose decimals Postgres picks per value, rounded, cast to numeric(p,s) or not, whatever the decimals of what it is computed from; is a JSON number's text cast to numeric with no scale (cast to numeric(p,s), it is stored); has more than 18 decimals, at the end or on the way; sums values whose decimals the rows decide; is a CASE, GREATEST, LEAST or COALESCE of values with differing decimals; or is cast from a double; nor a domain or a composite, and kind: live is refused. Each is refused by file and key before anything is built, said as Postgres types the value (a sqrt or an average of a double is a double, so cast to numeric it is a cast from a double), wherever it sits in the column (under an operator, a coalesce, a round, a cast or an aggregate, or in a scalar subquery, a WITH query, a subquery in FROM or a UNION's arm), with the way out your own SQL with the cast moved and the aggregate kept (sum(sqrt(x)::double precision)), or the WITH query, subquery or UNION the cast belongs in, given whole. Each of these keeps its reason read back from its text (::text, ::varchar, || '', a jsonb's ->>, a WITH column), cast to numeric(p,s) or not, so a typmod over a variance's text never stores an approximation; only a stored column's or a JSON number's text read as a numeric is said as a text: a JSON number's, a stored double's or any other. A run that fails (a value past 38 digits, bigint out of range, division by zero) records Postgres's error, SQLSTATE and DETAIL (numeric field overflow says the precision and scale it must fit, on every path), and is tried again after its back-off.
  • mason status shows generated tables; the web page doesn't yet.
  • Pages of every LIMIT with no ORDER BY over a join may overlap: LIMIT n OFFSET m with no ORDER BY over a join has
    no stable order, so a tool paging one can see a row twice and miss another. Add an ORDER BY where the pages matter.
  • A replicated table whose re-read never completes can rejoin the stream with no catch-up, and miss what the stream
    passed over, as in 0.8.0 to 0.15.0. It was found by reading the code and has not been reproduced.

Mason v0.15.0

Choose a tag to compare

@github-actions github-actions released this 10 Nov 20:37
546dd37

mason add now leaves out a column that looks like a secret however its name is written (passwordHash,
API_TOKEN), where it recognised only lowercase snake_case. A mason serve that is alive but no longer answering now
reads as hung, where it read as not running. mason serve's web page can show a site's own facts in its look (host
views and an Overview card), says and sets where each user connects from, and a read-only page can still manage its
users (MASON_UI_READ_ONLY_ALLOW=users).
Every project has something to do first: its pin moves. A project that
syncs a table with a camelCase or UPPER_SNAKE secret column should check its entries.

Before you upgrade

  • Move the pin: version: 0.15.0 in mason/mason.yml. A project pinned to 0.14.0 is refused by the 0.15.0 command
    and image, with the way out named: before 1.0 they run a project pinned to their own minor version only, at their own
    patch or an older one. Set the pin and start 0.15.0 in the same step.
  • On a hosted replica, upgrade the replica before you deploy a project pinned to 0.15.0. mason deploy asks the
    replica's Mason version first and refuses a project whose pin the replica would not run, naming both versions and the
    ways out; mason replica upgrade <org>/<name> is one of them, where the service offers it.
  • Check your synced tables for a secret column mason add did not recognise. Before 0.15.0, an every-column
    mason add <table> synced a column such as passwordHash, apiKey, accessToken or API_TOKEN with no
    exclude: and no warning, and any user who may read the table can read it. The upgrade does not change an entry
    already in your sources file. Run mason tables <table> for each table you sync: a column marked "looks like a
    secret" is synced unless the table's entry lists it under exclude: or leaves it out of columns:. Add it to
    exclude:, or run mason add <table> again, which rewrites the entry with the column under exclude: (the table is
    then synced again whole).
  • Restart a mason serve once after upgrading. A run now records when its process started, and that record is how
    a hung run is known. A run started by an earlier version has none and is judged as before: a mason serve already
    running when you upgrade still reads as not running if it hangs, until it is restarted.

New

  • A site that embeds mason serve's web page can show its own facts inside it, in the page's look. An entry of
    MASON_UI_NAV_LINKS may be {"label": "Billing", "view": "host/status.json"}, a view in the sidebar at #host/<n>
    (n its place in the list), and MASON_UI_OVERVIEW_CARD=host/card.json adds one card to the Overview. Each is JSON
    the page reads from a path relative to its own address (so the site's proxy answers it: no scheme, no //, no leading
    /, no ..), at most 256 KiB and within 10 seconds; every string is shown as text and every link held to
    MASON_UI_BACK_URL's rules. An answer the page cannot read is shown as the view's failure, and never holds up the
    rest of the page. docs/WEB-UI.md "Host views" has the shape. A path that breaks these rules refuses the start. Unset,
    the page is unchanged; mason start refuses MASON_UI_OVERVIEW_CARD.
  • The web page's Users says where each user connects from, and sets it, as mason user add --from and
    mason user from do: the add form's Connects from (From anywhere, the default, or Only from these addresses,
    comma-separated), and the "from …" line under each user's role ("from anywhere", or its address ranges), which
    changes them. A range Mason cannot read is refused in its own words.
  • A read-only web page can still manage its users. MASON_UI_READ_ONLY_ALLOW=users, alone or with tables
    (tables,users), lets a page with MASON_UI_READ_ONLY=1 still add, disable, enable and remove users and change where
    they connect from (POST /api/users, POST /api/users/<name>/disable, /enable and /from, and
    DELETE /api/users/<name>), and it refuses every other change as before. An item that is not tables or users, an
    empty one, or one named twice refuses the start, with a did you mean, naming the item by its place and never quoting
    the value. pageReadOnlyAllow in /api/state and mason status --json lists what the page still changes.
  • mason status --json says why a table's sync by hand would be refused: each table's syncRefusal, in the words
    mason sync <table> refuses with, or null when a sync would run.

What changes

  • On the web page, what an action answered stays until it is dismissed (Tables' Add, Remove and Sync now, Usage's
    cancel of a statement): by its Dismiss, by Escape, or by the next action's answer or question. Before, it went by
    itself after 8 seconds, sometimes before it had been read. A refusal stays as before, and Escape dismisses it too.
  • MASON_PG_PUBLIC_URL holds its host to a name or an address. It now takes a host name with an underscore (a
    Docker service's name) and one written with its trailing dot, printed as written. It now refuses, at the start, a host
    label over 63 characters, a name over 253, a host whose last label is a number but is no IPv4 address (192.0.2), an
    IPv4 address with a number over 255, and an IPv6 address with a zone id (%eth0). If you set it, check it against
    these rules before upgrading. Beside it, mason user add still gives a self-signed listener certificate's SHA-256.
  • MASON_UI_NAV_LINKS refuses the start when an entry has a key written twice
    ({"label": "a", "label": "b", …}), naming the entry; it was read as its last value. A key a link does not take is
    named by its place in the entry, never quoted. mason start with a MASON_UI_NAV_* name that is no setting says it
    is not a setting, with a did you mean.
  • The web page's DELETE /api/users/<name> takes the name as one path segment: a path with more segments
    under it is no route (404).

What was wrong and is now right

  • A mason serve run that was alive but no longer answering read as "not running": mason status said so, and
    mason destroy or a second start treated its project as free while the process still held the synced data. On Linux
    and macOS a silent run is now known as hung however it was started (mason start, mason serve in the image, from a
    checkout or through npx), and mason start refuses beside a hung run, as mason destroy does. A run started before
    the upgrade is judged as before ("Before you upgrade"). On Windows nothing changes.
  • mason add left in a secret column whose name was not lowercase snake_case (passwordHash, apiKey,
    accessToken, API_TOKEN, api-key): it now reads a name's words however the name is written. Every snake_case
    name is judged as before. An entry already in your sources file is not changed ("Before you upgrade").
  • The web page's Tables no longer offers Sync now, enabled, on a table whose sync would be refused: one still in its
    first sync or waiting for it, one whose first sync failed, one being read again whole or being caught up. The button
    is shown disabled there, and its tooltip says why, in the words mason sync <table> refuses with.
  • The web page's Tables no longer offers Add for a table whose row-level security hides every row from the user
    Mason connects as. It says why the table cannot be added, in the same words as mason tables and mason add: the
    policy the table's owner adds, then the mason add to run.
  • When the database cannot be reached, mason doctor's next step says where the URL is: the variable the sources
    file names, in .env. Before, it said .mason/config.json, which does not hold the URL.
  • mason deploy could fail with EPIPE … whether the deploy was made is not known when it sent a deploy again after
    a refusal naming thousands of tables: reading that refusal took longer than the service's idle timeout, and the deploy
    was written onto the connection the service had closed meanwhile. Each request to the service now asks for a
    connection of its own (Connection: close). That holds over HTTP/1.1; a service reached over HTTP/2 shares one
    connection whatever the request asks, and can still give this failure, as before.

Known

  • A generated table can't yet hold a numeric that divides (an average is one); is a variance or a standard deviation, which Postgres computes exactly at decimals it picks per value, rounded, cast to numeric(p,s) or not; is a sqrt, exp, ln, log or power, whose decimals Postgres picks per value, rounded, cast to numeric(p,s) or not, whatever the decimals of what it is computed from; is a JSON number's text cast to numeric with no scale (cast to numeric(p,s), it is stored); has more than 18 decimals, at the end or on the way; sums values whose decimals the rows decide; is a CASE, GREATEST, LEAST or COALESCE of values with differing decimals; or is cast from a double; nor a domain or a composite, and kind: live is refused. Each is refused by file and key before anything is built, said as Postgres types the value (a sqrt or an average of a double is a double, so cast to numeric it is a cast from a double), wherever it sits in the column (under an operator, a coalesce, a round, a cast or an aggregate, or in a scalar subquery, a WITH query, a subquery in FROM or a UNION's arm), with the way out your own SQL with the cast moved and the aggregate kept (sum(sqrt(x)::double precision)), or the WITH query, subquery or UNION the cast belongs in, given whole. Each of these keeps its reason read back from its text (::text, ::varchar, || '', a jsonb's ->>, a WITH column), cast to numeric(p,s) or not, so a typmod over a variance's text never stores an approximation; only a stored column's or a JSON number's text read as a numeric is said as a text: a JSON number's, a stored double's or any other. A run that fail...
Read more