Repository navigation
Mason v0.18.0
A replica that switched to another publication could lose rows or stop keeping tables current. When the sources
file's publication: changed between two starts, a write that was open during the switch could be skipped: on
Postgres 18 its rows never reached the synced tables, and on Postgres 16 the stream stopped for good. Mason now makes a
new replication slot at such a switch, so no row is lost and the stream goes on. It costs one full read of each
replicated table on your database, once. If you switched publication with an earlier Mason, check your replicated
tables ("What was wrong and is now right"). The web page now speaks plain English. Its Tables page lets you choose
the columns an added table syncs, keeps table names readable at laptop widths, and says why a table has no size.
mason deploy now works in a folder that holds only mason/, with no mason init. Two metrics and a Helm key were
renamed. Every project has something to do first: its pin moves ("Before you upgrade").
Before you upgrade
- Move the pin:
version: 0.18.0inmason/mason.yml. The 0.18.0 command and image refuse a project pinned to
0.17.x, and say how to fix it. Before 1.0 they only run a project pinned to their own minor version, at their own
patch or an older one. Set the pin and start 0.18.0 in the same step. - On a hosted replica, upgrade the replica before you deploy a project pinned to 0.18.0.
mason deployrefuses a
pin the replica would not run, before it sends anything, and names both versions.mason replica upgrade <org>/<name>is one way out, where the service offers it. - Rename two metrics and two phase values in your dashboards and alerts. On
GET /metrics,mason_copy_bytesis
nowmason_synced_bytesandmason_copy_rowsis nowmason_synced_rows. Thephaselabel ofmason_tablessays
syncedwhere it saidcopied, andsyncingwhere it saidcopying.waitingandfailedare unchanged. A query
or alert on an old name finds no series after the upgrade, so it reads as absent, not as zero. Rename it in the same
step. Thetablescounts in/healthzkeep their keys. - Helm: rename
memory.copytomemory.enginein your values file beforehelm upgrade. A values file that still
setsmemory.copyfails to render, namingmemory.engine, and nothing is installed. It is never run with the
default in its place. - EXPLAIN (FORMAT JSON) now says
syncing again in fullwhere it saidcopying againinKept Current By, for a
table read again whole at its interval.mason mcp'skept_current_bysays the same. A tool that matched the old
words must be updated to match the new ones. - A heavy statement may now fail where it used to finish. The limit on what statements may spill to disk (half of
the free disk under.mason/, orMASON_SPILL_LIMIT) is now enforced from the start. A statement, or several at
once, that spilled past it now fails with 53400. To allow more, setMASON_SPILL_LIMIT(for example40GB), or ask
for less at once. - An application that decodes Mason's rows by type numbers must look up enums, citext and composites on Mason.
Mason now sends its own number for such a type, not your database's (see "What was wrong and is now right"). A
number learned on your database no longer matches. - If you switched publication with an earlier Mason, check your replicated tables after upgrading ("What was wrong
and is now right"). Nothing on 0.18.0 repairs a table that already missed rows.
What changes
-
mason deployworks in any folder that holds a project'smason/, with nomason init. When there is no
.mason/in the folder or above it, the current folder'smason/is the project (or the folder thatmason -C <folder> deploynames). It needs amason.ymlor a YAML file inmason/sources/. Mason never looks for amason/
elsewhere. Only thatmason/is sent, and the output says which folder it was (folderin--json).--dry-run
finds it the same way. So a clone in CI, or a folder you copied the files into, deploys as it is. A project made with
mason initdeploys as before. -
The web page speaks plain English. It uses short, whole sentences, and says "query" where it said "statement".
It tells you how to do things on the page (Sync now, a table's Sync button, Enable, Users). It names amason
command or a file only when the page can't do it.mason statusand the other commands keep their own words. -
On the web page's Tables, an added table says which columns it syncs, and you can change them there. Its note
--columns …` does. Before you save, the form tells you the cost: the table is synced again in full from your database, and you can still query its synced data until that ends. Saving the same columns changes nothing. If the form can't show an entry as ticks (an `exclude:` written by hand), it describes it and says that saving replaces it. A read-only page shows the columns but can't change them. `mason status --json` adds `columnCount`, and for an added table `entry` and `everyColumnLeavesOut`.
says "All columns", "5 of 12 columns" or "All but 1 column". Click it to tick the columns to sync, as `mason add -
A synced table with no size says why, instead of "—". On the web page's Tables, a table whose rows are all still
in the write-ahead log shows< 3 MiB(less than the log's size). A table not measured yet since its sync, since data
was last written to disk, or since Mason started showsMeasuring…. A table whose first sync is not done shows
nothing. Hover over it to see why.mason usagelists these tables the same way under the sizes, and its--json
addssizedAt,walBytesandunsized. -
"Catching up" is now "Finishing sync". When a table's first sync is done and Mason is getting the changes made
during it, the Tables page says "Finishing sync", with "Getting the changes made since it was last synced. You can
query it meanwhile." under it.mason statusandmason waitsay that sentence too. A streamed table's line says,
for example, "getting the changes made since 09:00 UTC: reading the changes". Thecatching-upstate in--jsonis
unchanged. -
The Overview's "Mason" card shows only what you can use: Database (its name, host, Postgres version, and whether
it is your main database or a read replica), Mason (version and uptime), Disk (free space beside the space the synced
data uses) and Connected tools. Restarts show only when there were some. The sources file, the sync settings and the
page's socket are gone from it; the Tables page shows each table's sync. A database on another machine is now a
quiet line, not a warning, and a main database is no longer a warning on the page. How text sorts shows only when it
differs from your database. -
The web page no longer shows a psql line. Each place that showed one keeps the URL, with Copy: paste it after
psql.mason statusand thepsqlfield in--jsonare unchanged. -
The card for a user just added shows URL, User and Password only, each with Copy. Host, Port, Database and SSL
mode are on the Users page's Connect card above it.mason user add --jsonis unchanged. -
On the web page's Users, Connect comes first, and the cursor starts in the new user's Name (not on a touch
screen, a read-only page, or when you have already clicked into something else). -
Every example in a box on the web page starts with "e.g." (
e.g. orders,e.g. 2h,e.g. metabase), so an
example is never taken for a value already filled in. -
A switch of publication could lose rows or stop the stream. The switch is a change of the sources file's
publication:(orPG_PUBLICATION) between two starts: from Mason's own publication to one you named, from one you
named back to Mason's own, or from one named publication to another. Mason kept using its replication slot with the
new publication. A write transaction that was open while the new publication was created, and committed after, was
then skipped:- on Postgres 18, its rows never reached the synced tables. Your database's log had only a warning:
The publication does not exist at this point in the WAL; - on Postgres 16, the stream failed at every start (
publication "…" does not exist) and never moved again. The
replicated tables stopped changing, and the slot held your database's WAL.
Now a start that sees a switch drops the slot (and Mason's own publication, when that was the one read), checks that
it is gone, and makes a new slot. It then catches each table whose sync isreplicationup by one poll. That poll
is one full read of each of those tables on your database, once.mason startsays so when it makes the switch.This does not repair a table that already missed rows. If you switched publication with an earlier Mason while
your application was writing, find out like this:- On Postgres 18, search your database's log, from the time of the switch, for
The publication does not exist at this point in the WAL. No such line means no rows were skipped. - On Postgres 16, the stream has been failing since the switch.
mason statusandmason logsshow the stream
failing withpublication "…" does not exist, and the replicated tables are behind.
To fix a table, sync it again whole. With Mason running, run
mason remove <table>. Wait untilmason logsshows
dropped <table>'s synced data(the table named with its schema). Then runmason add <table> --sync replication.
The table is not served in between. If you add it back while Mason is stopped, or before that line appears, the add
cancels the remove and nothing is synced again. On Postgres 16, do this for every table you replicate: once none is
left, the remove drops the stuck slot, and the first add makes a new one. - on Postgres 18, its rows never reached the synced tables. Your database's log had only a warning:
-
Table names are readable on the web page's Tables at laptop widths. At 1024 px the name column was 2 px wide.
It now always has room for a typicalschema.tablename. A table's name and its state are never cut short with "…";
a long one wraps onto more lines. In a narrower window, the Key, Size, Last sync and Rows columns give way, in that
order, and the row's note says what they said. On a phone, a row's buttons are stacked so the names fit beside them. -
The Remove button is back on the web page's Tables. Its column was too narrow, so each added table showed "View
events", "Sync now" and then only "…". The column now fits all three buttons at every window width. -
The message after Add or Sync on the Tables page no longer covers the next row's buttons. It is placed where it
covers no other button, or, where there is no such place (on a phone, say), shown as a bar at the bottom of the
window. A tap on the message never presses a button under it, and its Dismiss is always in view. -
The web page no longer shows old data as current while Mason can't be reached. When the page lost Mason (the run
stopped, or the tunnel closed), it kept what it had read, its big "Ready" included, and that looked current. Now the
page greys out what it read before and says when: "As of 14:02 UTC. Mason can't be reached." With nothing read yet,
the Overview says "Unknown". The page is current again as soon as Mason answers. -
Smaller fixes to the web page's layout. The Users table's Added header is no longer in a framed box. On a phone,
a new user's password card is scrolled into view. The Overview's "Connect from anywhere" heading no longer overlaps
the tip above it. The Add a user form no longer jumps when "Connects from" is changed. Radio choices line up in one
column. A site's own links in the sidebar (MASON_UI_NAV_LINKS) line up with the page's own at every width. -
A BI tool now shows a column of an enum, a citext or a composite with its real type. Mason sent such a column
with your database's number for its type. In Mason's own catalog that number could be another type, often a synced
table's row type. So a tool that looks the number up (Metabase, DBeaver and Tableau through JDBC, .NET and asyncpg
clients) showed the column as that table, and a prepared statement that named the type by its number read it as the
other type. Mason now sends its own number for each type, the one its catalog knows. This number can change when the
synced tables change (for example, when a table with a new enum is added): a tool connected across that change may
show old type names until it reconnects.pg_typeofof such a column now matches Postgres too:other.levelfor an
enum in another schema, andmood[]for an array, where Mason saidleveland_mood. -
A database URL that names a missing database no longer lists every database on the server. The failure of
mason init,mason start,mason doctor,mason tablesand the other commands that connect named them all:
hundreds on a large server, and on a shared one, other teams' databases. Now it names the closest one when the name
looks like a typo ("did you meanshop?"), and otherwise only says how many others there are. The fix is the same:
put the right name after the port in the URL. -
The limit on what statements may spill to disk is in force from the start. Before, it was not enforced until the
free disk moved by 1 % or more, so a heavy statement could spill past it and fill the disk. Now a statement past it
fails alone with 53400 and Mason's hint ("Before you upgrade"). Mason also checks every engine setting it gives at
the start, and refuses to start, naming the setting, if one is not what it gave. -
A full disk under
.mason/no longer stops Mason for good. When the disk filled, the run died on its own log,
and the process that restarts it died too. It left.mason/restarts.jsonempty, somason statussaidUnexpected end of JSON inputand nothing started Mason again. Now the run keeps going, says once that its log could not be
written, and writes its log again once there is room. If it stops while the disk is full, it is started again once
there is 8 MiB free.mason statusreads a record it cannot parse as unreadable and says the rest. A write under
.mason/that fails for lack of space says so, with the way out: free disk where.mason/is. Nothing synced is
lost. - A generated table can't yet hold a numeric that divides (an average is one); is a variance or a standard deviation, which Postgres computes exactly at decimals it picks per value, rounded, cast to numeric(p,s) or not; is a sqrt, exp, ln, log or power, whose decimals Postgres picks per value, rounded, cast to numeric(p,s) or not, whatever the decimals of what it is computed from; is a JSON number's text cast to numeric with no scale (cast to numeric(p,s), it is stored); has more than 18 decimals, at the end or on the way; sums values whose decimals the rows decide; is a CASE, GREATEST, LEAST or COALESCE of values with differing decimals; or is cast from a double; nor a domain or a composite, and
kind: liveis refused. Each is refused by file and key before anything is built, said as Postgres types the value (a sqrt or an average of a double is a double, so cast to numeric it is a cast from a double), wherever it sits in the column (under an operator, a coalesce, a round, a cast or an aggregate, or in a scalar subquery, aWITHquery, a subquery inFROMor aUNION's arm), with the way out your own SQL with the cast moved and the aggregate kept (sum(sqrt(x)::double precision)), or theWITHquery, subquery orUNIONthe cast belongs in, given whole. Each of these keeps its reason read back from its text (::text,::varchar,|| '', a jsonb's->>, aWITHcolumn), cast to numeric(p,s) or not, so a typmod over a variance's text never stores an approximation; only a stored column's or a JSON number's text read as a numeric is said as a text: a JSON number's, a stored double's or any other. A run that fails (a value past 38 digits,bigint out of range,division by zero) records Postgres's error, SQLSTATE and DETAIL (numeric field overflowsays the precision and scale it must fit, on every path), and is tried again after its back-off. mason statusshows generated tables; the web page doesn't yet.- Pages of every LIMIT with no ORDER BY over a join may overlap:
LIMIT n OFFSET mwith no ORDER BY over a join has
no stable order, so a tool paging one can see a row twice and miss another. Add an ORDER BY where the pages matter. - A replicated table whose re-read never completes can rejoin the stream with no catch-up, and miss what the stream
passed over, as in every release since 0.8.0. It was found by reading the code and has not been reproduced. - While a replicated table is being compacted, it is reported as current before its newest changes are applied.
Those changes wait until the compaction ends (seconds, or minutes on a very large table), and a query meanwhile
reads the older rows. Nothing is lost. - If the replication slot is dropped under a running Mason, the stream may not start again for the rest of the
run. The replicated tables then stop changing, with no error. Restart Mason (mason stop, thenmason start) if
that happens.