v0.33.0

What's new in 0.33.0 (since 0.32.0)

• Wrong answers fixed. Two IN lists on one column were intersected as if they were sets of values, a NOT IN (NULL) beside another list was dropped, a projection pushed through a filter could swap two columns of one name and type, and - -3 written back as text became a comment. Checked against DuckDB on 100,000 random queries, 99,974 now agree. The other 26 are queries DataFusion refuses and DuckDB doesn't, not wrong answers. A table with an INTERVAL column never left the log, and held every other table of its lake there too. It now goes to files like any other, and its intervals read back exactly. A spread read of a history view counted the versions it had deleted, a session window could miss a new key's late row (seen on simulated R2), and a write to an attached lake could be missed by the next statement. All are fixed.
• Statements that did the wrong thing. CREATE TABLE … PARTITION BY and CLUSTER BY used to make a table with neither. They now set the partitions and clustering, and day(ts) and date_trunc('day', ts) give the same partitions. INSERT OR IGNORE and MySQL's INSERT IGNORE used to replace the row that was there and now leave it alone. CREATE TABLE … LIKE used to make a table with no columns and now copies the columns and types of the table it names.
• Functions and procedures that answered wrongly. A SQL function that reads a table took the caller's column of the same name. Now the call is refused when both read a table of one name, and an alias is the way out. Answers kept for f(1) and f(1.0) were one, and now are kept apart. A view that calls now(), reads files or calls a volatile Python function is no longer answered from the result cache. A FOR loop retried with its job could write one row twice and skip another when its rows came back in a different order. A function built from other functions now uses their current definitions, not the ones they had when it was made. Calls that would copy too much or write out too many expressions are refused by name, and the node stays up.
• Each database locked on its own. A database that has users who sign in is used only through itself or with the nodes' own tokens, so a user signed in next door is nobody there, not even a superuser. Before, GRANT SELECT ON ALL TABLES in one database read another database's tables, and a database with no users let anyone read a locked one beside it. A user granted another database's tables needs a grant that names them, such as GRANT SELECT ON crm.customers TO ann. A database with no sign-in at all stays open to anyone. A procedure started with pondra.start now runs with its caller's grants. Before, it ran with none, so a user granted nothing on a secret could copy it into a table it could write.
• Writes survive failures. In a 4.8-hour run on R2 with three nodes, eight nodes were stopped or killed in turn. Each of 43,138 acknowledged batches was there once, none lost or stored twice, and none of 423,085 reads was torn. A leader cut off from its bucket, while another node is there, now turns writes away at once instead of holding each one until its client gives up. A lake on a small local disk keeps a quarter of the disk free, up to 256 MB. Writes wait with no error when the free room drops below that, and the log drains again once room returns. A lake's format now moves on only when a write needs the new one, so the release before can still open a lake a newer release has only read.
• Every statement remembered, with its plan. SELECT * FROM pondra.history lists every statement from every door: who ran it, how long it took or how it ended, its rows and how many nodes it ran on. A statement whose client stopped waiting is a row too, with the outcome stopped. A slow one (over a second, PONDRA_SLOW_MS) keeps the plan that ran, with each step's rows and time and each node's share, and is written to the node's log too. An admin sees everyone's rows, and anyone else sees their own. EXPLAIN now gives each step's expected rows, and EXPLAIN ANALYZE puts them beside the actual ones. Queries of 100 ms or more note each filter the planner judged off by two times or more, and pondra.learned lists those notes. The planner does not use them yet. The console's History tab shows the same rows.
• Every materialized view stays current. Any SELECT over a lake's tables can now be a materialized view, and it stays current. Pondra picks the cheapest way that is right, and pondra.flows shows which. A GROUP BY with averages, standard deviations, HAVING or ORDER BY … LIMIT is kept as partial rows and finished as it is read. median, count(DISTINCT …) and string_agg recompute only the groups whose rows changed, and write those. Anything else runs whole after each commit to a table it reads, and only the rows that differ are written. A view without a GROUP BY that used ORDER BY … LIMIT, DISTINCT, a window or a subquery over its own table answered wrongly before. WITH (refresh = 'by key') (or 'incremental' or 'full') asks for one way, and lag = '30 seconds' spaces the runs out.
• Environments. CREATE DATABASE dev CLONE prod makes a database that starts as prod is now, copying no data file, and its rows keep their row ids and versions. Writes to either side stay apart, and DROP DATABASE dev removes only what dev wrote. WITH (schemas = (sales)) clones some schemas, and WITH NO DATA clones only the definitions. ALTER DATABASE dev REFRESH orders brings a table, and the materialized views that follow it, up to prod's present. A project is a folder of objects, migrations and tests. pondra plan shows what a deploy would change, and pondra deploy applies exactly that plan and then runs the tests. Migrations run once in each database, and a table changes in place only in ways that keep its rows. pondra branch and pondra diff give each developer a database of their own and compare it with prod, and CALL plan(…) and CALL deploy(…) do the same from any door. pondra workspace pull and push keep a lake's files in step with a folder. Secrets, shares and recipients never reach a branch or a deploy.
• Share a table with another company. Sharing is SQL: CREATE SHARE acme, ALTER SHARE acme ADD TABLE orders, CREATE RECIPIENT acme_corp EXPIRES IN '90 days', then GRANT SELECT ON SHARE acme TO RECIPIENT acme_corp. The other company reads the files from the lake's storage with its own tools, such as pandas, Spark, Power BI or another Pondra, through links that last 15 minutes. It uses the open Delta Sharing protocol, so its clients work as they are. A recipient sees only what is shared, at a published version, and every request is kept in the audit log. Partitions, deletes, history and renamed columns read correctly. The console's table menu has Share with another company….
• Writes spread over every node. INSERT … SELECT and CREATE TABLE … AS are now written by every node at once when their rows split as they are, and one commit records all their files. On three nodes sharing one 4-core machine, copying TPC-H's lineitem at scale factor 1 took 3.6 s, down from 5.9 s. Those whose rows don't split as they are are written from one node.
• Sequences, identity columns, UNIQUE, enum types and indexes. Sequences hand out each value once, even across three nodes and a leader's kill, and identity columns work the same way. UNIQUE is enforced by the leader on every SQL write to the table, and a duplicate fails with Postgres's code 23505. An enum column refuses labels it doesn't list (22P02), and casts to an enum and enum_range work. CREATE INDEX is kept as an object, and a notice says that nothing is built. One registry covers every kind of object: COMMENT ON sets a comment, SHOW CREATE gives the statement that makes an object again, and pondra.objects lists them all. CREATE OR ALTER TABLE brings a table to a new definition by adding columns, widening types and setting its layout, and it refuses anything that would lose or reinterpret rows.
• DuckDB's and other databases' spellings. Eighteen of the 22 everyday forms that failed now work, each answering as DuckDB 1.5.5 does: PIVOT and UNPIVOT, COLUMNS('re'), SELECT * RENAME, ORDER BY ALL, FETCH FIRST, list comprehensions, lambda functions, max_by and arg_max, ASOF JOIN … ON, SUMMARIZE and USING SAMPLE. A select's alias can be used in its WHERE. TABLESAMPLE used to be ignored and return every row, and now it samples. Spellings from DuckDB, Snowflake, Postgres and BigQuery work too, among them VERSION AS OF and FOR SYSTEM_TIME AS OF for time travel, SHOW SCHEMAS, SHOW USERS, SHOW SECRETS, and ANALYZE and REFRESH MATERIALIZED VIEW, which dbt and BI tools send and which now succeed with nothing to do.
• Windows as SQL, with EMIT FINAL. A window is now the query's own GROUP BY on a time bucket, so its size is stated once: date_bin(INTERVAL '1 minute', ts) for minutes, or SESSION(ts, INTERVAL '30 minutes') for sessions. EMIT FINAL writes each group once, when time passes it. lateness = '10 seconds' says how late a row may arrive, and idle = '1 minute' closes a quiet stream's last group. pondra.flows counts late rows per view, though session views don't count theirs yet. A time bucket that only grows with time works (date_bin, date_trunc, time_bucket, a cast to a date), and one that repeats, such as hour(ts), is refused by name. The old options form still works.
• Vector search and model calls. An exact top 10 of 200,000 vectors of 384 floats takes 0.063 s, against 0.154 s for DuckDB on the same 4-core machine, and returns numpy's answer. cosine_similarity, cosine_distance, l2_distance and dot_product also answer to DuckDB's and pgvector's names. ai_complete and ai_embed ask each distinct text once, send embeddings 64 to a request, and t...

v0.32.0

What's new in 0.32.0 (since 0.30.0)

0.31.0 and 0.31.1 were never released, so this covers both.

• Faster. The join order is tried from every input: TPC-DS q72 drops from 81 s to 0.17 s, and all 99 queries from 18.2 s to 12.1 s. Small queries cost half what they did. Flows stay within 5% of ingest. In-memory scans skip batches by their ranges and read a top-N in key order. A SELECT * top-N filters while it decodes Parquet: ClickBench from files beats DuckDB 1.5.5 (10.47 s against 11.10 s), and from memory takes 6.58 s. In-memory columns now hold what they count. Key lookups while writes land are 16× faster (27,600 a second).
• A wrong answer fixed. A global min and max of two things could skip rows it needed (wrong since before 0.30.0).
• Runs for years. Lakes have a format version. Every release's lake since 0.22 opens and answers as before. Rolling upgrades work. A stopped node drains (/healthz, /ready), and a leader steps down cleanly. A key lookup that could miss a live key is fixed.
• Survives failures. Every way Pondra runs is now checked under a failing bucket, a leader cut off, killed nodes and full disks. That check found five faults, all fixed: idempotent Kafka producers could lose acknowledged records across a failover, and a leader cut off from its bucket held every write. The Python and JavaScript clients now take every node's address and carry on when one goes down. A lake that fills its own small disk now recovers by itself once there is room.
• A table's past. UNDROP TABLE brings back a dropped table within its retention (a day unless the table says otherwise). SELECT … FROM t AT (VERSION => n | TIMESTAMP => '…' | OFFSET => -3600) reads an append table as it was. RESTORE TABLE t TO VERSION AS OF n puts it back in one change, and CREATE TABLE c CLONE t copies it without copying a file.
• Deployed your way. There is a container image, a compose cluster, a Helm chart and pondra service install (systemd, launchd, Windows). A node sizes itself to its container's memory.
• SQL. CREATE VIEW v (a, b) names the view's columns. A transaction's UPDATE followed by an INSERT into the same table commits.
• Scripts that decide. A SQL script can branch, loop, handle errors and return a value (IF, CASE, WHILE, REPEAT, LOOP, FOR r IN (query), BEGIN … EXCEPTION … END, RETURN, RAISE, PRINT, ASSERT, EXECUTE IMMEDIATE, CALL … INTO, IDENTIFIER), from every client: the console, files, POST /sql, psql and every Postgres driver, MCP, procedures and tasks.
• Changed: a file's parameters are marked. Only DECLARE PARAMETER $day DATE = … makes a parameter of a SQL file; a plain DECLARE is now the file's own variable, and a run given a name that isn't a parameter is refused. A file whose plain DECLAREs were its parameters needs PARAMETER added. A .py file's parameters are its # %% tags=["parameters"] cell.
• The console. Live cells share one connection. There is one grid everywhere, with undo, paste and filters. The Data tree has details, drag and drop and uploads. There is a schedule editor, users, roles and access, and a table's own tab that saves edits as one transaction.

The details are in docs/prototype-status.md (round 32) and on the performance page.

Install

You have	Run	Then
Windows	irm https://github.com/alimardon123/pondra/releases/latest/download/install.ps1 | iex	pondra
Linux, macOS	curl -fsSL https://github.com/alimardon123/pondra/releases/latest/download/install.sh | sh	pondra
Python	pip install pondra (add pyarrow for pandas, Polars and Arrow)	pondra, python -m pondra, or import pondra
Node	npm install -g pondra	pondra, or npx pondra with no install

The files below: the binary alone for each platform (pondra-<platform>.tar.gz / .zip, what the
installers download), the Python wheels and the npm packages. What changed is in the commits
below and in docs/prototype-status.md.

What's Changed

• Deploy and distribute: an image, a compose cluster, a Helm chart, pondra service, Homebrew, Scoop, winget and signing by @alimardon123 in #5
• Round 32, part 1 (0.31.0): join order from every input, small queries at half the cost, flows within 5% of ingest, hot batches skipped, answers without stray strings by @alimardon123 in #4
• Console: answers in place, Run and Run file, a session per tab, live queries on one connection, renaming everywhere by @alimardon123 in #7
• 0.31.1: a global min and max of two things skips no row it needs by @alimardon123 in #9
• Run it for years (ADR-039): lake format, every release's lake opens, rolling upgrades, drain, soak by @alimardon123 in #6
• CREATE VIEW v (a, b): the column list names the view's columns by @alimardon123 in #12
• Console batches B to E: one grid everywhere, the Data tree and Workspace, Jobs and access, a table's own tab by @alimardon123 in #8
• Round 32 part 2: a top-N of many columns filters as it decodes Parquet; a transaction's UPDATE then INSERT commits by @alimardon123 in #11
• Resilience: every mode under failure, the five faults it found; and faster CI for every branch by @alimardon123 in #10
• DECLARE PARAMETER: a file's parameters are marked; its other variables are its own (ADR-043) by @alimardon123 in #15
• A table's past: UNDROP, retention per table, time travel AT (…), RESTORE, zero-copy CLONE (ADR-043) by @alimardon123 in #13
• Scripts that decide (ADR-045, phase 1): IF, CASE, loops, handlers, RETURN, EXECUTE IMMEDIATE, from every door by @alimardon123 in #16
• A full local disk recovers on its own by @alimardon123 in #18
• Round 32 complete (0.32.0): in-memory columns that hold what they count, key lookups indexed over the log tail by @alimardon123 in #14

Full Changelog: v0.30.0...v0.32.0

v0.30.0

Install

You have	Run	Then
Windows	irm https://github.com/alimardon123/pondra/releases/latest/download/install.ps1 | iex	pondra
Linux, macOS	curl -fsSL https://github.com/alimardon123/pondra/releases/latest/download/install.sh | sh	pondra
Python	pip install pondra (add pyarrow for pandas, Polars and Arrow)	pondra, python -m pondra, or import pondra
Node	npm install -g pondra	pondra, or npx pondra with no install

The files below: the binary alone for each platform (pondra-<platform>.tar.gz / .zip, what the
installers download), the Python wheels and the npm packages. What changed is in the commits
below and in docs/prototype-status.md.

What's Changed

• Round 31, second part: OR REPLACE / IF NOT EXISTS everywhere, DETACH, SQL and Python in one notebook, Spark SQL in spark.sql; Windows builds again by @alimardon123 in #2
• SQL variables and a file's declared parameters (ADR-037); CI builds a commit once by @alimardon123 in #3

New Contributors

• @alimardon123 made their first contribution in #2

Full Changelog: v0.27.0...v0.30.0

v0.27.0

Install

You have	Run	Then
Windows	irm https://github.com/alimardon123/pondra/releases/latest/download/install.ps1 | iex	pondra
Linux, macOS	curl -fsSL https://github.com/alimardon123/pondra/releases/latest/download/install.sh | sh	pondra
Python	pip install pondra (add pyarrow for pandas, Polars and Arrow)	pondra, python -m pondra, or import pondra
Node	npm install -g pondra	pondra, or npx pondra with no install

The files below: the binary alone for each platform (pondra-<platform>.tar.gz / .zip, what the
installers download), the Python wheels and the npm packages. What changed is in the commits
below and in docs/prototype-status.md.

Full Changelog: v0.26.0...v0.27.0

v0.26.0

Install

You have	Run	Then
Windows	irm https://github.com/alimardon123/pondra/releases/latest/download/install.ps1 | iex	pondra
Linux, macOS	curl -fsSL https://github.com/alimardon123/pondra/releases/latest/download/install.sh | sh	pondra
Python	pip install pondra (add pyarrow for pandas, Polars and Arrow)	pondra, python -m pondra, or import pondra
Node	npm install -g pondra	pondra, or npx pondra with no install

The files below: the binary alone for each platform (pondra-<platform>.tar.gz / .zip, what the
installers download), the Python wheels and the npm packages. What changed is in the commits
below and in docs/prototype-status.md.

Full Changelog: v0.25.0...v0.26.0

v0.25.0

Install

You have	Run	Then
Windows	irm https://github.com/alimardon123/pondra/releases/latest/download/install.ps1 | iex	pondra
Linux, macOS	curl -fsSL https://github.com/alimardon123/pondra/releases/latest/download/install.sh | sh	pondra
Python	pip install pondra (add pyarrow for pandas, Polars and Arrow)	pondra, python -m pondra, or import pondra
Node	npm install -g pondra	pondra, or npx pondra with no install

The files below: the binary alone for each platform (pondra-<platform>.tar.gz / .zip, what the
installers download), the Python wheels and the npm packages. What changed is in the commits
below and in docs/prototype-status.md.

Full Changelog: v0.24.0...v0.25.0

v0.24.0

Install

You have	Run	Then
Windows	irm https://github.com/alimardon123/pondra/releases/latest/download/install.ps1 | iex	pondra
Linux, macOS	curl -fsSL https://github.com/alimardon123/pondra/releases/latest/download/install.sh | sh	pondra
Python	pip install pondra (add pyarrow for pandas, Polars and Arrow)	pondra, python -m pondra, or import pondra
Node	npm install -g pondra	pondra, or npx pondra with no install

The files below: the binary alone for each platform (pondra-<platform>.tar.gz / .zip, what the
installers download), the Python wheels and the npm packages. What changed is in the commits
below and in docs/prototype-status.md.

Full Changelog: v0.22.2...v0.24.0

v0.22.2

Install

You have	Run	Then
Windows	irm https://github.com/alimardon123/pondra/releases/latest/download/install.ps1 | iex	pondra
Linux, macOS	curl -fsSL https://github.com/alimardon123/pondra/releases/latest/download/install.sh | sh	pondra
Python	pip install pondra (add pyarrow for pandas, Polars and Arrow)	pondra, python -m pondra, or import pondra
Node	npm install -g pondra	pondra, or npx pondra with no install

The files below: the binary alone for each platform (pondra-<platform>.tar.gz / .zip, what the
installers download), the Python wheels and the npm packages. What changed is in the commits
below and in docs/prototype-status.md.

Full Changelog: v0.22.1...v0.22.2

v0.22.1

Install

You have	Run	Then
Windows	irm https://github.com/alimardon123/pondra/releases/latest/download/install.ps1 | iex	pondra
Linux, macOS	curl -fsSL https://github.com/alimardon123/pondra/releases/latest/download/install.sh | sh	pondra
Python	pip install pondra (add pyarrow for pandas, Polars and Arrow)	pondra, python -m pondra, or import pondra
Node	npm install -g pondra	pondra, or npx pondra with no install

The files below: the binary alone for each platform (pondra-<platform>.tar.gz / .zip, what the
installers download), the Python wheels and the npm packages. What changed is in the commits
below and in docs/prototype-status.md.

Full Changelog: v0.22.0...v0.22.1

v0.22.0

ADR-018: the first release run's other four platforms passed

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01PebNcp4uvoEu1UTbdRasHr