Skip to content

A connection with no file behind it - #20

Merged
tamnd merged 1 commit into
mainfrom
dx-memory-connection
Aug 19, 2026
Merged

A connection with no file behind it#20
tamnd merged 1 commit into
mainfrom
dx-memory-connection

Conversation

@tamnd

@tamnd tamnd commented Aug 19, 2026

Copy link
Copy Markdown
Owner

connect(':memory:') made a file called :memory: in whatever directory the caller happened to be standing in. The name says nothing is on disk and something is, which is the worst of both worlds, and it is the last open row of the DuckDB comparison in the engine's docs/clients/duckdb.md.

What changed

connect() with no path, connect(null) and connect(':memory:') open a database in memory. No file is made anywhere and nothing survives the last connection.

It is the whole engine and not a reduced one. Writes, transactions, the appender, registered frames and streams all work exactly as they do on a file, because underneath it is the same header, the same catalog, the same write-ahead log and the same recovery, running on bytes that are not a file. That is tamnd/zu#405, which put a virtual filesystem under the storage layer rather than bolting a mode to the side of it.

Options may stand where the path would, so connect({ threads: 2 }) is a call and not a mistake. The first argument is read in the function rather than declared in the signature, because a value that is a string in one call and an object in the next is a value napi would refuse with a StringExpected thrown out of the call before this client got to say anything about it, which is the same reason text() already exists here.

conn.memory is new and says which kind of database is behind the connection. It exists because path cannot quite answer that: a file can be called :memory: on any filesystem that allows a colon. The path is reported back as it was asked for rather than as the engine spells it, since the engine mints a unique name per database so that two of them never share a writer, and that counter is its business and not a caller's.

Tests

Seven new in test/connect.test.mjs: that a bare connect() writes and reads and leaves the directory empty, that ':memory:' makes no file called that, that options standing first are read, that two of them share nothing, that one takes a transaction and rolls it back, that a database on disk is not one in memory, and that read-only is refused. The README gains a section whose program is run like the others on that page, with the directory checked empty afterwards.

etc/zudb.api.md and binding.d.cts move with the signature.

What the engine bump moved

The pin goes from 8aa27d9c to 95c7c990, and test/plan.test.mjs moved with it. It asserted scan.estimate === 2 on a table two INSERT statements old. The estimate comes off the catalog's summary, and rows written through the engine's fold-free write path have not reached it yet, so a freshly written table now estimates 1 against 2 rows. Reopened, the same table estimates 2 with a q-error of 1. Both are now asserted, because the difference between them is a real one a reader would otherwise take for noise. Whether the planner should see rows that have not been folded is the engine's question and not this client's, and it is worth asking there.

Gate

npm test is 311 passing and 0 failing, npm run check:types, npm run check:api, cargo clippy -- -D warnings and cargo fmt --check are all green locally.

Part of tamnd/zu#169.

`connect(':memory:')` made a file called `:memory:` in whatever
directory the caller happened to be standing in, which is the worst of
both worlds: the name says nothing is on disk and something is. It is
the last open row of the DuckDB comparison, and it is now the engine's
answer rather than a special case here.

`connect()` with no path, with `null`, and with `':memory:'` open a
database in memory. No file is made anywhere and nothing survives the
last connection. It is the whole engine and not a reduced one: writes,
transactions, the appender, registered frames and streams all work
exactly as they do on a file, because underneath it is the same header,
the same write-ahead log and the same recovery running on bytes that
are not a file.

Options may stand where the path would, so `connect({ threads: 2 })` is
a call and not a mistake. The first argument is read here rather than
declared, because a value that is a string in one call and an object in
the next is a value napi would refuse before this client got to say
anything about it.

`conn.memory` says which kind you have, since `path` cannot quite
answer it on a filesystem that allows a colon in a name. The path is
reported back as it was asked for, `:memory:`, rather than as the
engine spells it: the engine mints a unique name per database so two of
them never share a writer, and that counter is its business.

The engine pin moves to 95c7c990, which is where the in-memory database
landed, and the profile test moves with it. It pinned the optimizer's
estimate to the row count, and rows written without folding the file
have not reached the catalog's summary yet, so a table two statements
old estimates low and the same table reopened estimates right. Both are
now asserted, because the difference is a real one a reader would
otherwise take for noise.
@tamnd
tamnd merged commit f573a39 into main Aug 19, 2026
21 checks passed
@tamnd
tamnd deleted the dx-memory-connection branch August 19, 2026 11:50
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant