v0.4.0

Persistent storage breaks compatibility with v0.3.0 and earlier. v0.4.0 introduces page format 3, which stores each row as packed binary columns rather than as JSON text, and encodes keys so that they sort in the same order SQL compares their values. Rows take less space, a query can read the columns it needs without parsing the rest of the row, and ALTER TABLE ... ADD COLUMN no longer rewrites every existing row.

v0.1.0 through v0.3.0 use page format 2. Opening a page format 2 database with v0.4.0 fails with UNSUPPORTED_PAGE, and so does opening a page format 3 database with an earlier release. There is no automatic migration, and changing the OPFS name does not copy existing data.

• For data that can be reconstructed, use a new OPFS namespace, for example change opfs://my-app-v2 to opfs://my-app-v3, and rebuild the database. The old namespace remains intact.
• To retain existing data, export the application's known tables using a client pinned to tinyjoin@0.3.0 before upgrading. Create the schema in a new namespace with v0.4.0, import rows with parameterized statements, and verify the data before retiring the old database. TinyJoin does not yet provide a general database export or migration API.
• New apps generated by create-tinyjoin v0.4.0 target tinyjoin@^0.4.0 and use a -db-v3 storage name. Updating the generator does not migrate applications it generated previously.

In-memory databases, including every tinyjoin/node database, are unaffected.

Object rows now list their columns in field order. A row's keys follow the result's fields, as in PostgreSQL, so SELECT title, id returns rows whose first key is title, and a changed key lists a composite primary key's columns in the order the key declares them. Earlier releases sorted both alphabetically. Code that reads a row or key by column name is unaffected; only code that depends on key order, such as Object.keys(row) or JSON.stringify(row), sees a difference.

The Worker protocol version moves from 8 to 10. Client and Worker ship together and are upgraded together by a normal install, so this affects only a deployment that pins or caches a Worker file independently of the client bundle. A mismatched pair fails cleanly with PROTOCOL_MISMATCH, and tabs running different releases fail with DATABASE_VERSION_MISMATCH, rather than misreading each other's results.

Numbers keep every bit. Numbers in JSON values and SQL text are now read with correct rounding. Earlier releases read about one in ten numbers that need all 17 significant digits, such as many results of Math.random(), as a neighboring value. In v0.3.0 and earlier, a row holding one could then not be read at all: reading it failed with PAGED_STORAGE_CORRUPT, because the row no longer matched its stored text.

Opening a database checks its catalog, not every row. Earlier releases read every row and index entry when opening a database, so opening took time in proportion to its size, and refused a database that failed. Each page is still checked when it is read, and the new check() reads the whole database on demand, rejecting with the first problem it finds. A database whose damage no statement reads now opens, and only check() reports it. See checking a database.

This release is also much faster. In the comparative benchmarks, most workloads in v0.3.0 took 10 to 1,400 times as long as the faster of SQLite and PGlite. None now takes as much as twice as long, and reopening a database, reading every row, LIKE scans, GROUP BY, joins, building indexes, and committing a single insert are quicker than in either.

• Updates, upserts, and deletes inside a transaction no longer slow down as the transaction grows. Each statement is checked against running totals rather than the whole write set, so a thousand of them take about 50 ms rather than more than half a minute.
• Comparisons, BETWEEN, and LIKE patterns that start with literal characters read a range of an index or of the primary key, rather than every row. An aggregate that reads only the index's columns is answered from the index alone. See constraints and indexes for when a range is used.
• Joins find each table's matching rows by key lookup, through an index, or in a hash table, rather than comparing every pair of rows. The join budget now counts the rows each lookup actually returns, so a join is never rejected because the Cartesian product of its tables is large.
• A query ordered by the leading primary-key columns reads rows in key order and stops at its LIMIT, rather than collecting and sorting every match, so it is no longer held to the limit on ordered matching rows. Inside a transaction that has changed the table, it still sorts.
• Scans read only the columns a statement uses, straight from the stored row, and so does a statement that finds its row by primary key.
• Each row a statement writes is planned, checked, and measured once. A transaction keeps the rows it stages as the encoded records its commit writes, rather than as maps it copies and encodes again at commit, and reads them in place, as it does stored rows.
• Writes apply each statement's rows to each B-tree in one pass, and index builds sort their entries first. A commit writes each page once, a run of consecutive pages with a single storage call, and makes them durable with one flush rather than three. Its superblock records a fingerprint of the pages it wrote, so reopening after an interrupted commit returns to the last complete one.
• Rows travel from the engine to the page as JSON text, which only the page parses, rather than as objects built one property at a time and then copied between threads. A point query takes about a quarter less time, and reading 10,000 rows well under half. A statement that publishes nothing, such as a read or a statement inside a transaction, passes the rest of its result through the Worker as text too, and point statements take 4 to 7% less time.
• On the tab that owns a database, statements reach the engine as calls rather than as messages checked again at every layer.
• A plain INSERT plans each row straight into the record its commit writes, rather than into a map that staging encodes again. So does an UPDATE that leaves each row's key in place: it rewrites the stored record, keeping the bytes of every column it does not assign, and 1,000 updates by key take about a tenth less of the engine's time. A single-row upsert rewrites the stored row it conflicts with the same way.
• Staging a single-row statement no longer copies the transaction's bookkeeping, and each statement's request and result are checked, and its result read, with less work on both sides of the Worker. Point inserts, updates, and deletes in a transaction take 5 to 10% less time.
• Scans take each leaf's rows straight from the copy of the leaf they hold, checking the tree only as they move between leaves, and read an inline cell and an integer column without the general decoders. A range aggregate over 10,000 rows takes about a quarter less time in the browser.
• A statement's changed rows are kept in the order its scan finds them, rather than sorted into maps as they arrive, so deleting 8,000 rows takes about a fifth less time. A DELETE outside a transaction also plans each row by its stored key, rather than as a map of its key columns, which takes a further fifth off.
• A result's and a commit's changed keys are ordered and deduplicated by their encoded keys, rather than by the JSON text of each, so each table's keys are listed in primary-key order: earlier releases listed {"id": 10} before {"id": 9}. They are kept as rows of values rather than a map for each key. Inserting 10,000 rows 200 at a time takes a tenth less of the engine's time, and a write by primary key 3% less.
• Inside a transaction that has changed a table, a scan finds the rows the transaction replaced leaf by leaf, rather than comparing every row's key with the next staged one, and an integer column is compared with integer bounds as integers. A transaction's 100 range UPDATEs take a fifth less of the engine's time, and 100 range aggregates about a tenth less.
• Writing a page stores each cell's fixed-size header and slot directly, rather than copying them a few bytes at a time, and an index key copies text whole. Creating two indexes over 10,000 rows takes 8% less time.
• Stored entries are fingerprinted with XXH64, which reads eight bytes at a time, rather than with FNV-1a, which hashed them a byte at a time. Every leaf a write creates fingerprints its entries, an index's included, so creating two indexes takes 9% less of the engine's time.
• A commit checksums each page with XXH64, which reads eight bytes at a time, rather than with CRC-32, which looks each byte up in a table, and passes over the zero bytes that fill most of each page. It encodes less besides, writing its allocation bitmap and superblock straight into their pages. Committing a single insert takes over a third less of the engine's time, and pages read back from storage are checked faster too.
• Each superblock carries the allocation bitmap for the first 123 MB of the database in the rest of its page, and a commit writes the bitmap's other pages only when it changes them, where it rewrote three bitmap pages every time. A commit to a smaller database writes its data pages and its superblock and nothing else: committing a single insert takes about 5% less time with OPFS.
• A write transaction looks for free pages 64 at a time, rather than one at a time from the start of the file, and a commit visits only the cached pages it wrote or freed, rather than every page in the cache. A commit's cost now grows much more slowly with the size of the database: committing single inserts into a 100,000-row table takes under half the engine's time it did.
• Opening a database reads its catalog rather than every row and index entry, so it no longer slows down as the database grows. Reopening the benchmark's 10,000-row database takes about half as long, and a 100,000-row database with one index opens in 2 ms of the engine's time rather than 270.
• Several of the engine's small collections are kept in vectors rather than B-tree maps, whose code is compiled anew for each type they hold, which took about 15 KiB off the compressed engine. Its hash maps and sets share one quick hash rather than SipHash, whose resistance to colliding keys needs random keys that WebAssembly without a host source of randomness never gave it, and four sets that needed no hashing became vectors: 3 KiB more. The catalog's tables and indexes, and the keys a write reports, took 5 KiB more, and building the superblock's CRC-32 table when it is first needed, rather than shipping it, 1 KiB more.
• Once per page, create() also starts a short-lived second Worker that runs the engine's common statements on a scratch in-memory database for about a tenth of a second, and then exits. Chromium compiles WebAssembly one function at a time, as each is first called, and Workers running the same module share what either compiles, so a database's first statements of each kind no longer wait for their code to compile. See custom Workers.

The compressed download is now 301 KiB, up from 295 KiB in v0.3.0.