v0.6.3

ChalkQL v0.6.3

A patch release for hosts that keep many keyed views. A maintained view's result now holds one slot per key rather than a list. When the key is the whole row, the view keeps no separate key index at all. A view of keys alone holds about 40% less, and two such views over 100,000 keys now fit where they did not. RefreshBuilder now also says what becomes of the rows a host hands it.

Targets .NET 10. A JDK 21 or newer is needed only to run the planner locally. Assemblies and namespaces retain the Chalk.* names.

NuGet packages:

Package	Licence
ChalkQL	Elastic License 2.0	engine, planner client, catalogue, execution, entitlements and standing views
ChalkQL.Sources	Elastic License 2.0	POCO, Akade.IndexedSet, ADO.NET, DuckDB and source conformance implementations

Keyed views hold less

A host measured an incremental view of keys alone, SELECT id FROM t WHERE … with Key = ["id"], at about 330 B a row for a 16 B payload. Two such views went over a 64 MiB budget at about 85,000 keys, short of the 100,000 the host needed.

Most of that cost was the result's key index. It held a byte array, a list and the list's array for every key, even though a key holds exactly one row unless the view is about to fault. Where the key was every column of the result, the index repeated what the result's identity map already held.

• One slot per key. The index maps each encoded key to the one slot that holds it. A key that holds several rows for a moment, such as an upsert that adds before it takes out, uses a list shared from key to key, so an update allocates nothing for it.
• No index for a whole-row key. Such a key is the row's identity, so the identity map serves as its index.
• Counts grow by half again rather than doubling, as the view's other stores have since 0.6.1.
• The fault for a key given twice is unchanged. A view whose statement gives one key two rows still faults at the end of the update, coded DeclarationNotHeld.

In the host's repro, with 100,000 keys and memory read after a collection:

	0.6.2	0.6.3
The result, per row	about 245 B	about 90 B
A keys-only view, per row	about 330 B	about 175 B
One full 100,000-row view	32.03 MB	17.08 MB
Two unfiltered views	47.85 MB	25.45 MB

Under steady turnover, with keys replaced round after round, a view settles at about 2.4 times its static cost per row and stays there. Removed keys are reclaimed when the view's maps compact.

Who owns a refresh's rows

RefreshBuilder said what becomes of an appended list, and now says it for the other three:

• Replace: a POCO source holds the collection itself as the table's rows, so neither it nor its rows may change while the table reads them. It comes back through PocoSource.SnapshotReleased, as the same reference, once the table has moved on and its last reader has finished. A host that wants an array back in a pool of its own hands it over through ChalkEngine.PrepareReplaceAsync(table, rows, count, …) and Commit, with the table declared ArrayPool(…).
• Upsert: the list is read while the refresh runs, and its rows are copied into arrays ChalkQL owns. It is not held once RefreshAsync returns, so one buffer can carry every batch. The rows themselves become the table's: for a class, give fresh objects rather than change one the table holds.
• Delete: the list is read while the refresh runs. Neither it nor its rows are held afterwards.

This documents what 0.6.2 already did. Nothing about it changes.

Changed

• A keyed view's result holds about 40% less, and a view whose key is the whole row keeps no key index.

Compatibility

• No breaking changes. There is no change to the API, the protocol or the IR. A 0.6.2 planner serves a 0.6.3 client, and a 0.6.3 planner serves a 0.6.2 client.
• StateBytes reads lower for a keyed view. A host that set a view budget from 0.6.2's readings has more room than before.

Full Changelog: v0.6.2...v0.6.3

Licence and attribution

ChalkQL is source available under the Elastic License 2.0 (ELv2).

Apache Calcite and its JVM dependencies are bundled into the planner artefact distributed with ChalkQL. Their licences and attribution are recorded in THIRD-PARTY-NOTICES.txt.