Releases: HeliosDatabase/HeliosDB-Nano
Release list
v4.7.0
Transaction and shutdown correctness. One atomicity violation, plus three fixes
that together are the reason the engine's clean-shutdown work never ran in a
real deployment.
- Fix (atomicity, pre-existing):
ON DELETE CASCADEandON DELETE SET NULL
escaped the enclosing transaction. Both ran in their own autocommit
transaction that committed immediately, soBEGIN; DELETE parent; ROLLBACK;
restored the parent row while the cascaded child deletions — or the NULLed
child FK columns — stayed permanently applied. The child effects now join the
caller's transaction and roll back with it. Autocommit behavior is unchanged. - Fix (durability, pre-existing): at close, index snapshots were persisted
before row counters were flushed. Because a valid index snapshot is what
tells the next open that shutdown was clean, a crash between those two steps
produced a database that looked cleanly shut down while carrying a stale row
counter — and the next insert would reuse a live row id, overwriting data.
This is the failure 4.6.1's counter reseed was meant to prevent. Counters are
now flushed first. - Fix (shutdown, pre-existing): the server had no
SIGTERMhandler; only
Ctrl+Cwas handled. Since the Unix default for an unhandledSIGTERMis
immediate termination, no close-time work ran undersystemctl stop,
docker stop, Kubernetes pod termination — orheliosdb-nano stop, which
sendsSIGTERMitself and then waits two seconds for a graceful shutdown that
could not occur. The documented way to stop a server was an abrupt kill.
SIGTERMandSIGINTnow follow the same shutdown path, and the log records
which signal arrived. - Fix (shutdown, pre-existing): the HTTP endpoint's accept loop held a
reference to the database and was never shut down, so the database was never
dropped and its close-time work never ran under the default--http-port 8080
— not even onCtrl+C. Without this fix the two items above have no effect on
a defaultheliosdb-nano start.
Upgrade note: if you relied on cascaded deletes persisting after a rolled-back
parent DELETE — behavior no standard SQL database exhibits — that no longer
happens. Servers now do measurable work on SIGTERM; if you run under a process
supervisor with a very short kill timeout, verify it allows shutdown to finish.
heliosdb-nano stop still hardcodes a 2-second grace period before SIGKILL,
which a large database may exceed; making it configurable is tracked for 4.8.
Also adds docs/plans/ROADMAP_V5.md, an audited inventory of known outstanding
issues sequenced toward 5.0. It documents several limitations candidly,
including that row-level security is currently not enforced on write paths.
v4.6.3
Constraint enforcement is now identical across both DML executor families.
HeliosDB Nano runs two parallel DML paths, selected by the wire frame type: a
text family (PostgreSQL simple-query, all MySQL wire, the REPL, the embedded
execute() API) and a params family (PostgreSQL extended protocol, plus every
REST/BaaS write). The params family had drifted, and three constraint checks the
text family performed were missing from it. Any client that uses the extended
protocol — psycopg2 server-side cursors, JDBC, sqlx, Drizzle, node-postgres — or
that writes through /rest/v1/, could bypass them.
- Fix (data integrity, pre-existing): a parameterized
DELETEdid not
enforce referencing (inbound) foreign keys.NO ACTION/RESTRICTwere
not rejected, andON DELETE CASCADE/SET NULLnever ran — so deleting a
parent row through the extended protocol silently orphaned its children, with
no error and no cascade. The same statement over the simple-query protocol
behaved correctly, which made this hard to spot: the bug depended on the
client driver, not the SQL. - Fix (data integrity, pre-existing): a parameterized
UPDATEdid not
enforceUNIQUE, for either single-column constraints or multi-column table
constraints, admitting duplicate keys. - Fix (data integrity, pre-existing):
INSERT ... ON CONFLICT DO UPDATE
never revalidated the post-merge row on either family.CHECKconstraints
were not re-evaluated — a conflict update could set a column to a value the
table's ownCHECKforbids — and foreign keys on the updated row went
unverified.
The three enforcement blocks now live in shared helpers called from both
families rather than in two implementations that can diverge again. Error
messages are unchanged. New suite tests/constraint_parity_tests.rs runs each
affected statement through both families and asserts they agree.
Known limitations left unchanged by this release, each pre-existing and tracked
separately: ON CONFLICT DO UPDATE does not re-check NOT NULL (matching the
general UPDATE arm); the UNIQUE self-collision guard tests "value changed"
rather than "different row", so a same-statement key swap or cycle is rejected;
ON DELETE CASCADE / SET NULL run in their own autocommit transaction, so
child-row effects survive a ROLLBACK of the parent statement; and the UNIQUE
probe reads the branch-blind shared index.
v4.6.2
Three fixes closing out the latent-bug backlog. The wire-protocol one is the
significant find:
- Fix (wire-protocol correctness, pre-existing): the PostgreSQL catalog
dispatch matchedpg_type/pg_tables/pg_views/pg_settings/
pg_indexes/information_schema.as bare substrings of the raw statement
text, so a statement that merely mentioned one — inside a string literal,
a SQL comment, or as part of a longer identifier — had its real semantics
discarded and a canned catalog response returned instead. Worst case, an
UPDATE ... SET note='see pg_tables'silently never executed while the
client received a reply it could not distinguish from success; a
CREATE TABLE pg_type_registry (...)silently created nothing; and a user
table namedapp_pg_settingswas permanently shadowed. Interception is now
restricted to read statements, matched against a literal- and
comment-stripped view of the query, with word-boundary-aware marker
matching. Client introspection (psql\dt, SQLAlchemy/pgAdmin/DBeaver/
Drizzle probes) is unaffected. SIMILAR TOnow matches PostgreSQL semantics. Three bugs fixed:
alternation was not bound by the implicit full-string anchoring (so
'zzxyz' SIMILAR TO 'abc|xyz'wrongly returned true); the repetition
quantifiers*+?{m,n}were treated as literal characters rather
than metacharacters (so'aaa' SIMILAR TO 'a+'wrongly returned false);
and theESCAPEclause was parsed but silently ignored.ESCAPE 'x',
ESCAPE ''(escaping disabled), and the default backslash escape all
behave per PostgreSQL now. This surface previously had no test coverage at
all; it now has 10 tests.- Fix: bulk loads persisted their row-id counter to a key the engine never
reads back on open, leaving the canonical counter stale. It now writes
through the canonical path — relevant for crash recovery of internal
bookkeeping tables, which are skipped by the counter reseed added in 4.6.1.
v4.6.1
Four latent-bug fixes, none release-blocking on their own but worth a prompt
patch given the severity of the wire-protocol issue:
- Fix (silent data corruption, pre-existing): a hard crash (
kill -9,
power loss, abort — anything that skips a clean shutdown) could leave a
table's row-id counter stale if fewer than 64 rows had been inserted since
its last periodic flush. On reopen, the nextINSERTwould then reuse an
already-in-use row id, silently overwriting the pre-existing row. Fixed by
reconciling the counter against the actual max row id scanned whenever the
index-rebuild-on-open path runs without a valid snapshot (which happens
precisely on a crash reopen, and is a no-op on a clean one). - Fix (wire-protocol correctness hazard, pre-existing): the PostgreSQL
wire handler intercepted any statement whose raw text merely contained
version(),current_database(), orcurrent_user— anywhere, including
inside a WHERE clause — and answered with a hardcoded canned reply instead
of executing the real statement. In the worst case, anUPDATE/DELETE
referencing one of these in itsWHEREclause never executed at all, while
the client received a reply indistinguishable from a real "0 rows matched"
result. Fixed by removing the interceptors; the real query engine already
answers all three correctly and session-aware. ~!~~*!~*(PostgreSQL POSIX regex match operators) and the raw
~~~~*!~~!~~*operator spellings of LIKE/ILIKE are now
implemented — previously errored with "Binary operator not yet supported"
for every case, including inWHEREclauses.CREATE SCHEMA name CREATE TABLE ... CREATE TABLE ...(PostgreSQL's
multi-element schema-creation form) is now accepted — previously any
statement with more than one embedded element failed to parse entirely.
Bare names inside the block (including cross-references between sibling
tables, e.g.REFERENCES/PARTITION OF) resolve into the new schema; the
whole statement is all-or-nothing (a failure partway through leaves nothing
behind, matching PostgreSQL).
v4.6.0
Two schema Stage-2 follow-up items (leased to and delivered by the fleet's
HDB session) plus a round-3 cheap-root cascade:
- PostgreSQL
NaN/Infinity/-InfinityNUMERICsupport — the three
special values are now accepted end-to-end (cast, comparison, equality,
sort, arithmetic, aggregates, predicate pushdown), matching PostgreSQL's
numeric.ccontract:NaNsorts greater than all non-NaNvalues and
NaN = NaNisTRUE(deliberately, for btree/GROUP BY/DISTINCT/
ORDER BYusability, unlike IEEE-754);+Infinity/-Infinitycompare and
propagate through arithmetic as PostgreSQL does (Inf-Inf,Inf/Inf,
Inf*0=NaN;finite/Inf=0;Inf/0still raises division-by-zero). CREATE FUNCTION … RETURNS TABLE(cols)— the composite return form is
now accepted (previously rejected with "Custom data type not yet
supported"); the column list is not yet persisted and set-returning
execution is not yet wired (function calls keep existing scalar behavior).- Real catalog schema truth (
pg_namespace/relnamespace) —
pg_namespacelists actual registered schemas (no longer hardcoded to
public);pg_class.relnamespacemaps each relation to its real schema's
oid via a stable, deterministic name→oid map, so the two catalogs join
consistently.current_schema()is now session-aware;current_schemas(bool)
added. - Multi-entry
search_path—SET search_path TO a, bnow resolves bare
table references by walking the full ordered list (first match wins),
replacing the prior first-non-public-entry-only behavior;SHOW search_path
keeps the established append-publicdisplay convention. - Fix (pre-existing): cross-type foreign keys on the
INSERT ... SELECT
path reported phantom violations (the twin of the direct-INSERTfix in
4.5.0); now routes through the same type-aware validator, which also fixes
deferred-FK skip on that path. - Measured on the PG regression corpus: +14 statements flip to passing across
both handbacks, 0 regressions.
v4.5.0
Real schema support:
- Schemas coexist — same-named tables in different schemas are now
distinct objects (a.tandb.tno longer collide). Public-schema
behavior and existing data directories are unchanged (fully backward
compatible). search_pathis honored per session —SET/SHOW/RESET search_path
work over the wire and embedded; bare names resolve to the session's
schema, falling back to public; connection isolation guaranteed (two
connections with different search_paths cannot see each other's
resolution).- Schema DDL —
CREATE SCHEMA [IF NOT EXISTS]registers (duplicates
error);DROP SCHEMA [IF EXISTS] … RESTRICT|CASCADE(cascade drops member
tables, composing with the partition-child cascade);ALTER TABLE … SET SCHEMAmoves tables between schemas. - Deferred constraints —
SET CONSTRAINTS {ALL|names} {DEFERRED| IMMEDIATE}implemented (wire + embedded); column-levelINITIALLY DEFERREDhonored; deferred FK checks validate at COMMIT and evaporate if
the referenced table was dropped in-transaction (PG parity). - Introspection —
information_schemareports realtable_schema
values;pg_class.relnamestays bare. - Fix (wrong-data, pre-existing): the first
INSERTafter any
ALTER TABLE … RENAMEsilently overwrote the oldest row (volatile row
counter was not migrated with the rename). - Fix (pre-existing): cross-type foreign keys (e.g.
int8child
referencingint4parent) reported phantom violations on the indexed
fast path; probe values now coerce to the referenced columns' types. - Measured on the PG regression corpus: +158 statements flip to passing;
~60 prior "passes" that depended on cross-schema name collisions now
correctly error (they were silently operating on the wrong tables).
v4.4.0
Wave-3 implementation (leased to and delivered by the fleet's HDB session) and
round-3 declarative-partitioning Stage 0:
- PARTITION BY support (Stage 0) — PostgreSQL declarative-partitioning DDL
is now accepted:CREATE TABLE … PARTITION BY RANGE/LIST/HASHparents
(including multi-column, expression, and opclass keys),CREATE TABLE … PARTITION OF … FOR VALUES/DEFAULTchildren (columns cloned from the
parent), andATTACH/DETACH PARTITIONas accepted no-ops.DROP TABLE parentcascades to its partition children (PG parity);
pg_class.relpartboundis present (NULL). Stage-0 semantics: each child
is an independent table — INSERT into the parent is not routed and SELECT
from the parent does not union children yet (Stage 1). Measured on the
PG regression corpus: +1,427 statements flip to passing. - Version-copy elision — new
[storage] elide_latest_version(default
off): the latest row version lives only indata:, cutting single-INSERT
version-write volume from 65% to 41% of bytes. One-way door: once
enabled on a data dir, downgrade requires dump/restore. - Statement retry on write conflicts —
[locks] statement_retry_max
(default 0 = off) auto-retries autocommit statements that hit the typed
SQLSTATE 40001 write conflict; backoff knobs included. - Snapshot schema evolution —
[storage] snapshot_schema_evolutionnow
defaults to"null_pad": reads through a snapshot taken before anALTER TABLE ADD/DROP COLUMNreturn isolation-correct NULL-padded/truncated rows
instead of erroring. Set"strict"to restore the previous error behavior. [locks] timeout_msis now genuinely wired to the lock manager
(previously orphaned); default 1000 preserves the prior effective bound.- Fix (durability): the fast-INSERT row counter is flushed at clean
shutdown — prevents silent row overwrites on reopen after short sessions in
relaxed-WAL mode (crash-path reseed tracked separately). - COPY constraint batching: ART maintenance batched per-table; CHECK
expressions evaluated via a batched path (constrained-COPY follow-up from
the Wave-3 measurements).
v4.3.0
Wave 3 (design-first) of the 2026-07 perf & stability campaign, the July
compat round 2, and a supply-chain hardening pass:
- Typed write-conflict errors — same-row write conflicts now surface as a
typedWriteConflicterror carrying table/row/holder context and map to
SQLSTATE 40001 (serialization_failure) on the wire (previously a
generic 25000), so PostgreSQL drivers' retry machinery engages. Timeout and
locking behavior are unchanged. - Performance attribution instrumentation (all zero-cost when disabled):
lock-contention census behind the newlock-censuscargo feature +
[performance] lock_censusknob (heliosdb_lock_censussystem view);
per-statement-class write-volume byte accounting ([performance] write_volume_stats→heliosdb_write_volume); COPY wall-time phase
breakdown ([performance] copy_phase_stats→heliosdb_copy_phase_stats).
Measured verdicts and next-step designs live in
docs/plans/PERF_STABILITY_2026_07/W3_*_DESIGN.md(notable: the version
chain is 65% of single-INSERT byte volume; CHECK evaluation is 35% of
constrained-COPY time; the c≥32 wire plateau is not mutex-blocking). - Compatibility round 2 (12 fixes + 1 hardening):
CREATE/DROP DOMAIN
no-op intercepts,CREATE TABLE … INHERITSclause stripping, and other
pg-dialect gaps;catch_unwindisolation extended to the extended-protocol
DML-RETURNING path (an erroringUPDATE … RETURNINGno longer kills the
connection). - Supply chain: the openssl stack is fully out of the default dependency
graph (reqwest/oauth2moved to rustls with the system trust store) —
this also unblocks manylinux Python-wheel builds; thecargo denygate was
repaired (config had rotted unparseable) and now enforces hard bans on
openssl/native-tls, an explicit license allowlist, and remediated
advisories (aws-lc-rs 1.17, bytes 1.12.1, rustls-webpki 0.103.13,
tokio-postgres 0.7.18, rand 0.8.7, time 0.3.53, crossbeam-epoch 0.9.20). - Python wheel:
heliosdb-nano-embedded(abi3, manylinux_2_28) now builds
in CI and publishes to PyPI via thepy-v*tag lifecycle. - The CI perf-gate baseline (
benches/public/ci_baseline.json) was
regenerated post-W1/W2 — the previous 2026-06-11 baseline sat 1.9–3.9×
below current throughput, leaving the 2.5× cliff-catcher unable to catch
anything.
v4.2.0
Waves 1–2 of the 2026-07 performance & stability campaign
(docs/plans/PERF_STABILITY_2026_07/), every item implemented via a reviewed
subagent workflow and gated per docs/GATES.md:
- Extended/prepared protocol unlock — parameterized autocommit reads no
longer serialize on the session-transaction mutex (atomic fast-out):
extended point-read 30k-TPS ceiling → 122k @ 64 clients (4.0×); prepared
34k → 202k @ 32 clients (5.9×) — the PostgreSQL driver-path gap is
reversed to ~2.4× in Nano's favor. - Extended-protocol Parse reuse — Describe metadata now derives from the
shared parameterized plan cache: extended point-read +10–18% on top of the
unlock (c64 115k → 132k TPS). View DDL (CREATE OR REPLACE / DROP VIEW) now
invalidates the plan cache (stale-Describe fix). - COPY fast path for FK/CHECK tables — bulk loads into constrained tables
no longer fall back to row-at-a-time validation: FK+CHECK COPY 100k rows
3172 ms → 326 ms (9.7×), with batched index FK probes (including
batch-local parents), CHECK parity with the slow path, and single-WriteBatch
all-or-nothing semantics. - Streaming COPY decode with bounded memory — the wire COPY path decodes
incrementally across protocol frames (partial-line/quote/UTF-8 state carried)
instead of buffering the whole stream; new[server] copy_max_buffered_rows
knob (0 = unlimited) aborts cleanly with zero rows applied when exceeded. - MVCC bookkeeping diet — prefix-bounded version scans, epoch-micros
version timestamps (permanent fallback deserializer keeps old on-disk data
readable), O(1) snapshot-cache invalidation, and unchanged-value
index-maintenance elision on UPDATE (PG-HOT-style). - In-transaction read watermark — reads inside a transaction of tables
unchanged since the snapshot serve from the normal fast read path instead of
the snapshot scan (30–150×); fail-closed policy (any version-skipping write
funnel invalidates back to the snapshot path). - Fix (branch isolation, wrong-data class): ~11 DML sites across both
execution engines maintained the process-wide value index for branch-routed
writes — phantom UNIQUE violations on main, and branch DELETEs stripping
main's index entries for inherited rows. All sites now gate on the same
predicate that routes branch data. - Fix: SQL/plan/result caches and the row cache survived
USE BRANCHand
could serve another branch's data; both are invalidated at branch switch. - Fix (planner):
ORDER BYover grouped plans uses the select-list
aggregate rewrite (was fragile alias-position slicing). - Fix:
TRUNCATEreports no affected-row count (PostgreSQL parity);
ALTER TABLEbumps the schema generation so stale fast-path caches cannot
leak backfilled values to open snapshot readers.
v4.1.0
Next performance batch (all gated through regression + scalability suites):
- COPY → PostgreSQL parity — the time-travel COPY fast batch now writes one
durablevmeta:range marker per batch instead of a per-rowv:/v_idx:
version pair. COPY 100k: ~397 ms → ~160 ms (2.5×), near PostgreSQL parity
(~115–133 ms). AS-OF visibility and insert-version materialization are wired
at every mutation path (fast update/delete, general branch-aware, the txn
commit-apply, and both TRUNCATE arms); an in-memory interval index is rebuilt
fromvmeta:at open (crash-safe) behind a one-atomic fast-out. Kill switch
HELIOS_COPY_VRANGE_OFF=1. (PR #13) - Normalizer widening —
IN/BETWEEN/ cast literals now parameterize for
shared cached plans, with IN-list power-of-two arity padding;INon an
indexed column gained an index multi-probe path (was a full filter-scan),
measured 100–161× on the wire. (PR #12) - Fix (embedded):
$nplaceholders inside a scalar subquery in
UPDATE … SET n = (SELECT … WHERE a=$1) …now bind (previously
"Parameter $1 not provided"). Wire path was never affected. Reported by a2h. (PR #14)
Also investigated and stopped with evidence (no code shipped): columnar OLAP
activation (#3 — shipped kernels measured best 3.5× / median 1.0× vs the
already-fast row store) and aggregate-over-join column pruning (#4 — pruning is
correct but the row-store scan full-decodes the blob, so it's a no-op; the real
lever is projected fast-skip decode in the join-input scan, deferred). See
docs/plans/NEXT_PERF_BATCH_2026_07.md.