Skip to content

Releases: HeliosDatabase/HeliosDB-Nano

v4.7.0

Choose a tag to compare

@github-actions github-actions released this 27 Jul 19:26

Transaction and shutdown correctness. One atomicity violation, plus three fixes
that together are the reason the engine's clean-shutdown work never ran in a
real deployment.

  • Fix (atomicity, pre-existing): ON DELETE CASCADE and ON DELETE SET NULL
    escaped the enclosing transaction. Both ran in their own autocommit
    transaction that committed immediately, so BEGIN; DELETE parent; ROLLBACK;
    restored the parent row while the cascaded child deletions — or the NULLed
    child FK columns — stayed permanently applied. The child effects now join the
    caller's transaction and roll back with it. Autocommit behavior is unchanged.
  • Fix (durability, pre-existing): at close, index snapshots were persisted
    before row counters were flushed. Because a valid index snapshot is what
    tells the next open that shutdown was clean, a crash between those two steps
    produced a database that looked cleanly shut down while carrying a stale row
    counter — and the next insert would reuse a live row id, overwriting data.
    This is the failure 4.6.1's counter reseed was meant to prevent. Counters are
    now flushed first.
  • Fix (shutdown, pre-existing): the server had no SIGTERM handler; only
    Ctrl+C was handled. Since the Unix default for an unhandled SIGTERM is
    immediate termination, no close-time work ran under systemctl stop,
    docker stop, Kubernetes pod termination — or heliosdb-nano stop, which
    sends SIGTERM itself and then waits two seconds for a graceful shutdown that
    could not occur. The documented way to stop a server was an abrupt kill.
    SIGTERM and SIGINT now follow the same shutdown path, and the log records
    which signal arrived.
  • Fix (shutdown, pre-existing): the HTTP endpoint's accept loop held a
    reference to the database and was never shut down, so the database was never
    dropped and its close-time work never ran under the default --http-port 8080
    — not even on Ctrl+C. Without this fix the two items above have no effect on
    a default heliosdb-nano start.

Upgrade note: if you relied on cascaded deletes persisting after a rolled-back
parent DELETE — behavior no standard SQL database exhibits — that no longer
happens. Servers now do measurable work on SIGTERM; if you run under a process
supervisor with a very short kill timeout, verify it allows shutdown to finish.
heliosdb-nano stop still hardcodes a 2-second grace period before SIGKILL,
which a large database may exceed; making it configurable is tracked for 4.8.

Also adds docs/plans/ROADMAP_V5.md, an audited inventory of known outstanding
issues sequenced toward 5.0. It documents several limitations candidly,
including that row-level security is currently not enforced on write paths.

v4.6.3

Choose a tag to compare

@github-actions github-actions released this 27 Jul 16:34

Constraint enforcement is now identical across both DML executor families.

HeliosDB Nano runs two parallel DML paths, selected by the wire frame type: a
text family (PostgreSQL simple-query, all MySQL wire, the REPL, the embedded
execute() API) and a params family (PostgreSQL extended protocol, plus every
REST/BaaS write). The params family had drifted, and three constraint checks the
text family performed were missing from it. Any client that uses the extended
protocol — psycopg2 server-side cursors, JDBC, sqlx, Drizzle, node-postgres — or
that writes through /rest/v1/, could bypass them.

  • Fix (data integrity, pre-existing): a parameterized DELETE did not
    enforce referencing (inbound) foreign keys. NO ACTION / RESTRICT were
    not rejected, and ON DELETE CASCADE / SET NULL never ran — so deleting a
    parent row through the extended protocol silently orphaned its children, with
    no error and no cascade. The same statement over the simple-query protocol
    behaved correctly, which made this hard to spot: the bug depended on the
    client driver, not the SQL.
  • Fix (data integrity, pre-existing): a parameterized UPDATE did not
    enforce UNIQUE, for either single-column constraints or multi-column table
    constraints, admitting duplicate keys.
  • Fix (data integrity, pre-existing): INSERT ... ON CONFLICT DO UPDATE
    never revalidated the post-merge row on either family. CHECK constraints
    were not re-evaluated — a conflict update could set a column to a value the
    table's own CHECK forbids — and foreign keys on the updated row went
    unverified.

The three enforcement blocks now live in shared helpers called from both
families rather than in two implementations that can diverge again. Error
messages are unchanged. New suite tests/constraint_parity_tests.rs runs each
affected statement through both families and asserts they agree.

Known limitations left unchanged by this release, each pre-existing and tracked
separately: ON CONFLICT DO UPDATE does not re-check NOT NULL (matching the
general UPDATE arm); the UNIQUE self-collision guard tests "value changed"
rather than "different row", so a same-statement key swap or cycle is rejected;
ON DELETE CASCADE / SET NULL run in their own autocommit transaction, so
child-row effects survive a ROLLBACK of the parent statement; and the UNIQUE
probe reads the branch-blind shared index.

v4.6.2

Choose a tag to compare

@github-actions github-actions released this 26 Jul 18:02

Three fixes closing out the latent-bug backlog. The wire-protocol one is the
significant find:

  • Fix (wire-protocol correctness, pre-existing): the PostgreSQL catalog
    dispatch matched pg_type / pg_tables / pg_views / pg_settings /
    pg_indexes / information_schema. as bare substrings of the raw statement
    text, so a statement that merely mentioned one — inside a string literal,
    a SQL comment, or as part of a longer identifier — had its real semantics
    discarded and a canned catalog response returned instead. Worst case, an
    UPDATE ... SET note='see pg_tables' silently never executed while the
    client received a reply it could not distinguish from success; a
    CREATE TABLE pg_type_registry (...) silently created nothing; and a user
    table named app_pg_settings was permanently shadowed. Interception is now
    restricted to read statements, matched against a literal- and
    comment-stripped view of the query, with word-boundary-aware marker
    matching. Client introspection (psql \dt, SQLAlchemy/pgAdmin/DBeaver/
    Drizzle probes) is unaffected.
  • SIMILAR TO now matches PostgreSQL semantics. Three bugs fixed:
    alternation was not bound by the implicit full-string anchoring (so
    'zzxyz' SIMILAR TO 'abc|xyz' wrongly returned true); the repetition
    quantifiers * + ? {m,n} were treated as literal characters rather
    than metacharacters (so 'aaa' SIMILAR TO 'a+' wrongly returned false);
    and the ESCAPE clause was parsed but silently ignored. ESCAPE 'x',
    ESCAPE '' (escaping disabled), and the default backslash escape all
    behave per PostgreSQL now. This surface previously had no test coverage at
    all; it now has 10 tests.
  • Fix: bulk loads persisted their row-id counter to a key the engine never
    reads back on open, leaving the canonical counter stale. It now writes
    through the canonical path — relevant for crash recovery of internal
    bookkeeping tables, which are skipped by the counter reseed added in 4.6.1.

v4.6.1

Choose a tag to compare

@github-actions github-actions released this 21 Jul 06:54

Four latent-bug fixes, none release-blocking on their own but worth a prompt
patch given the severity of the wire-protocol issue:

  • Fix (silent data corruption, pre-existing): a hard crash (kill -9,
    power loss, abort — anything that skips a clean shutdown) could leave a
    table's row-id counter stale if fewer than 64 rows had been inserted since
    its last periodic flush. On reopen, the next INSERT would then reuse an
    already-in-use row id, silently overwriting the pre-existing row. Fixed by
    reconciling the counter against the actual max row id scanned whenever the
    index-rebuild-on-open path runs without a valid snapshot (which happens
    precisely on a crash reopen, and is a no-op on a clean one).
  • Fix (wire-protocol correctness hazard, pre-existing): the PostgreSQL
    wire handler intercepted any statement whose raw text merely contained
    version(), current_database(), or current_user — anywhere, including
    inside a WHERE clause — and answered with a hardcoded canned reply instead
    of executing the real statement. In the worst case, an UPDATE/DELETE
    referencing one of these in its WHERE clause never executed at all, while
    the client received a reply indistinguishable from a real "0 rows matched"
    result. Fixed by removing the interceptors; the real query engine already
    answers all three correctly and session-aware.
  • ~ !~ ~* !~* (PostgreSQL POSIX regex match operators) and the raw
    ~~ ~~* !~~ !~~* operator spellings of LIKE/ILIKE
    are now
    implemented — previously errored with "Binary operator not yet supported"
    for every case, including in WHERE clauses.
  • CREATE SCHEMA name CREATE TABLE ... CREATE TABLE ... (PostgreSQL's
    multi-element schema-creation form) is now accepted — previously any
    statement with more than one embedded element failed to parse entirely.
    Bare names inside the block (including cross-references between sibling
    tables, e.g. REFERENCES/PARTITION OF) resolve into the new schema; the
    whole statement is all-or-nothing (a failure partway through leaves nothing
    behind, matching PostgreSQL).

v4.6.0

Choose a tag to compare

@github-actions github-actions released this 19 Jul 05:32

Two schema Stage-2 follow-up items (leased to and delivered by the fleet's
HDB session) plus a round-3 cheap-root cascade:

  • PostgreSQL NaN/Infinity/-Infinity NUMERIC support — the three
    special values are now accepted end-to-end (cast, comparison, equality,
    sort, arithmetic, aggregates, predicate pushdown), matching PostgreSQL's
    numeric.c contract: NaN sorts greater than all non-NaN values and
    NaN = NaN is TRUE (deliberately, for btree/GROUP BY/DISTINCT/
    ORDER BY usability, unlike IEEE-754); +Infinity/-Infinity compare and
    propagate through arithmetic as PostgreSQL does (Inf-Inf, Inf/Inf,
    Inf*0 = NaN; finite/Inf = 0; Inf/0 still raises division-by-zero).
  • CREATE FUNCTION … RETURNS TABLE(cols) — the composite return form is
    now accepted (previously rejected with "Custom data type not yet
    supported"); the column list is not yet persisted and set-returning
    execution is not yet wired (function calls keep existing scalar behavior).
  • Real catalog schema truth (pg_namespace/relnamespace)
    pg_namespace lists actual registered schemas (no longer hardcoded to
    public); pg_class.relnamespace maps each relation to its real schema's
    oid via a stable, deterministic name→oid map, so the two catalogs join
    consistently. current_schema() is now session-aware; current_schemas(bool)
    added.
  • Multi-entry search_pathSET search_path TO a, b now resolves bare
    table references by walking the full ordered list (first match wins),
    replacing the prior first-non-public-entry-only behavior; SHOW search_path
    keeps the established append-public display convention.
  • Fix (pre-existing): cross-type foreign keys on the INSERT ... SELECT
    path reported phantom violations (the twin of the direct-INSERT fix in
    4.5.0); now routes through the same type-aware validator, which also fixes
    deferred-FK skip on that path.
  • Measured on the PG regression corpus: +14 statements flip to passing across
    both handbacks, 0 regressions.

v4.5.0

Choose a tag to compare

@github-actions github-actions released this 18 Jul 21:20

Real schema support:

  • Schemas coexist — same-named tables in different schemas are now
    distinct objects (a.t and b.t no longer collide). Public-schema
    behavior and existing data directories are unchanged (fully backward
    compatible).
  • search_path is honored per sessionSET/SHOW/RESET search_path
    work over the wire and embedded; bare names resolve to the session's
    schema, falling back to public; connection isolation guaranteed (two
    connections with different search_paths cannot see each other's
    resolution).
  • Schema DDLCREATE SCHEMA [IF NOT EXISTS] registers (duplicates
    error); DROP SCHEMA [IF EXISTS] … RESTRICT|CASCADE (cascade drops member
    tables, composing with the partition-child cascade); ALTER TABLE … SET SCHEMA moves tables between schemas.
  • Deferred constraintsSET CONSTRAINTS {ALL|names} {DEFERRED| IMMEDIATE} implemented (wire + embedded); column-level INITIALLY DEFERRED honored; deferred FK checks validate at COMMIT and evaporate if
    the referenced table was dropped in-transaction (PG parity).
  • Introspectioninformation_schema reports real table_schema
    values; pg_class.relname stays bare.
  • Fix (wrong-data, pre-existing): the first INSERT after any
    ALTER TABLE … RENAME silently overwrote the oldest row (volatile row
    counter was not migrated with the rename).
  • Fix (pre-existing): cross-type foreign keys (e.g. int8 child
    referencing int4 parent) reported phantom violations on the indexed
    fast path; probe values now coerce to the referenced columns' types.
  • Measured on the PG regression corpus: +158 statements flip to passing;
    ~60 prior "passes" that depended on cross-schema name collisions now
    correctly error (they were silently operating on the wrong tables).

v4.4.0

Choose a tag to compare

@github-actions github-actions released this 18 Jul 00:57

Wave-3 implementation (leased to and delivered by the fleet's HDB session) and
round-3 declarative-partitioning Stage 0:

  • PARTITION BY support (Stage 0) — PostgreSQL declarative-partitioning DDL
    is now accepted: CREATE TABLE … PARTITION BY RANGE/LIST/HASH parents
    (including multi-column, expression, and opclass keys), CREATE TABLE … PARTITION OF … FOR VALUES/DEFAULT children (columns cloned from the
    parent), and ATTACH/DETACH PARTITION as accepted no-ops. DROP TABLE parent cascades to its partition children (PG parity);
    pg_class.relpartbound is present (NULL). Stage-0 semantics: each child
    is an independent table — INSERT into the parent is not routed and SELECT
    from the parent does not union children yet (Stage 1). Measured on the
    PG regression corpus: +1,427 statements flip to passing.
  • Version-copy elision — new [storage] elide_latest_version (default
    off): the latest row version lives only in data:, cutting single-INSERT
    version-write volume from 65% to 41% of bytes. One-way door: once
    enabled on a data dir, downgrade requires dump/restore.
  • Statement retry on write conflicts[locks] statement_retry_max
    (default 0 = off) auto-retries autocommit statements that hit the typed
    SQLSTATE 40001 write conflict; backoff knobs included.
  • Snapshot schema evolution[storage] snapshot_schema_evolution now
    defaults to "null_pad": reads through a snapshot taken before an ALTER TABLE ADD/DROP COLUMN return isolation-correct NULL-padded/truncated rows
    instead of erroring. Set "strict" to restore the previous error behavior.
  • [locks] timeout_ms is now genuinely wired to the lock manager
    (previously orphaned); default 1000 preserves the prior effective bound.
  • Fix (durability): the fast-INSERT row counter is flushed at clean
    shutdown — prevents silent row overwrites on reopen after short sessions in
    relaxed-WAL mode (crash-path reseed tracked separately).
  • COPY constraint batching: ART maintenance batched per-table; CHECK
    expressions evaluated via a batched path (constrained-COPY follow-up from
    the Wave-3 measurements).

v4.3.0

Choose a tag to compare

@github-actions github-actions released this 17 Jul 09:33

Wave 3 (design-first) of the 2026-07 perf & stability campaign, the July
compat round 2, and a supply-chain hardening pass:

  • Typed write-conflict errors — same-row write conflicts now surface as a
    typed WriteConflict error carrying table/row/holder context and map to
    SQLSTATE 40001 (serialization_failure) on the wire (previously a
    generic 25000), so PostgreSQL drivers' retry machinery engages. Timeout and
    locking behavior are unchanged.
  • Performance attribution instrumentation (all zero-cost when disabled):
    lock-contention census behind the new lock-census cargo feature +
    [performance] lock_census knob (heliosdb_lock_census system view);
    per-statement-class write-volume byte accounting ([performance] write_volume_statsheliosdb_write_volume); COPY wall-time phase
    breakdown ([performance] copy_phase_statsheliosdb_copy_phase_stats).
    Measured verdicts and next-step designs live in
    docs/plans/PERF_STABILITY_2026_07/W3_*_DESIGN.md (notable: the version
    chain is 65% of single-INSERT byte volume; CHECK evaluation is 35% of
    constrained-COPY time; the c≥32 wire plateau is not mutex-blocking).
  • Compatibility round 2 (12 fixes + 1 hardening): CREATE/DROP DOMAIN
    no-op intercepts, CREATE TABLE … INHERITS clause stripping, and other
    pg-dialect gaps; catch_unwind isolation extended to the extended-protocol
    DML-RETURNING path (an erroring UPDATE … RETURNING no longer kills the
    connection).
  • Supply chain: the openssl stack is fully out of the default dependency
    graph (reqwest/oauth2 moved to rustls with the system trust store) —
    this also unblocks manylinux Python-wheel builds; the cargo deny gate was
    repaired (config had rotted unparseable) and now enforces hard bans on
    openssl/native-tls, an explicit license allowlist, and remediated
    advisories (aws-lc-rs 1.17, bytes 1.12.1, rustls-webpki 0.103.13,
    tokio-postgres 0.7.18, rand 0.8.7, time 0.3.53, crossbeam-epoch 0.9.20).
  • Python wheel: heliosdb-nano-embedded (abi3, manylinux_2_28) now builds
    in CI and publishes to PyPI via the py-v* tag lifecycle.
  • The CI perf-gate baseline (benches/public/ci_baseline.json) was
    regenerated post-W1/W2 — the previous 2026-06-11 baseline sat 1.9–3.9×
    below current throughput, leaving the 2.5× cliff-catcher unable to catch
    anything.

v4.2.0

Choose a tag to compare

@github-actions github-actions released this 17 Jul 03:25

Waves 1–2 of the 2026-07 performance & stability campaign
(docs/plans/PERF_STABILITY_2026_07/), every item implemented via a reviewed
subagent workflow and gated per docs/GATES.md:

  • Extended/prepared protocol unlock — parameterized autocommit reads no
    longer serialize on the session-transaction mutex (atomic fast-out):
    extended point-read 30k-TPS ceiling → 122k @ 64 clients (4.0×); prepared
    34k → 202k @ 32 clients (5.9×) — the PostgreSQL driver-path gap is
    reversed to ~2.4× in Nano's favor.
  • Extended-protocol Parse reuse — Describe metadata now derives from the
    shared parameterized plan cache: extended point-read +10–18% on top of the
    unlock
    (c64 115k → 132k TPS). View DDL (CREATE OR REPLACE / DROP VIEW) now
    invalidates the plan cache (stale-Describe fix).
  • COPY fast path for FK/CHECK tables — bulk loads into constrained tables
    no longer fall back to row-at-a-time validation: FK+CHECK COPY 100k rows
    3172 ms → 326 ms (9.7×)
    , with batched index FK probes (including
    batch-local parents), CHECK parity with the slow path, and single-WriteBatch
    all-or-nothing semantics.
  • Streaming COPY decode with bounded memory — the wire COPY path decodes
    incrementally across protocol frames (partial-line/quote/UTF-8 state carried)
    instead of buffering the whole stream; new [server] copy_max_buffered_rows
    knob (0 = unlimited) aborts cleanly with zero rows applied when exceeded.
  • MVCC bookkeeping diet — prefix-bounded version scans, epoch-micros
    version timestamps (permanent fallback deserializer keeps old on-disk data
    readable), O(1) snapshot-cache invalidation, and unchanged-value
    index-maintenance elision on UPDATE (PG-HOT-style).
  • In-transaction read watermark — reads inside a transaction of tables
    unchanged since the snapshot serve from the normal fast read path instead of
    the snapshot scan (30–150×); fail-closed policy (any version-skipping write
    funnel invalidates back to the snapshot path).
  • Fix (branch isolation, wrong-data class): ~11 DML sites across both
    execution engines maintained the process-wide value index for branch-routed
    writes — phantom UNIQUE violations on main, and branch DELETEs stripping
    main's index entries for inherited rows. All sites now gate on the same
    predicate that routes branch data.
  • Fix: SQL/plan/result caches and the row cache survived USE BRANCH and
    could serve another branch's data; both are invalidated at branch switch.
  • Fix (planner): ORDER BY over grouped plans uses the select-list
    aggregate rewrite (was fragile alias-position slicing).
  • Fix: TRUNCATE reports no affected-row count (PostgreSQL parity);
    ALTER TABLE bumps the schema generation so stale fast-path caches cannot
    leak backfilled values to open snapshot readers.

v4.1.0

Choose a tag to compare

@github-actions github-actions released this 06 Jul 19:44

Next performance batch (all gated through regression + scalability suites):

  • COPY → PostgreSQL parity — the time-travel COPY fast batch now writes one
    durable vmeta: range marker per batch instead of a per-row v:/v_idx:
    version pair. COPY 100k: ~397 ms → ~160 ms (2.5×), near PostgreSQL parity
    (~115–133 ms).
    AS-OF visibility and insert-version materialization are wired
    at every mutation path (fast update/delete, general branch-aware, the txn
    commit-apply, and both TRUNCATE arms); an in-memory interval index is rebuilt
    from vmeta: at open (crash-safe) behind a one-atomic fast-out. Kill switch
    HELIOS_COPY_VRANGE_OFF=1. (PR #13)
  • Normalizer wideningIN / BETWEEN / cast literals now parameterize for
    shared cached plans, with IN-list power-of-two arity padding; IN on an
    indexed column gained an index multi-probe path (was a full filter-scan),
    measured 100–161× on the wire. (PR #12)
  • Fix (embedded): $n placeholders inside a scalar subquery in
    UPDATE … SET n = (SELECT … WHERE a=$1) … now bind (previously
    "Parameter $1 not provided"). Wire path was never affected. Reported by a2h. (PR #14)

Also investigated and stopped with evidence (no code shipped): columnar OLAP
activation (#3 — shipped kernels measured best 3.5× / median 1.0× vs the
already-fast row store) and aggregate-over-join column pruning (#4 — pruning is
correct but the row-store scan full-decodes the blob, so it's a no-op; the real
lever is projected fast-skip decode in the join-input scan, deferred). See
docs/plans/NEXT_PERF_BATCH_2026_07.md.