Skip to content

Releases: FlavioCFOliveira/Graphus

Graphus v0.0.9

Choose a tag to compare

@FlavioCFOliveira FlavioCFOliveira released this 08 Jul 11:01

Graphus v0.0.9

Version: 0.0.9
Date: 2026-07-08

Summary

Graphus v0.0.9 is a feature release for Graphus, the Label Property Graph (LPG) database
server written in Rust for extreme load and concurrency. It makes the server fit the hardware it
runs on out of the box
and interoperate with strict or legacy Neo4j drivers, while keeping the
honest defaults, the conformance guarantees, and the drop-in upgrade path intact.

Two capabilities are added and one defect is fixed:

  • Hardware-aware startup auto-tuning. At startup the server probes the host — logical CPUs,
    physical and available RAM, and the store filesystem's capacity, free space, and an
    SSD-versus-rotational hint — and sizes its resource-hungry parameters to the real hardware. The
    buffer pool, previously a fixed 32 MiB regardless of RAM, now auto-sizes to roughly one eighth of
    available RAM (clamped to a safe range); the reader and morsel pools resolve to the number of
    cores, capped. Any value set in the TOML file or a GRAPHUS_* environment variable overrides the
    auto-detection — operator configuration always wins.
  • Configurable Bolt server agent. Graphus keeps announcing the honest, fully conformant
    Graphus/<version> in the Bolt HELLO reply by default, but a new opt-in option lets operators
    advertise a Neo4j-compatible agent string for strict or legacy drivers that verify it. This changes
    only the advertised string; it never alters Bolt or PackStream behaviour.
  • Fixed a tiny buffer-pool re-entrancy abort. A pathologically small buffer pool could abort the
    process with a RefCell already borrowed panic during a re-entrant eviction. It now fails closed
    with a clean, recoverable error, and an explicit sub-minimum pool size is rejected at startup.

This is a drop-in upgrade from v0.0.8: no public Cypher, Bolt, or REST
contract changed, the two new capabilities are opt-in and default to the previous behaviour, and no
existing configuration needs to change.

Throughout, the four inviolable guarantees held:

  • 100% ACID — full transactional reliability under power loss, faults, and crashes.
  • 100% openCypher TCK3914 / 3914 scenarios passing.
  • 100% Bolt protocol — byte-for-byte interoperability with the Neo4j driver ecosystem.
  • 100% PackStream — exact wire-level serialization.

Highlights

  • The server fits its hardware automatically. On first start Graphus detects the host's CPUs,
    RAM, and store filesystem, then sizes the buffer pool, reader pool, and morsel parallelism to what
    it finds — instead of shipping a fixed 32 MiB buffer pool on every machine from a Raspberry Pi to a
    many-core server.
  • Operator configuration always wins. Auto-tuning is driven by a 0 = auto sentinel: any value
    set in the TOML file or a GRAPHUS_* environment variable (including the new
    GRAPHUS_BUFFER_POOL_PAGES) overrides the detected default verbatim.
  • Never worse than before. The auto-sized buffer pool floor equals the previous fixed default and
    its ceiling bounds worst-case resident memory, so the change can only match or improve on prior
    behaviour.
  • Interoperability with strict or legacy Neo4j drivers. A new opt-in startup option advertises a
    vetted Neo4j/5.13.0 (or any value you choose) in the Bolt HELLO reply, so drivers and tooling
    that verify the server agent string accept Graphus — without changing Bolt or PackStream
    behaviour.
  • Safety kept #![forbid(unsafe_code)]. Hardware detection lives in a new, separately-audited
    leaf crate, graphus-sysres, which isolates the single platform-specific unsafe system call so
    the graphus-server crate remains free of unsafe code. Every probe degrades gracefully and never
    panics or blocks startup.
  • A tiny buffer pool no longer aborts the process. A re-entrancy hazard that could panic under a
    pathologically small pool now fails closed with a clean, recoverable error while preserving
    write-ahead-log-before-data ordering.

Added

  • Hardware-aware startup auto-tuning (rmp #617, decision D-hw-autotune). At startup the server
    probes the host — logical CPUs, physical/available RAM, and the store filesystem's capacity/free
    space plus an SSD-vs-rotational hint — and sizes its resource parameters to the real hardware, while
    any value set in the TOML file or a GRAPHUS_* environment variable overrides the auto-detection.
    The buffer pool (previously a fixed 32 MiB regardless of RAM) auto-sizes to roughly one eighth of
    available RAM, clamped to [32 MiB, 2 GiB]; the reader and morsel pools resolve to min(cpus, 16).
    Auto-tuning is driven by a 0 = auto sentinel (including the new GRAPHUS_BUFFER_POOL_PAGES), so
    operator configuration always wins. The auto-size is computed by a pure, deterministic step
    (apply_hardware_defaults) run before validation and before any store opens. Detection lives in a
    new, separately-audited leaf crate, graphus-sysres, that isolates the single macOS unsafe
    sysctl call, keeping graphus-server #![forbid(unsafe_code)]. A single structured startup log
    line reports what was detected and how each parameter resolved (auto vs config). See
    docs/configuration.md for the full parameter table and the
    auto-tuning rules.
  • Configurable Bolt server agent for legacy/strict Neo4j-driver interoperability
    (bolt_server_agent / GRAPHUS_BOLT_SERVER_AGENT, rmp #614).
    By default Graphus keeps
    announcing Graphus/<version> in the Bolt HELLO SUCCESS — 100% Bolt-conformant and accepted by
    every modern Neo4j driver. Some strict or legacy clients (1.x-era drivers and tooling derived from
    them) instead verify this string and reject any product that is not the case-sensitive literal
    Neo4j. The new opt-in startup option controls the advertised agent: unset or blank → the
    Graphus/<version> default; the neo4j-compat shortcut → the vetted Neo4j/5.13.0; any other
    value → announced verbatim (for example Neo4j/5.13.0-graphus-<version>). Neo4j/5.13.0 is the
    floor of the Neo4j version window whose native Bolt maximum matches Graphus's negotiated Bolt 5.4
    (Bolt 5.4 ⇒ Neo4j 5.13–5.22, per the official compatibility matrix), so no version-keyed client
    assumes Bolt features Graphus does not serve; a regression test couples this compatibility constant
    to the handshake's MAX_MINOR so a future Bolt bump fails loudly. Announcing Neo4j/… does not
    change Bolt/PackStream conformance nor unlock capabilities — drivers gate features on the negotiated
    Bolt version, never on this string. A startup warning fires (warning only, never a rejection) when
    the configured value claims Neo4j identity in a shape a strict or legacy parser would reject. See
    docs/bolt.md and docs/configuration.md.

Fixed

  • Tiny buffer-pool re-entrancy abort (rmp #302). Building a buffer pool with a pathologically
    small capacity could abort the process with a RefCell already borrowed panic during a re-entrant
    eviction. This was root-caused (proven empirically) to the index SharedWal::ensure_durable, which
    performed a re-entrant borrow_mut when a caller held the WAL latch across an evicting page
    write-back — the classic ARIES "don't nest the buffer-pool flush under the log latch" hazard. It now
    fails closed with a clean, recoverable error instead of aborting, preserving
    write-ahead-log-before-data ordering: the error short-circuits before the home write, so no page is
    written home ahead of its log record and the page stays resident and dirty. As defence in depth,
    an explicit buffer_pool_pages below the documented minimum (64) is now rejected at startup with a
    clear message rather than reaching the pool, and an internal safety net ensures an unresolved config
    can never reach store-open with a zero-page pool. Regression tests were added at pool sizes 1, 2, 3,
    and 4 across the buffer-pool and index-recovery paths.

Verification

This release was validated empirically; no functional test reported an error. The four inviolable
guarantees are unchanged — the openCypher TCK remains at 3914 / 3914 (100%), and the ACID, Bolt,
and PackStream guarantees hold.

  • Hardware-aware auto-tuning. The new graphus-sysres crate is covered by 9 unit tests and 2
    documentation tests, and the graphus-server configuration layer adds a deterministic auto-tuning
    battery (clamp arithmetic, floor and ceiling, override precedence, and idempotence) alongside
    dedicated hardware-detection tests. It was verified end to end on the real binary in three modes:
    auto-detection (a 2 GiB buffer pool selected on a 27 GiB host), an environment-variable override
    (operator config wins over the detected default), and a sub-minimum value (a clean fail-fast error,
    with no panic). The graphus-sysres crate — which isolates the single platform unsafe sysctl
    call — received a clean security review (SECURE).
  • Configurable Bolt server agent. Unit tests cover the agent resolver (default, the neo4j-compat
    shortcut, and verbatim values), input normalization, and the warning heuristic (including
    case-variant and truncated-version inputs). Core Bolt tests assert the custom agent is reflected in
    the HELLO SUCCESS and that the compatibility constant is coupled to the handshake's MAX_MINOR.
    An end-to-end test drives a real Bolt client that reads Neo4j/5.13.0 back from a live server's
    HELLO. The change was certified GO by the Bolt-protocol specialist: no HELLO/PackStream
    regression, a spec-legal value safe for the widest driver ecosystem, and a warning heuristic with
    zero false positives.
  • Tiny buffer-pool re-entrancy fix. RED→...
Read more

Graphus v0.0.8

Choose a tag to compare

@FlavioCFOliveira FlavioCFOliveira released this 07 Jul 17:09

Graphus v0.0.8

Version: 0.0.8
Date: 2026-07-07
Tag: v0.0.8

Summary

Graphus v0.0.8 is a Bolt-protocol / connection-pool safety fix for Graphus, the Label
Property Graph (LPG) database server written in Rust for extreme load and concurrency. It restores
safe reuse of pooled connections after a statement error inside an explicit
BEGIN … COMMIT transaction — the exact pattern the official Neo4j drivers rely on when they run
managed transactions against a connection pool.

After any statement error inside an explicit transaction — even a trivial RETURN 1/0 — the Bolt
RESET failed to abort the executor's still-open transaction, so the connection was permanently
poisoned: every subsequent BEGIN on it failed with
Neo.TransientError.Transaction.Outdated ("a transaction is already open"). The root cause was a
state-machine gap: a statement failure moves the Bolt state machine to Failed, which erases the
TxReady / TxStreaming marker that RESET used to decide whether to roll back — so RESET
skipped the rollback in exactly the case where it was needed most. Because the official Neo4j
drivers reuse connections from a pool, one poisoned connection returned to the pool and failed the
work of other, unrelated sessions, so session.execute_write(...) could not converge under
contention. RESET now rolls back the executor's open transaction unconditionally, in
conformance with the Bolt RESET specification ("set the connection back to its initial state …
stopping any unit of work").

This is a drop-in upgrade from v0.0.7: the transaction, isolation, Cypher,
and data contracts are unchanged, and no new user-facing feature was added. The only behavioural
change is that RESET now reliably returns a connection to a clean READY state. ACID was never
at risk
— no transaction ever committed partially and no data was lost; only connection reuse
after an in-transaction error was broken.

Throughout, the four inviolable guarantees held:

  • 100% ACID — full transactional reliability under power loss, faults, and crashes.
  • 100% openCypher TCK3914 / 3914 scenarios passing.
  • 100% Bolt protocol — byte-for-byte interoperability with the Neo4j driver ecosystem.
  • 100% PackStream — exact wire-level serialization.

Highlights

  • Pooled connections survive an in-transaction error. After a statement fails inside an
    explicit BEGIN … COMMIT transaction, a RESET now aborts the underlying transaction and the
    connection is immediately reusable, instead of being permanently poisoned for every later BEGIN.
  • No cross-session contamination. A single connection that hit an in-transaction error can no
    longer return to the driver's pool and fail the work of other, unrelated sessions.
  • Managed retries converge under contention. With connections no longer poisoned by the
    serialization-conflict retries that v0.0.7 enabled,
    session.execute_write(...) now converges under contention as intended.
  • Bolt RESET conformance restored. RESET returns the connection to a clean READY state
    unconditionally, matching the specification ("stopping any unit of work"), across every path that
    reaches the Failed state from inside a transaction.
  • No ACID or data-contract change. Correctness was already intact; this release only fixes the
    connection-lifecycle handling of RESET.

Fixed

  • A pooled Bolt connection was permanently poisoned after any error inside an explicit
    transaction (Bolt-protocol conformance / connection-pool safety).
    A statement failure inside an
    explicit BEGIN … COMMIT transaction moves the Bolt state machine to Failed (via fail_with),
    which erases the TxReady / TxStreaming marker. handle_reset gated its executor.rollback()
    on exactly that marker, so a RESET issued after an in-transaction error skipped the
    rollback
    : the executor's transaction leaked and the connection was left holding an open
    transaction it could no longer see. Every subsequent BEGIN on that connection then failed with
    Neo.TransientError.Transaction.Outdated ("a transaction is already open"). Because the official
    Neo4j drivers reuse connections from a pool, a single poisoned connection returned to the pool and
    failed the work of other, unrelated sessions, so session.execute_write(...) could not
    converge under contention. handle_reset now calls executor.rollback_open_tx()
    unconditionally — a method that consults the executor's own current_tx, is a no-op when
    nothing is open, and is idempotent and infallible — so RESET returns the connection to a clean
    READY regardless of the Bolt state enum. This covers every path into Failed from inside a
    transaction: a RUN error in TxReady, a runtime error during PULL / DISCARD, or an
    out-of-order message. ACID itself was never at risk: no partial commit and no lost data ever
    occurred — only connection reuse after an in-transaction error was broken.

Verification

This release was validated empirically; no functional test reported an error.

  • Empirical, real-driver discovery. The defect was discovered and reproduced deterministically
    against the live server instance using the real neo4j Python driver 6.2.0: a plain
    RETURN 1/0 executed inside begin_transaction poisoned the connection, after which every
    managed execute_write(...) retry failed to converge under contention. This surfaced only after
    v0.0.7 enabled managed retry of serialization conflicts — the previous
    .Terminated poison-title behaviour made drivers discard the connection instead of reusing
    it, which masked the poisoning entirely.
  • RED→GREEN regression coverage. A new graphus-bolt regression drives the exact failing
    sequence — BEGIN, RUN 1/0 (FAILURE), RESET, BEGIN — and asserts that the second
    BEGIN now succeeds
    (it was a FAILURE, "a transaction is already open", before the fix). The
    existing mid-transaction rollback test was updated to assert the RESET-triggered
    rollback_open_tx of the open transaction.
  • Conformance gates intact. The openCypher TCK remains at 3914 / 3914 (100%), and the ACID,
    Bolt, and PackStream guarantees are unchanged. cargo test -p graphus-bolt is green, including
    the new regression, and cargo fmt is clean.

Compatibility

v0.0.8 is a drop-in upgrade from v0.0.7. The transaction and isolation model is exactly as in
v0.0.7 (MySQL-style autocommit reads at Snapshot Isolation, fully Serializable
writes and explicit BEGIN … COMMIT transactions; see
docs/transactions.md), and no new user-facing feature was added.

The only behavioural change is that a Bolt RESET now reliably rolls back an in-flight transaction
and returns the connection to a clean READY state, even when a previous statement failed inside
the transaction. No action is required — the official Neo4j drivers already issue a RESET
when returning a connection to their pool, so this fix takes effect transparently. Applications that
manually reuse a connection after an in-transaction error will simply find that the next BEGIN
now succeeds instead of failing with Neo.TransientError.Transaction.Outdated.

Upgrading

# Docker Hub (multi-arch: linux/amd64 + linux/arm64)
docker pull flaviocfo/graphus:v0.0.8             # or :latest

See docs/getting-started.md for first-run instructions,
docs/transactions.md for the read/write isolation model and how to retry
a serialization failure, and docs/bolt.md for the Bolt-over-TCP and
Bolt-over-UDS interfaces.

Graphus v0.0.7

Choose a tag to compare

@FlavioCFOliveira FlavioCFOliveira released this 07 Jul 16:07

Graphus v0.0.7

Version: 0.0.7
Date: 2026-07-07
Tag: v0.0.7

Summary

Graphus v0.0.7 is a Bolt / Neo4j-driver interoperability fix for Graphus, the Label
Property Graph (LPG) database server written in Rust for extreme load and concurrency. It restores
automatic retry of contended transactions for every application that uses the Neo4j driver
ecosystem's recommended managed-transaction pattern (session.execute_write(...) /
session.execute_read(...)).

Under a serialization conflict, Graphus correctly classified the aborted transaction as a
retriable TransientError, but emitted the error title Neo.TransientError.Transaction.Terminated.
That title is a driver poison title: every official Neo4j driver keeps a fixed
ERROR_REWRITE_MAP that rewrites it to a non-retriable Neo.ClientError.Transaction.Terminated,
regardless of the class the server sent. The retry logic therefore never fired, and every contended
transaction failed permanently for apps following the idiomatic pattern. The abort now carries
Neo.TransientError.Transaction.Outdated — the semantically-accurate optimistic-concurrency code
that drivers do not rewrite — across both the Bolt and REST interfaces.

This is a drop-in upgrade from v0.0.6: the transaction, isolation, Cypher,
and data contracts are unchanged, and no new user-facing feature was added. The only behavioural
change is the retriable error code returned on a conflict. ACID was never at risk — the conflict
was always resolved correctly, with no lost updates; only the driver-side retry signalling was
broken.

Throughout, the four inviolable guarantees held:

  • 100% ACID — full transactional reliability under power loss, faults, and crashes.
  • 100% openCypher TCK3914 / 3914 scenarios passing.
  • 100% Bolt protocol — byte-for-byte interoperability with the Neo4j driver ecosystem.
  • 100% PackStream — exact wire-level serialization.

Highlights

  • Contended transactions retry again on the official drivers. The idiomatic
    session.execute_write(...) / session.execute_read(...) managed-transaction functions once more
    recover automatically from a serialization conflict, instead of surfacing a permanent, non-retriable
    failure to the application.
  • Bolt ↔ REST parity preserved. Both interfaces share the same corrected retriable code, so a
    conflict looks identical over Bolt-over-TCP, Bolt-over-UDS, and the REST WebAPI.
  • Proven against a real driver on a live server. The fix was found and A/B-verified against the
    live instance with the real neo4j Python driver 6.2.0, exercising the driver's own classification
    path, and guarded by anti-poison-title regression tests on both interfaces.
  • No ACID or data-contract change. Correctness was already intact; this release only corrects the
    wire signal that tells a driver a conflict is safe to retry.

Fixed

  • Retriable serialization conflicts were poisoned for the Neo4j driver ecosystem (Bolt / driver
    conformance).
    Under a serialization conflict — an SSI dangerous-structure abort or a write–write
    conflict — both the Bolt and REST interfaces emitted the error title
    Neo.TransientError.Transaction.Terminated. The TransientError classification was correct
    (the failure is safe to retry), but .Terminated — like .LockClientStopped — is a driver
    poison title
    : every official Neo4j driver keeps a fixed ERROR_REWRITE_MAP that downgrades it to
    Neo.ClientError.Transaction.Terminated, a non-retriable ClientError, regardless of the class
    the server sent. As a result, the idiomatic managed-transaction retry (session.execute_write(...))
    never recovered from a serialization conflict, so every contended transaction failed permanently
    for applications following Neo4j's recommended pattern — a violation of the "100% Bolt-protocol /
    Neo4j-driver-ecosystem compatible" guarantee. ACID itself was never at risk: the conflict was always
    resolved correctly, with no lost updates. The abort now carries
    Neo.TransientError.Transaction.Outdated, the semantically-accurate optimistic-concurrency code
    ("transaction state invalidated by concurrent updates; retry may succeed"), which drivers do not
    rewrite — so a driver correctly sees is_retryable() == true. The retriable-code constant is renamed
    (CODE_TXN_CONFLICT_RETRYABLE) and documented with the reason, the Bolt (graphus-bolt) and REST
    (graphus-rest) code paths are kept byte-identical for cross-wire parity, and regression tests assert
    the emitted code ends with neither .Terminated nor .LockClientStopped on both interfaces.

Verification

This release was validated empirically; no functional test reported an error.

  • Empirical, real-driver proof. The defect was discovered and the fix A/B-verified against the
    live server instance using the real neo4j Python driver 6.2.0, driven through the driver's
    own classification path (_hydrate_neo4j): .Outdated resolves to is_retryable() == true, whereas
    the previous .Terminated resolved to false. This was surfaced by the real-driver stability run,
    not by a mock.
  • Regression coverage on both interfaces. New anti-poison-title regression asserts were added to
    graphus-bolt and graphus-rest, checking that a transaction abort still classifies as
    TransientError and that its title is neither .Terminated nor .LockClientStopped; the REST
    router integration test for a conflicting single-statement write was updated to expect the corrected
    retriable code on the 409 response.
  • Conformance gates intact. The openCypher TCK remains at 3914 / 3914 (100%), and the ACID,
    Bolt, and PackStream guarantees are unchanged. cargo test -p graphus-bolt -p graphus-rest is green,
    including the new regression asserts, and cargo fmt is clean.

Compatibility

v0.0.7 is a drop-in upgrade from v0.0.6. The transaction and isolation model is exactly as in
v0.0.6 (MySQL-style autocommit reads at Snapshot Isolation, fully Serializable
writes and explicit BEGIN … COMMIT transactions; see docs/transactions.md),
and no new user-facing feature was added.

The single behavioural change is the error code returned on a serialization conflict: over both Bolt
and REST it changes from Neo.TransientError.Transaction.Terminated to
Neo.TransientError.Transaction.Outdated. Both remain in the retriable TransientError class, so a
conformant Neo4j driver — which classifies by class, not by the exact title — now retries the conflict
automatically. No action is required unless an application special-cased the exact string
.Terminated; such code should match the retriable TransientError class (or the driver's
is_retryable()), not the specific title.

Upgrading

# Docker Hub (multi-arch: linux/amd64 + linux/arm64)
docker pull flaviocfo/graphus:v0.0.7             # or :latest

See docs/getting-started.md for first-run instructions,
docs/transactions.md for the read/write isolation model and how to retry a
serialization failure, and docs/rest-api.md for the REST WebAPI error codes.

Graphus v0.0.6

Choose a tag to compare

@FlavioCFOliveira FlavioCFOliveira released this 07 Jul 14:30

Graphus v0.0.6

Version: 0.0.6
Date: 2026-07-07
Tag: v0.0.6

Summary

Graphus v0.0.6 is a reliability-and-performance hardening release of Graphus, the
Label Property Graph (LPG) database server written in Rust for extreme load and concurrency.
It fixes a series of critical ACID and durability defects that could silently lose
committed data under extreme multi-core concurrency, bounds several memory and liveness
failure modes so that an adversarial or interrupted workload degrades gracefully instead of
aborting the server, and continues to scale reads, writes, and graph analytics across
cores
. It introduces no breaking changes and no new user-facing feature — the public
Cypher, Bolt, and REST contracts are unchanged, so v0.0.6 is a drop-in upgrade from
v0.0.5.

Throughout, the four inviolable guarantees held:

  • 100% ACID — full transactional reliability under power loss, faults, and crashes.
  • 100% openCypher TCK3914 / 3914 scenarios passing.
  • 100% Bolt protocol — byte-for-byte interoperability with the Neo4j driver ecosystem.
  • 100% PackStream — exact wire-level serialization.

Highlights

  • Committed data is safe under extreme multi-core concurrency. Three separate critical
    defects — an incremental garbage-collection pass that could silently forget a committed value
    written by an in-flight writer, a slot-reuse race that could make a committed edge invisible to
    a concurrent lock-free reader, and a free-list double-allocation on live rollback that could
    corrupt a property or incidence chain — are fixed, each with a gating regression test proven to
    fail before the fix. No crash is required to trigger any of them, and none leaves committed data
    at risk after the fix.
  • A healthy database always reopens. A transient device write error on a new page's unlogged
    seed flush can no longer leave an unrecoverable checksum-invalid page, a page torn during
    recovery is healed by its checksum, and crash recovery after a large or interrupted bulk-load no
    longer exhausts memory — so a database that shut down cleanly, crashed, or was force-detached
    comes back online without a larger host.
  • A single query can no longer take down the server. A deeply recursive Cypher query is now a
    bounded, recoverable error across every deep-recursion input shape rather than an uncatchable
    stack-overflow abort of the whole process, and hosted databases stay available.
  • Reads, writes, and analytics scale further across cores. A single heavy read is split across
    the reader pool with an adaptive morsel width, more concurrent committers share one durable
    fsync, seven more Graph Data Science algorithms run in parallel, and a group-by-over-expand
    aggregation is executed across cores.
  • Adversarial and slow clients degrade gracefully. Oversized .gcol uploads are bounded, a
    stalled reader can no longer pin garbage collection forever, and a slow consumer resumes across
    hardened group-commit batches instead of wedging.

Changed

  • A single heavy read now scales across cores. The off-thread reader pool uses an adaptive
    morsel width, so even one large frontier-seeded traversal — such as a friends-of-friends
    recommendation query — is split across the reader pool instead of running on a single worker
    thread.
  • Writes coalesce further under concurrency. Auto-commit write fsyncs are coalesced through
    group commit, and the commit pipeline is deepened past the previous batch-width ceiling, so more
    concurrent committers share a single durable fdatasync. Every durability invariant is
    preserved: a committer is acknowledged only after the fsync covering its commit record
    completes.
  • Graph Data Science algorithms run in parallel. The PageRank, weakly-connected-components,
    and degree-centrality parallelism gaps were closed, and seven previously single-threaded
    algorithms were parallelized across cores.
  • Parallel group-by aggregation. A group-by-over-expand aggregation — for example, counting
    each node's neighbours — is now executed across cores instead of on the engine thread alone.
  • Reader-safe procedures dispatched off-thread. Reader-safe procedures, including full-text
    and spatial procedures feeding an expansion, are dispatched to the off-thread reader pool and
    capture their index state in the read view, so they no longer serialize on the single engine
    thread (and no longer misfire a spurious "no such index" error when feeding an expansion).
  • Bounded write-ahead-log-to-store ratio. Background maintenance now runs at a
    store-proportional cadence built on an incremental, O(Δ) freeze sweep, bounding the
    WAL-to-store size ratio without reintroducing the O(store) per-checkpoint cost that a
    create-only workload would otherwise accrue as the store grows.
  • Durability invariants promoted to release-active. The two load-bearing durability
    invariants (doublewrite and seed-flush ordering) are now asserted in release builds, not only in
    debug builds.
  • Honest, multi-core example. The GDS-analytics example was corrected so that it demonstrates
    real multi-core execution rather than overstating it.
  • Leaner, better-scoped CI. The GitHub Actions pipeline no longer runs the DST / soak gate on
    plain feature-branch pushes (a feature branch reaches CI through its pull request), splits the
    fast cross-platform test job from a dedicated DST gate, adds incremental build caching, and
    cancels superseded runs. A trigger and permissions security audit added least-privilege
    contents: read permissions and hardened several workflow-input paths. The
    .github/dependabot.yml file was removed by request.

Fixed

  • Silent loss of a committed value under incremental garbage collection (critical, ACID). With
    an explicit BEGIN … COMMIT transaction in flight, a maintenance garbage-collection pass could
    advance its freeze frontier past the in-flight writer's records and then forget the writer, so
    once it committed a reader at the latest snapshot resolved its value as aborted — permanently
    losing a committed value with no crash required. The freeze frontier now tests live writer
    membership and keeps a writer's records unfrozen until it commits, restoring the
    freeze-before-prune ordering the prune's soundness depends on.
  • Lost committed edges under concurrent off-thread reads (critical, ACID isolation). A
    maintenance garbage collection could reclaim and reuse a record slot while a lock-free off-thread
    reader still held a pointer into it, so the reader decoded a foreign record and a committed live
    edge became invisible. A freed slot is now shadow-held from physical reuse until every
    transaction that predates the free has retired (an epoch / QSBR barrier), while remaining
    immediately reusable after a restart, when no readers are in flight. The path is allocation-free
    and deterministic when no reader holds a slot, so the deterministic simulator remains
    byte-identical.
  • Chain corruption from physical-id double-allocation on live rollback (critical, ACID
    durability).
    Under statement-interleaved SSI write load, restoring a rolling-back transaction's
    free list to the last durably-committed image could hand a freed id to a concurrent writer twice,
    producing a self-cyclic property or incidence chain and losing the data threaded below the cycle.
    Rollback now snapshots and restores the in-memory free lists around the catalog reload and
    withdraws only the aborting transaction's own pushes. Two related MVCC-stamp undo fixes landed
    alongside it: a non-LIFO-safe compare-and-set undo for tombstone stamps, and reclaiming an
    aborting transaction's own reused-id pops on rollback.
  • A transient write error could brick store reopen (critical, ACID / availability). A transient
    device error on a new page's unlogged seed flush left an all-zero, checksum-invalid page that no
    store mapped and no WAL record covered, so the next open failed checksum verification and a
    healthy database could never reopen. Cold-open reconstruction now classifies such an orphan page:
    an all-zero aborted-allocation phantom is safely skipped (crash recovery has already
    re-materialized every committed logged page), while a non-zero bad checksum still fails closed as
    genuine corruption — preserving the never-serve-an-untrusted-page mandate.
  • Torn page during crash recovery. A page torn mid-write while recovery is running is now healed
    by checksum-gating its page LSN, instead of propagating a partially written page.
  • Crash recovery could exhaust memory on a large or interrupted load (critical, recoverability).
    A large Mode A network bulk-import interrupted by a crash, kill, power loss, or force-detach left
    the entire retained WAL un-reclaimed, so crash recovery read gigabytes into memory and the
    out-of-memory killer aborted the reopen — committed data stayed correct but the database could
    not come back online without a larger host. Reopen memory is now bounded to the retained window
    for all large-WAL reopens, a mid-load maintenance pass reclaims the WAL prefix using the
    incremental freeze sweep, and a force-detach can no longer corrupt the store on reopen.
  • A deep query could stack-overflow-abort the whole server (critical, availability). A single
    authenticated Cypher query could drive native recursion deeper than the engine / reader-pool
    stack and trigger an uncatchable stack-overflow abort of the entire process, taking down every
    hosted database and connection with a crash-loop on retry. Query recursion depth is now bounded
    across all deep-recursion input shapes — runtime value nesting, clause chains, and long MATCH
    paths — and turned into a recoverable error; on aarch64 the runtime threads run on an 8 MiB stack
    so a deeply nested REST value cannot overflow the server.
  • **A stal...
Read more

Graphus v0.0.5

Choose a tag to compare

@FlavioCFOliveira FlavioCFOliveira released this 03 Jul 07:29

Graphus v0.0.5

Version: 0.0.5
Date: 2026-07-03
Tag: v0.0.5

Summary

Graphus v0.0.5 is a performance-and-hardening release of Graphus, the Label Property
Graph (LPG) database server written in Rust for extreme load and concurrency. It makes reads
and writes scale across cores, adds a network bulk-import path for loading data over
the wire, and closes a multi-vector production-readiness certification that fixed a
critical encryption-at-rest disk leak and several pre-authentication denial-of-service
vectors. It introduces no breaking changes — the public Cypher, Bolt, and REST contracts
are unchanged, so v0.0.5 is a drop-in upgrade from v0.0.4.

Throughout, the four inviolable guarantees held:

  • 100% ACID — full transactional reliability under power loss, faults, and crashes.
  • 100% openCypher TCK3914 / 3914 scenarios passing.
  • 100% Bolt protocol — byte-for-byte interoperability with the Neo4j driver ecosystem.
  • 100% PackStream — exact wire-level serialization.

Highlights

  • Lock-free reads that scale across cores. A standalone auto-commit read is now dispatched
    across the off-thread reader pool by its structural query type — so a bare MATCH sent
    without a routing hint is no longer pinned to the single engine thread — and runs as a
    lock-free Snapshot-Isolation snapshot read that takes no serializability overhead and can
    never cause a writer to abort. This is the MySQL / MariaDB / SQL-Server autocommit model:
    a transaction exists only when you open one.
  • Reads and writes scale with concurrency. REST reads now engage the reader pool,
    read-only transactions perform zero fdatasync, explicit-transaction commits batch into a
    single group-commit fdatasync, and that fsync is pipelined off the engine thread. On a
    concurrent read workload server CPU rose from roughly one core to nearly five; trivial reads
    climbed from a ~450 requests/s fsync-bound ceiling to tens of thousands per second; and
    committed-write throughput scales several-fold with concurrency on durable storage.
  • Network bulk-import. A new POST /admin/db/{db}/bulk-import streams CSV / .gcol data
    over the wire — Mode A for a fresh or empty database, Mode B for an already-live database
    concurrently with ordinary traffic — behind the Admin RBAC gate with quota, disk, and
    session-timeout protection.
  • Certification pass. A critical encrypted-at-rest WAL disk leak, a crash-recovery
    out-of-memory path, and multiple pre-authentication denial-of-service vectors were found and
    fixed, each with a gating regression test.

Added

  • Network bulk-import over REST. POST /admin/db/{db}/bulk-import streams CSV / .gcol
    data into a database without buffering the payload, gated by the same Admin RBAC check as
    BACKUP / RESTORE / CREATE DATABASE and protected by a per-byte quota, an ongoing
    free-disk check, and a session timeout (all configurable). Mode A loads a new or empty
    database through the low-level bulk-write path with a crash-durable per-batch checkpoint
    sentinel; Mode B loads an already-live, serving database concurrently with ordinary
    Bolt/REST traffic, its correctness coming entirely from participating in the same MVCC/SSI
    machinery every ordinary Cypher transaction uses, with automatic bounded retry of a batch
    that loses a serialization conflict. Ratified as decision D-bulk-import-network and
    specified in specification/08-network-bulk-import.md.
  • Product-recommendations example. examples/product-recommendations boots a real server,
    network-bulk-loads a recommendation multigraph over the wire, and drives a concurrency ladder
    of many simultaneous Bolt-over-UDS clients running a realistic read battery (direct-friend,
    second- and third-level, and collaborative-filtering traversals) plus a few concurrent
    writes, while sampling the server's CPU, RSS, and I/O to expose read-path bottlenecks —
    backed by a new deterministic generator and client tooling (graphus-reco-gen).
  • Server startup banner. A single structured startup line names the application, its
    version, the build platform (OS / architecture / pointer width), and the pid.
  • docs/transactions.md. Documents autocommit-by-default, explicit transactions, lock-free
    reads, the per-work isolation table, and how to opt a read back into serializable isolation.

Changed

  • Reads are lock-free and scale across cores; MySQL-style autocommit isolation. A standalone
    auto-commit read is dispatched off-thread by its structural query type (not the
    client-declared access mode) and demoted to Snapshot Isolation: it drops its serializability
    tracking and can never make a concurrent writer abort, while still reserving its MVCC snapshot
    so it reads a consistent view and pins the GC watermark for the versions it reads. Writes
    and explicit BEGIN … COMMIT transactions are untouched — full Serializable SSI.
    A read
    that needs serializable isolation opts in by running inside an explicit transaction. This is
    the InnoDB read-only model; see docs/transactions.md.
  • Read throughput scales with concurrency. The off-thread reader pool is now engaged for
    single-statement REST reads (previously every REST read ran inline on the engine thread), and
    a read-only transaction performs zero WAL append and zero fdatasync across its whole
    lifecycle. Measured: server CPU on a concurrent REST read workload rose from ~0.8 core (engine
    the sole hot thread) to ~4.8 cores with the reader pool engaged, and trivial reads scaled from
    ~450 requests/s to 36k–54k requests/s at concurrency 8–32.
  • Write throughput scales with concurrency. Explicit-transaction commits batch their commit
    records into a single write() + single fdatasync (cross-transaction group commit), and the
    batch fsync is offloaded to a dedicated sync thread so the engine overlaps it with preparing
    the next batch and retiring off-thread reads (commit pipelining). Committed-write throughput
    scales several-fold under concurrency on durable storage — the larger the fsync latency, the
    larger the gain — while a committer is acknowledged only after the fsync covering its commit
    record completes.
  • Lower per-statement engine cost. Compiled query plans are Arc-shared through the plan
    cache and executor, so a plan-cache hit ships a reference-count bump instead of a deep tree
    clone — a ~64–233× reduction in the per-statement clone cost that dominates the
    parameterized-repeated production case.
  • Cheaper bulk loading. During a Mode A bulk-import session the background
    maintenance-checkpoint interval is widened 16×, cutting the maintenance overhead that a
    create-only workload would otherwise accrue as the store grows; every other workload keeps the
    unchanged, frequent reclamation cadence.

Fixed

  • Encrypted write-ahead-log disk leak (critical). On an encryption-at-rest database the
    encrypted WAL stored its sink header in the first backing segment, which the prefix-only
    segment reclaimer could never free — so the encrypted WAL grew on disk without bound and a
    heavily-reclaimed log could not be reopened after a crash. The header is relocated into the
    never-deleted anchor (matching the plaintext layout); the on-disk sink version is bumped and
    an old-layout encrypted WAL is rejected fail-closed at open. AEAD framing, nonce-budget resume,
    and key-check fail-closed are preserved exactly.
  • Crash-recovery out-of-memory on a long-lived WAL. Recovery scans sized their buffer by the
    log's absolute lifetime rather than its small retained window, so a heavily-reclaimed log could
    abort on reopen with an out-of-memory error and leave the database unable to reopen (an ACID
    violation). Recovery now reads only from the reclaimed floor; behaviour is byte-identical when
    nothing has been reclaimed.
  • Pre-authentication denial-of-service vectors (PackStream). Decoding an attacker-supplied
    message before authentication could burn minutes of CPU (a many-key map decoded in O(N²) — now
    an order-preserving O(N) accumulator, ~780× faster) or amplify a few megabytes into gigabytes
    of heap (a deeply-sized collection with no breadth budget — now a per-message decoded-element
    budget caps decoded heap at a small multiple of the framing limit).
  • Availability and correctness hardening (certification pass). A Bolt PULL of a full
    result no longer buffers the entire result in the per-connection write buffer before flushing;
    an off-thread reader whose consumer stops draining now aborts at its statement deadline instead
    of blocking forever and pinning the GC watermark; the coordinator's serialization-conflict
    tracker is pruned from the maintenance checkpoint instead of leaking an entry per committed
    transaction and per read; and a full-text/spatial procedure feeding an expansion is no longer
    mis-dispatched off-thread into a spurious "no such index" error.
  • REST conflict handling. A conflicting single-statement write auto-commit returns a
    retriable 409 with the connection kept alive, instead of dropping the HTTP connection
    mid-stream; single-statement reads still stream. The buffered write result is bounded (16 MiB)
    so a large authenticated write cannot exhaust memory — it never commits half-way and never
    silently truncates.
  • Bulk-import retry safety. The offline bulk importer stages each batch's external-id
    bindings and row-count statistics separately and merges them only after the batch durably
    commits, so an aborted batch can be retried without falsely rejecting a duplicate id or
    resolving relationships against rolled-back physical ids.

Verification

This release was validated empirically; no functional test reported an error.

  • Conformance gates intact. The openCypher TCK remains at 3914 / 3914 (100%), and the
    Bolt, PackStream, and ACID guarantees ar...
Read more

Graphus v0.0.4

Choose a tag to compare

@FlavioCFOliveira FlavioCFOliveira released this 30 Jun 21:38

Graphus v0.0.4

Version: 0.0.4
Date: 2026-06-30
Tag: v0.0.4

Summary

Graphus v0.0.4 is a small, focused conformance release of Graphus, the Label
Property Graph (LPG) database server written in Rust for extreme load and concurrency. It
delivers a single user-visible change — a correct, populated query result summary on
both the Bolt and REST interfaces — and introduces no breaking changes: the public
Cypher, Bolt, and REST contracts are unchanged, so v0.0.4 is a drop-in upgrade from
v0.0.3.

Before this release, every query returned an empty result summary: zero update counters
and a null query type, even when the statement created nodes, set properties, or ran schema
DDL. That gap broke the Bolt/Cypher contract that Neo4j drivers and tooling rely on
(summary().counters, summary().query_type). v0.0.4 closes it.

Throughout, the four inviolable guarantees held:

  • 100% ACID — full transactional reliability under power loss, faults, and crashes.
  • 100% openCypher TCK3914 / 3914 scenarios passing.
  • 100% Bolt protocol — byte-for-byte interoperability with the Neo4j driver ecosystem.
  • 100% PackStream — exact wire-level serialization.

Highlights

  • Query result summary, populated at last. Both the Bolt and REST interfaces now emit the
    per-statement result summary that was previously always empty. The trailing summary carries
    the query type and the Neo4j-compatible statistics counters, so a write that
    persisted data now reports exactly what it changed.
  • Neo4j-compatible counters. The summary exposes the full Neo4j operation-count model with
    kebab-case keys, present only when non-empty: nodes-created / nodes-deleted,
    relationships-created / relationships-deleted, properties-set, labels-added /
    labels-removed, indexes-added / indexes-removed, constraints-added /
    constraints-removed, system-updates, contains-updates, and contains-system-updates.
  • Query-type classification. Each statement is classified as r (read), w (write),
    rw (read-write), or s (schema / admin), matching the Bolt/Cypher query-type contract.
  • Schema and admin DDL accounted for. Index, constraint, and other schema/administrative
    statements report query type s together with their schema and system update counters,
    rather than presenting as no-ops.

Fixed

  • Query result summary (side-effect counters + query type). Both the Bolt and REST
    interfaces now populate the per-statement result summary that was previously always empty —
    every query reported zero update counters and a null query type even though writes
    persisted, breaking conformance with the Bolt/Cypher contract and the Neo4j driver
    ecosystem. The trailing summary now carries the query type (r read, w write, rw
    read-write, s schema/admin) and the Neo4j-compatible stats counters (nodes-created /
    -deleted, relationships-created / -deleted, properties-set, labels-added /
    -removed, indexes-added / -removed, constraints-added / -removed, system-updates,
    contains-updates, contains-system-updates), present only when non-empty. Counters follow
    Neo4j's operation-count model and use kebab-case keys; over REST they are plain JSON numbers
    (for example "nodes-created": 1), matching the Neo4j HTTP API.

Verification

This release was validated empirically; no functional test reported an error.

  • End-to-end interface check. Verified against a locally-built server with the official
    Neo4j driver over Bolt and over the REST API: a write statement now reports its
    counters and query type, a read reports type r with no counters, and schema DDL reports
    type s with schema/system counters.
  • Regression coverage. New and extended tests assert the wire-level summary so the field
    can never silently regress to empty again — counter and query-type unit tests in the Cypher
    engine, a record-store graph counter suite, a database admin-surface DDL summary suite, and a
    REST knowledge-graph summary assertion.
  • Conformance gates intact. The openCypher TCK remains at 3914 / 3914 (100%), and the
    Bolt, PackStream, and ACID guarantees are unchanged.

Compatibility

v0.0.4 is a drop-in upgrade from v0.0.3 — no public Cypher, Bolt, or REST API contract
changed. The result summary moves from always empty to correctly populated; clients that
ignored the summary are unaffected, and clients that read it now receive accurate, Neo4j-
compatible data. The four inviolable guarantees (ACID, openCypher TCK, Bolt, PackStream)
remain at 100%.

Upgrading

# Docker Hub (multi-arch: linux/amd64 + linux/arm64)
docker pull flaviocfo/graphus:v0.0.4             # or :latest

See the repository docs/ for first-run instructions (docs/getting-started.md) and for how
each interface now reports the query result summary (docs/bolt.md, docs/rest-api.md).

Graphus v0.0.3

Choose a tag to compare

@FlavioCFOliveira FlavioCFOliveira released this 30 Jun 15:39

Graphus v0.0.3

Version: 0.0.3
Date: 2026-06-30
Tag: v0.0.3

Summary

Graphus v0.0.3 is a small, focused release of Graphus, the Label Property Graph (LPG)
database server written in Rust for extreme load and concurrency. It contains 8 changes
since v0.0.2 and introduces no breaking changes: the public
Cypher, Bolt, and REST contracts are unchanged, so v0.0.3 is a drop-in upgrade from
v0.0.2.

The theme of this release is client access and distribution — making Graphus easier to
reach, learn, and deploy: a REST login endpoint that issues Bearer tokens, first-class
usage documentation with runnable Go client examples for all three interfaces, and official
Docker Hub image publishing with conformant :latest + :vX.Y.Z tags.

Throughout, the four inviolable guarantees held:

  • 100% ACID — full transactional reliability under power loss, faults, and crashes.
  • 100% openCypher TCK3914 / 3914 scenarios passing.
  • 100% Bolt protocol — byte-for-byte interoperability with the Neo4j driver ecosystem.
  • 100% PackStream — exact wire-level serialization.

Highlights

  • POST /auth/login REST endpoint. Exchange a username + password for a short-lived
    HS256 Bearer JWT, so the authenticated REST WebAPI is usable from any HTTP client without
    distributing the server's jwt_secret. Credentials are verified with Argon2 (the same
    path as Bolt LOGON); failed attempts are rate-limited per account, and unknown-user and
    wrong-password failures return an identical 401.
  • Usage documentation + Go client examples. New guides under docs/ cover
    getting started, the REST WebAPI, Bolt over TCP/UDS, security and RBAC, and every
    configuration key. Runnable Go client programs under
    examples/clients-go/ demonstrate all three interfaces: REST,
    Bolt-over-TCP via the official Neo4j Go driver, and Bolt-over-UDS via a dependency-free
    raw client.
  • Official Docker Hub image publishing. A new GitHub Actions workflow
    (.github/workflows/dockerhub.yml) builds the multi-architecture (linux/amd64 +
    linux/arm64) image and publishes it to Docker Hub — alongside the existing GHCR
    workflow — on a published GitHub Release or a manual dispatch. Every publish carries
    conformant tags only: :latest plus a :vX.Y.Z version, never a commit-sha or
    branch tag. A community-standard repository overview ships in
    docker/dockerhub-overview.md.
  • CI action modernization. All GitHub Actions across ci.yml, docker.yml, and
    nightly-fuzz.yml were bumped to their latest major versions (Node 24 runtimes),
    keeping the pipeline on maintained, security-patched actions.

Added

  • POST /auth/login REST endpoint (username + password → short-lived HS256 Bearer JWT),
    with per-account rate limiting and uniform 401 on failure.
  • Usage documentation under docs/ (getting started, REST WebAPI, Bolt over TCP/UDS,
    security and RBAC, configuration).
  • Go client examples under examples/clients-go/ for REST, Bolt-over-TCP (official Neo4j
    Go driver), and Bolt-over-UDS (dependency-free raw client).
  • Docker Hub image publishing workflow with conformant :latest + :vX.Y.Z tagging, plus
    a Docker Hub repository overview document.

Changed

  • All CI GitHub Actions updated to their latest major versions across ci.yml,
    docker.yml, and nightly-fuzz.yml.

Compatibility

v0.0.3 is a drop-in upgrade from v0.0.2 — no public Cypher, Bolt, or REST API contract
changed. The four inviolable guarantees (ACID, openCypher TCK, Bolt, PackStream) remain at
100%.

Upgrading

# Docker Hub (multi-arch: linux/amd64 + linux/arm64)
docker pull flaviocfo/graphus:v0.0.3             # or :latest

See docs/getting-started.md for first-run instructions and
the docker/dockerhub-overview.md overview for the full
container usage reference.

Graphus v0.0.2

Choose a tag to compare

@FlavioCFOliveira FlavioCFOliveira released this 30 Jun 06:29

Graphus v0.0.2

Version: 0.0.2
Date: 2026-06-29
Tag: v0.0.2

Summary

Graphus v0.0.2 is a hardening, performance, and production-readiness release of
Graphus, the Label Property Graph (LPG) database server written in Rust for extreme
load and concurrency. It contains 156 changes since v0.0.1
(55 features, 47 fixes, 35 performance improvements, plus tests, refactors, and docs) and
introduces no breaking changes: the public Cypher, Bolt, and REST contracts are
unchanged, so v0.0.2 is a drop-in upgrade from v0.0.1.

The headline of this release is reliability and resource discipline under hostility.
Multiple empirical audit-and-remediation rounds — crash × fault × interleaving durability
audits and an extreme-concurrency, production-confidence security audit — closed every
reachable critical and high-severity issue and added a full denial-of-service (DoS)
resistance
suite. In parallel, performance campaigns brought native columnar storage,
intra-query and off-thread parallelism, and broad CPU, RAM, and storage optimizations.

Throughout this work, the four inviolable guarantees held:

  • 100% ACID — full transactional reliability under power loss, faults, and crashes.
  • 100% openCypher TCK3914 / 3914 scenarios passing, zero failed, errored, or
    unsupported.
  • 100% Bolt protocol — byte-for-byte interoperability with the Neo4j driver ecosystem.
  • 100% PackStream — exact wire-level serialization.

Highlights

  • Denial-of-service resistance. A per-statement execution timeout, a per-transaction
    maximum-age cap, per-source-IP connection caps, pre-authentication deadlines, per-value
    memory budgets, a bounded query planner, and PackStream decode guards — closing CPU-bomb,
    memory-bomb, connection-flood, slow-loris, and stack-overflow vectors.
  • Durability hardening. The doublewrite buffer is wired into the production checkpoint
    and recovery paths, closing several committed-data-loss windows under crash combined with
    disk faults, validated deterministically by the DST simulator.
  • Native columnar storage. A dependency-light columnar codec, a .gcol bulk format, a
    vectorized columnar property store, zone-map data skipping, a Roaring-bitmap index, and an
    immutable cold tier for aged time-series.
  • Parallel query execution. Morsel-driven intra-query parallelism and off-thread
    concurrent reads, so a single heavy query and concurrent readers can use multiple cores.
  • Deterministic Simulation Testing (DST) maturation. First-class VOPR safety and
    liveness modes, a unified seeded fault scheduler, crash + ARIES interleaving, swarm
    testing, and a CI gate plus a nightly fuzz job.
  • Instrumented examples suite. Realistic end-to-end scenarios that each collect CPU,
    RAM, and storage evidence against committed baselines.
  • UDS on Apple Silicon. Unix-domain-socket peer-credential authentication now works on
    macOS/BSD (via getpeereid), not only Linux, so the IPC interface is usable on every
    Tier-1 target.

What changed

Reliability and durability

This release is anchored by repeated, empirical audit-and-remediation rounds focused on the
hardest regime for a database: crash, combined with disk and clock faults, combined with
concurrent interleavings
.

  • Doublewrite durability. The doublewrite buffer is now wired into the production
    checkpoint and open paths, with disjoint checkpoint-batch and eviction regions, a
    per-eviction serialized stage → home-write → sync, a persisted checkpoint-floor LSN
    gate, a multi-slot eviction ring, WAL-before-data enforcement, and an orphan-page check on
    open. Together these closed several committed-data-loss windows under crash × disk-fault.
  • Recovery correctness. Committed nodes and self-loops are recovered after interleaved
    live-rollback plus crash-undo; ARIES double-crash defects (including transaction-id reuse
    across recovery) are fixed; a double-panic in recovery rollback is caught.
  • Transaction isolation. A concurrent NODE KEY duplicate commit is closed, dangling
    SSI read-write edges are scrubbed, abort state is released unconditionally so a panicking
    undo cannot leak a transaction, and a cross-type equality seek (1 = 1.0) no longer
    admits duplicates or misses the index.
  • Index correctness. MVCC-correct full-text and spatial indexed reads fix a
    cross-snapshot stale-read; geographic (WGS-84) spatial seeks decline to scan rather than
    return silent false negatives.
  • Server robustness. A panicking statement is isolated and can no longer brick the
    engine; the server tracks a per-engine degraded state with a clean startup and shutdown
    lifecycle.

Security and denial-of-service resistance

A production-confidence audit, conducted under extreme hostility and concurrency, added a
comprehensive suite of resource bounds. Each control ships with a safe default:

Control Default Vector closed
Per-statement execution timeout 2 minutes CPU-bomb queries
Per-transaction maximum-age cap 1 hour GC-watermark pinning
Per-source-IP connection cap + pre-auth deadline enabled Connection floods
Pre-authentication read deadline enabled Slow-loris clients
Per-value materialized-size budget enabled Single-value memory bombs
Bounded join-order planner greedy above 8 operands Plan-time blow-up
Bounded expression depth + larger engine stack enabled ~1 KB query process abort
PackStream struct-decode depth guard MAX_DECODE_DEPTH 64 Pre-auth stack overflow
PackStream decode-bomb preallocation ceiling 512 KiB Decode-bomb allocation
Incremental REST result egress enabled Remote out-of-memory

The per-value budget extends to list and string builtins and to list + concatenation,
map literals, and properties(). The audit also fixed a graph-scoped RBAC defect,
discovered through the security-multitenant example, where the REST interface false-denied
every per-tenant grant.

Performance

Performance campaigns spanned the query engine, the storage layer, and concurrency:

  • Query execution. Hash-bucket aggregation grouping removes an O(rows × groups) cliff;
    schema-shared positional rows and reduced hot-loop allocation cut per-row overhead; result
    cells are moved, not cloned, into Bolt and REST values; the planner reverses
    expand-direction by cost and filters relationship types by integer id.
  • Concurrency and scaling. A sharded RwLock device buffer pool serves concurrent
    cache-miss reads; a reverse SSI write-index makes record_read O(writers-of-key); the
    TimestampOracle releases timestamps in O(log N); the compute-thread budget is bounded;
    the RUN path reuses the plan cache.
  • Storage, I/O, and WAL. A page-batched scan primitive amortizes latching; write-back is
    coalesced with a copy-free pwritev fast path; WAL patches use inline buffers with
    borrowed redo; B+-tree validation is amortized per (page, LSN); a live-engine checkpoint
    trigger with memory-freeing log-sink reclaim bounds RAM and WAL growth.
  • Cryptography. Per-target AES/GHASH compilation seals WAL frames in place, and a
    buffered ChaCha20 CSPRNG nonce source eliminates a per-nonce getrandom syscall.
  • Graph Data Science. Betweenness and closeness centrality are parallelized across cores
    over a shared flat-CSR adjacency built once per sweep.

Native columnar storage

Graphus gains a native, dependency-light columnar subsystem for analytical workloads:

  • graphus-columnar, a native columnar codec foundation.
  • A .gcol bulk dump and import format.
  • A complementary columnar property store with vectorized aggregation.
  • A zone-map data-skipping sidecar for non-indexed scans.
  • A Roaring-bitmap secondary index for low-cardinality columns.
  • An immutable columnar cold tier for aged time-series data.
  • Internal-id-aligned numeric GDS node columns with zero-copy export.
  • A native columnar REST result channel for analytical queries.

Concurrency and parallelism

The read path was refactored to a &self, Send + Sync model over a shared buffer pool and
a metadata snapshot, unlocking two forms of parallelism:

  • Off-thread concurrent reads via a reader pool, so concurrent readers scale across
    cores.
  • Morsel-driven intra-query parallelism for grouped aggregation, for
    scan → filter → project with a stable ORDER BY/top-k, and for ExpandAll, so a single
    heavy query can use multiple cores.

The loom-validated concurrent buffer pool is now the production pool, and the SSI tracker is
concurrency-ready via per-reader deferred read buffers.

Deterministic Simulation Testing (DST)

The DST simulator matured into a first-class correctness engine:

  • First-class VOPR safety and liveness modes.
  • A deterministic cooperative interleaver over overlapping explicit transactions.
  • A unified, seeded fault scheduler covering disk faults (bit-rot, misdirected I/O, latent
    sectors, ENOSPC, write reordering, sector-granular torn writes), clock faults, and
    transport faults.
  • Crash plus ARIES restart woven into the running interleave, with an acked-versus-in-flight
    crash-split oracle and a cell-by-cell reference-model oracle.
  • Swarm testing, failing-seed minimization with replayable artifacts, and a continuous,
    time-budgeted, multi-core fuzzer.
  • A pull-request CI gate (safety, liveness, and determinism sweeps) and a nightly swarmed
    fuzz job with artifact upload.

Examples suite

A new examples/* suite provides realistic, end-to-end demonstrations, each instrumented to
collect CPU, RAM, and storage evidence against a committed baseline:

  • social-network-uds and social-network-large (1M-user target).
  • fraud-oltp (extreme-concurrency SSI detection).
  • gds-analytics (full gds.* algorithm workload with exact ground truth).
  • bulk-etl (...
Read more