Skip to content

Releases: kraison/vivace-graph

VivaceGraph 2.1.1

Choose a tag to compare

@kraison kraison released this 06 Jul 09:24

A bug-fix release.

Fixed

  • ECL: cross-graph edge-exists-p / adjacency read the wrong heap. A ve/vev
    index lookup for a graph other than the current *graph* deserialized its
    index-list against (heap *graph*) — the deserialize-index-list default — instead
    of the owning graph's heap. So a cold (edge-exists-p … :graph B) (or a generated
    make-<type>'s type-id resolution) while *graph* named a different graph walked
    the wrong heap and returned NIL, and the per-index cache was then poisoned with the
    mis-bound list. Fixed by binding *graph* to the owning graph at the
    lookup-vev-index-list / lookup-ve-in-index-list / lookup-ve-out-index-list
    read boundaries. It manifested on ECL (SBCL's cache/timing masked it in the
    regression test), but the underlying flaw was implementation-independent. Full test
    suite green on both SBCL and ECL.

VivaceGraph 2.1.0

Choose a tag to compare

@kraison kraison released this 05 Jul 16:34

A large, backward-compatible feature release: a pluggable ordered-index backend
(an mmap B+ tree alongside the skip list), :unique slot constraints, offline-first
peer replication, an in-memory backend, idempotent views, a modernized Prolog engine
with a safe web query surface, and cross-cutting correctness fixes. Existing on-disk
v2 graphs open without migration.

Added

  • Pluggable ordered-index backend — an mmap B+ tree (opt-in). All heap-backed
    ordered indexes (map/reduce views, :unique constraints, the spatial index) are
    now built on a shared ordered-map protocol with two interchangeable backends: the
    skip list (default) and a new page-oriented (4 KB slotted-page) B+ tree
    (bplus-tree.lisp). The backend is a per-graph choice — :index-backend :bplus-tree on make-graph/open-graph, or the global graph-db:*index-backend*
    — and each index persists the backend it was written with, so a graph reopens
    every index on its own engine. On disk the B+ tree beats the skip list on every
    operation once warm (page-packed keys → far fewer cache-line and page misses,
    sequential in-leaf range scans, less space), with in-place cell edits and
    merge-on-delete rebalancing. Existing graphs migrate in place via
    regenerate-all-views / regenerate-unique-indexes / rebuild-spatial-index
    (or snapshot + replay). (Manual Chapter 3.)
  • :unique slot constraints (issue #6). def-vertex / def-edge slots may
    carry :unique t | equal | equalp | <canonicalizer> (the value is the uniqueness
    key — identity, case/edge folding, or an arbitrary canonical form). Enforced at
    the commit boundary: a violation aborts the whole transaction with
    unique-constraint-violation; NULL-exempt (SQL-style); shared across subclasses
    of the declaring type; commits racing for the same value are serialized so exactly
    one wins. Backed by a persistent, per-graph unique index (skip-list or B+ tree;
    in-RAM map on a memory-graph) reopened with the graph — not rebuilt by scanning on
    open. (Manual Chapter 8. Distributed cross-device arbitration is tracked in #51.)
  • Peer replication — offline-first, hub-and-spoke sync (Chapter 16). A
    bidirectional peer mode for mobile/edge fleets, alongside the existing
    master/slave replication: each device is synced only the authorized subset of the
    graph it may see (:export-predicate), authors locally while disconnected, and
    reconciles on reconnect. Closed-subgraph export + manifest reconciliation, node
    re-homing, per-node origin identity, Lamport clocks, and a pluggable
    conflict-resolution policy (:origin partitioning by default). Runs on both the
    on-disk and in-memory backends (verified SBCL hub ↔ ECL device).
  • In-memory backend — make-memory-graph (issue #50, Chapter 15). An in-RAM
    storage backend that holds the whole graph as live Lisp objects, eliminating
    per-read deserialization and pcons-chain walking — lowest-latency reads when the
    graph fits in memory (aimed at mobile/ECL). Durable via the same journal plus a
    checkpoint image; eager or fault-on-access (lazy) open. The graph model,
    with-transaction, OCC validation, views, spatial, :unique, peer replication,
    and the Prolog engine all work against it unchanged.
  • Idempotent def-view (issue #49). def-view is now declarative and
    idempotent: redefining a view with an unchanged definition is O(1) at open (no
    rescan), and a changed definition is diffed and rebuilt automatically via a
    two-phase registry. open-graph / open-memory-graph install views on open and
    accept :regenerate-views t; regenerate-all-views forces a full rebuild.
  • Streaming results: select :callback + NDJSON web responses (issue #44).
    select accepts :callback FN, which hands each result row to FN as it is
    produced -- consing nothing onto a result list -- and returns the row count.
    An embedded consumer can stream an unbounded result set with constant memory.
    The web layer uses it: a query with format=ndjson (a parameter for
    def-query, a body field for the pattern query) streams each row as its own
    JSON object on its own line (application/x-ndjson) instead of buffering a
    JSON array.
  • ISO exceptions: catch/3 + throw/1 (issue #45). throw(Ball) raises a
    ball and catch(Goal, Catcher, Recovery) recovers from one that unifies with
    Catcher, propagating others to an outer catch. Only Goal is protected --
    a throw in the continuation after catch/3 succeeds is not caught (the
    continuation-swallowing trap, handled with a per-frame marker). Built-in
    errors now carry an ISO-style ball so they are catchable: an unknown predicate
    is an existence_error, an uninstantiated meta-call an instantiation_error,
    a non-callable goal a type_error. The error vocabulary is keywords
    ((:error (:existence-error :procedure foo/2) Ctx)) so a ball unifies
    regardless of the query's package. Resource (budget/timeout) and permission
    (effect-policy) errors are deliberately not catchable, so a bounded,
    untrusted query cannot catch(Goal, _, true) to swallow its own enforcement.
  • Prolog control-flow core (issue #45, Phase 0). not/\+, if (the
    two- and three-argument Cond -> Then [; Else] soft cut), once, and forall
    are now first-class compiler constructs: they expand through compile-body, so
    they thread bindings and compose with conjunction and cut instead of routing
    through the runtime call/1 functors. Each opaque construct (not, once,
    the condition of if) is a proper cut barrier, while a cut in a Then/Else
    branch or in the tail after the construct still cuts the enclosing clause.
    A non-static (meta-call variable) sub-goal, e.g. (not ?G), transparently
    falls back to the runtime functor, so existing behavior is preserved.
  • Compiled call/N and a runtime meta-call solver (issue #45, Phase 0.2).
    (call Goal Extra...) now appends the extra arguments to Goal (call/N).
    When Goal is a static template the call compiles inline through
    compile-body, so it composes with cut and the control constructs (e.g.
    (call (or ...)), (call (g-knows ?a) ?b)). When Goal is a variable, the
    new %solve runtime solver proves it -- handling conjunction, disjunction,
    call/N and atomic/compound goals. call/1 and the control runtime functors
    (not/if/once/forall) route through %solve.
  • All-solutions aggregation (issue #45, Phase 0.3). New findall/3 (collects
    every template instance in order, always succeeds, [] on no solutions).
    bagof/3 and setof/3 now group by the goal's free (witness) variables --
    the variables in the goal but not in the template and not existentially
    quantified -- yielding one solution per witness binding (and still failing when
    the goal has no solutions). The ^ operator ((^ Var Goal), nestable, accepts
    a single var or a list) marks variables as existential so they are not treated
    as witnesses. setof sorts each group by the standard order of terms and
    removes duplicates.
  • Query resource bounds (issue #45, Phase 0.4). Queries can now be bounded by
    a maximum inference count (:max-inferences select option / *inference-budget*
    / *default-inference-budget*) and a wall-clock timeout (:timeout seconds /
    *default-query-timeout*). Exceeding either aborts the query with a catchable
    prolog-resource-error, so a runaway, non-terminating, or cyclic-recursive
    query fails cleanly instead of hanging or overflowing the Lisp control stack.
    Both default to nil (unlimited): trusted queries are unchanged, untrusted ones
    (e.g. the planned #44 web surface) opt in. Solution count remains bounded by
    the existing :limit.
  • Effect partitioning / query effect policy (issue #45, Phase 1). The
    side-effecting Prolog functors are now tagged by effect -- :write (graph
    mutation: retract), :eval (arbitrary Lisp: lisp/lispp/is/trigger),
    :io (read/write/nl) -- and check the per-query policy before acting.
    The :effects select option (or *allowed-effects* / *default-allowed-effects*)
    is t for all (the default) or a list of permitted tags; a disallowed effect
    aborts with a catchable prolog-permission-error. Reads and pure logic are
    always allowed, so :effects nil is a safe read-only query mode (the basis for
    exposing queries to untrusted callers). The check is transitive -- an effect
    reached through a user rule or meta-call is gated the same way.
  • Snapshot query mode (issue #45, Phase 1). select accepts :snapshot t,
    which runs the query under a single consistent MVCC read snapshot: every read
    resolves at one epoch, so the result is stable against concurrent writers (a
    vertex committed after the query started is invisible to it). Implemented as
    a lightweight read transaction (with-read-snapshot / call-with-read-snapshot)
    that registers active for the query's extent -- holding the reaper's retention
    floor -- and is discarded without commit or validation. It inherits an
    enclosing transaction if one is already active. Together with the resource
    bounds, the effect policy, and :limit/:skip, this gives a query surface
    safe to expose to untrusted callers.

Changed

  • Unknown Prolog predicates are now noisy. A goal naming an undefined
    predicate signals a prolog-error -- on both the compiled query path and the
    dynamic meta-call path -- instead of silently yielding no answers (the
    compiled path) or aborting with an opaque message (the old call/1). This
    surfaces mistyped predicate names instead of letting them masquerade as empty
    results. (A future catch/3 + ISO existence_error will make this
    recoverable; see #45.)

  • select-count / select :count. select-count (already exported but
    never implemented) now returns the integer number of solutions to a query
    without projecting...

Read more