Skip to content

Releases: danReynolds/resqlite

v0.7.0

Choose a tag to compare

@danReynolds danReynolds released this 30 Jun 13:17
80c9362

Performance and correctness release, with one small breaking change to selectBytes.

Breaking

  • Database.selectBytes now returns a BytesResult ({Uint8List bytes, int rowCount}) instead of a bare Uint8List. Read .bytes at call sites.

Fixed

  • Windows: local databases now open. resqlite.dll exported no FFI symbols (MSVC default), so every call failed with runtime error 127. The build hook now emits /export: directives for the full FFI surface; the export set is scanned from the @Native bindings and shared with the Linux version script, which also closes a parallel gap where the resqlite_reader_* helpers (used on every read) were hidden on Linux (#216).

New

  • selectBytes reports rowCount — counted in C during the same serialization pass (free for has_more-style paging).

Performance — selectBytes JSON serialization

Per-experiment order-flipped A/B deltas; output byte-identical.

  • Integer-valued REAL fast path — −72% to −81% integral-REAL lanes (#200)
  • Integer column formatting (two-digit itoa + direct-to-buffer + row-level reserve) — −8% to −26% (#198, #206, #208)
  • Column-name token pre-encoding — −4% to −11% wide cols (#196, #201)
  • Per-cell sqlite3_column_value reuse — −1% to −6% (#213, #215)
  • Large single-row text bind encoder — −15% to −39% at 16 KB–1 MB (#190, #193)

Companions

  • resqlite_vector and resqlite_js0.1.3 (compatibility release for resqlite: ^0.7.0; no functional changes).

Also explored (rejected — no runtime code shipped)

exp 189 (savepoint name compression), 193 (Row.values list-view), 196 (inter-row framing), 197 (true group commit moonshot), 200 (stable-type selectBytes moonshot), 201 (base64 quote reservation), 202 (TEXT string reservation), 204 (UTF-8 over-reserve bind sizing).

v0.6.0

Choose a tag to compare

@danReynolds danReynolds released this 18 Jun 15:12
92bb2b2

Performance and observability release. No breaking changes, and no changes to the
public export surface — safe to upgrade from 0.5.x.

Wins at a glance: up to ~30% faster concurrent writes on top of 0.5.0's
pipelining, a new diagnostics counter for native read-buffer memory, and automatic
reclaim of that memory after large-selectBytes bursts — bounding the RSS trade-off
0.5.0 introduced with native-view transfer.

  • New API: Diagnostics.readerJsonBufHighWaterBytes — the summed capacity of
    every reader isolate's native json_buf, making the read-buffer high-water mark
    directly observable. Surfaces workloads that pin large native buffers after big
    selectBytes reads (#183,
    exp 183).
  • Behavior change: a buffered group of execute() calls that races
    Database.close() is now atomic — every call in the group either flushes or is
    rejected, never the old lock-order-dependent partial outcome. It still never hangs,
    and per-call success/failure semantics are unchanged (#184,
    exp 180).
  • Memory: native read buffers that grow to serve a large selectBytes read now
    reclaim back to their initial capacity on the next small read, once a reader's
    json_buf exceeds 1 MB. A one-off concurrent burst of large reads (e.g. 8 × 8 MB)
    previously pinned tens of MB for the connection's lifetime; that high-water now
    settles back to ~64 KB. Warm large-read workloads keep their capacity (a 256 KB
    last-read guard prevents shrink-then-regrow churn) (#183,
    exp 183).
  • Performance. See the interactive benchmark dashboard
    for current cross-library numbers.
    • Cross-call write batching (group commit) — standalone execute() calls that
      pile up while a write is in flight now coalesce into a single cross-isolate
      request (each statement still its own autocommit), collapsing a concurrent
      burst's per-write round-trips toward two. −26% to −32% on the concurrent
      single-insert lane, on top of 0.5.0's pipelining; isolated and sequential writes
      are unaffected (#184,
      exp 180).

Companion compatibility releases: resqlite_vector 0.1.2, resqlite_js 0.1.2 (both require resqlite: ^0.6.0).

Perf validation (v0.5.0 → 0.6.0) and the full rejected-experiment list are in #187.

v0.5.0

Choose a tag to compare

@danReynolds danReynolds released this 16 Jun 18:45

0.5.0

Performance and reliability release. No breaking changes, and no changes to the
public export surface — safe to upgrade from 0.4.x.

Wins at a glance: up to 45% faster concurrent writes, ~1.8× faster large
selectBytes reads
(>256 KB), faster Row lookups and batch writes, a new
diagnostics counter for silent re-query fallbacks, and a reader-isolate crash fix
for diagnostics() under load.

  • New API: Diagnostics.unknownDependencyFallbackCount — a cumulative counter
    of writes that conservatively re-queried every registered stream because native
    dependency tracking overflowed its caps (or hit OOM). Surfaces workloads silently
    paying full re-query fan-out on every write, which was previously unobservable
    (#151).
  • Bug fix: Database.diagnostics() could intermittently crash a reader isolate
    (SEGV) when polled while readers were mid-query — it toggled SQLite memory
    accounting on live NOMUTEX reader connections. Read workers now bracket each
    request with a real busy guard, and diagnostics reports busy readers as a partial
    snapshot (#156).
  • Performance. Each accepted experiment below; see the interactive benchmark dashboard
    for current cross-library numbers.
    • Writer-request pipelining over a persistent reply port — −36% to −45% on
      concurrent standalone writes (#153,
      exp 159).
    • selectBytes native-view transfer — sends a view over the reader's native
      json_buf instead of taking the sacrifice/respawn path; −44% (~1.8×) on large
      (>256 KB) byte reads, at a bounded ~+15 MB RSS (#169,
      exp 174).
    • Row.containsKey identity fast path — pointer-identity membership scan for
      interned keys on schemas ≤ 32 columns; −23% on the containsKey lane
      (#173,
      exp 176).
    • RowSchema lookup fast path — schema-name identity scan with HashMap
      fallback; ~2× faster main-isolate map consumption at 10K rows
      (#150,
      exp 158).
    • Six-parameter batch packing — extends guarded ASCII batch packing to the
      six-parameter shape; executeBatch p50 88 → 75 µs (#141,
      exp 149).
    • Nullable batch packing — nullable-aware packed batch encoder for first-row
      NULL text columns; nullable ASCII 10k×20 25.7 → 21.7 ms (#145,
      exp 150).

Companion packages released alongside this version: resqlite_vector 0.1.1 and resqlite_js 0.1.1 (now require resqlite: ^0.5.0).

v0.3.0

Choose a tag to compare

@danReynolds danReynolds released this 04 May 03:17
  • Behavior change: PRAGMA foreign_keys = ON is now applied by default on every connection (#77). Code that relied on FK constraints being silently ignored will now see them enforced.
  • Column-level reactive invalidation. Streams now record the columns they read and writes record the columns they touch; a write to a column no stream watches no longer dispatches re-queries. Adds public TableDependencies / TableDependency / TableColumnDependency types (#48). +82% on disjoint-column writer-throughput benchmarks (A11c).
  • Further write-path and stream-dispatch wins: cached BEGIN/COMMIT/ROLLBACK statements, inline-packed parameter buffer, direct batch parameter matrix encoding, FIFO reader-pool dispatch with bounded synchronous stream admission. See the interactive benchmark dashboard for current cross-library numbers.
  • Documented that streams over virtual tables (FTS5, R-Tree, JSON1 json_each, etc.) don't get reactive invalidation; use select instead.