Skip to content

LionSQL v0.90.0

Choose a tag to compare

@MaroShim MaroShim released this 03 Nov 06:31
· 618 commits to main since this release

LionSQL v0.90.0 Release Notes (notes/RELEASE_NOTES_v0.90.0.md)

  • Initial Draft: 2026-11-01
  • Last Updated: 2026-11-03
  • Target Release: v0.90.0 (Official GA)
  • Theme: Durability & Reliability Hardening, Core Concurrency Defect Remedies, and Ecosystem Integrations Across 8 Services

1. Release Highlights

LionSQL v0.90.0 represents a landmark reliability milestone toward the 1.0 production release. This release is dedicated to proving crash resilience under kill -9, zero data loss, concurrency isolation without torn reads, and mathematical SQL three-valued logic compliance under rigorous automated test harnesses.

  • 100% Zero-Loss Crash Recovery Verification (cmd/crashtest):
    • Validated cumulative crash resilience across WiredTiger and In-Memory OCC under brutal interruption scenarios (SIGKILL, SIGKILL+DOUBLE_KILL, SIGKILL+TORN_WAL_TAIL, SIGTERM).
    • 100% of acknowledged transactions survived across both engines with zero data loss according to the SHA-256 crash ledger (crash_ledger) and the 4-point consistency oracle (D1 Commit Durability, D2 Atomicity SUM(balance)=const, D3 Checksum Integrity, and D4 Recoverability).
  • Long-Duration Multi-Worker Soak Testing (cmd/soaktest):
    • Sustained continuous 8-worker transactional workload with zero resource leaks (RSS memory, goroutines, and open file descriptors maintained a strict 1.00 ratio after warm-up).
    • Maintained the global bank ledger invariant (SUM(balance) = 1,000,000) throughout execution with zero p99 latency degradation.
  • SQLite SQLLogicTest Conformance Suite (90 Standard Files 100% PASS) (cmd/sqllogictest & pkg/slt):
    • Embedded pure-Go blackbox oracle test runner supporting rowsort, valuesort, and MD5 hash verification.
    • Passed 90 standard test suites (tens of thousands of queries) with 100% oracle match (0 failures) across subqueries, CASE expressions, DISTINCT, window functions, CTEs, 3VL predicates, self/cross joins, JSON operators, upserts, aggregates, and whitespace boundaries.
    • Isolated 14 SQLite-specific fuzz suites into test/sqlite_fuzz/ to protect release gate hermeticity.
  • Ultra-High OLTP Throughput (cmd/tpcc_runtime):
    • Achieved 732,956 tpmC (27,118.95 TPS) on In-Memory OCC (10 Warehouses / 10 Workers).
    • New-Order P50 0.30ms / P99 2.79ms; Payment P50 0.08ms / P99 1.29ms; Serialization Abort rate 0.00% (11 aborts / 407,059 attempts).
  • Remedied Latent Kernel Concurrency Defects:
    • Eliminated WiredTiger thread-affinity cursor close SIGSEGV during concurrent DDL drops and restored transaction_sync=(enabled=true,method=none) for crash durability.
    • Eliminated In-Memory OCC Scan() torn reads by capturing consistent table snapshots under canonical ascending shard read locks.
    • Fixed parseInCondition compound expression hijacking with permanent regression test coverage.
  • Completed Production Integrations Across 8 Major Services:
    • Grafana, Casdoor, Apache Answer, Vaultwarden, Metabase BI, Umami, Woodpecker CI, and Vikunja.

2. Detailed Technical Improvements

2.1. Durability & Reliability Test Harnesses

  • cmd/crashtest (kill -9 Crash Recovery Harness):
    • Real-time SHA-256 crash ledger (crash_ledger) tracking acknowledged commits versus in-flight state.
    • Enforced 4-point consistency oracle: D1 Commit Durability, D2 Atomicity (SUM(balance) invariant), D3 Checksum Integrity, and D4 Recoverability.
    • 100% zero-loss pass rate across both WiredTiger and In-Memory OCC.
  • cmd/soaktest (Long-Duration Resource & Integrity Harvester):
    • 8-worker concurrent transfers, periodic connection churn, continuous invariant verification, and runtime metrics scraping via Prometheus (/metrics).
    • Automated trend verdict detecting linear RSS growth, goroutine leakage, file descriptor exhaustion, or p99 latency degradation.
    • Verified on WiredTiger and In-Memory OCC with zero memory leaks and fixed file descriptor counts.
  • cmd/sqllogictest & pkg/slt (SQLite Blackbox Oracle Runner):
    • Validated core relational algebra, three-valued logic (NULL = NULL, val = NULL, NOT BETWEEN, NOT IN), window functions, CTEs, self/cross joins, operator associativity, and whitespace invariance across 90 standard test files.

2.2. Critical Kernel Bug Remedies

  • WiredTiger Thread-Affinity Cursor Eviction SIGSEGV Remedy:
    • Diagnosed a thread-boundary violation where dropping a table across sessions triggered cross-thread cursor closing in WiredTiger's C engine.
    • Resolved via thread-local pending closure queues (s.toClose) and go_wt_session_drop(..., "force=true").
  • In-Memory OCC Scan() Atomic Snapshot Remedy:
    • Resolved torn read anomalies during table scans (SELECT SUM(balance)) under high concurrent write load.
    • Implemented snapshotRowsAndKeys() with ascending canonical shard read locks (0 to 255), guaranteeing lock-free, torn-read-immune scan cursors.
  • WiredTiger transaction_sync Restoration:
    • Restored transaction_sync=(enabled=true,method=none) in pkg/storage/wiredtiger/wt_engine.go to prevent user-space log buffer loss upon unannounced SIGKILL.
  • parseInCondition Compound Condition Hijacking Remedy:
    • Fixed false-positive IN-condition detection on compound expressions (col1 IN (1, 2) AND (col2 = 3)) with paren matching boundary checks and isolated regression tests.
  • WiredTiger WT_ROLLBACK Serialization Mapping:
    • Wrapped write conflicts (-31800) into storage.ErrSerializationFailure mapped to PGWire SQLSTATE 40001 for transparent client retries.

2.3. SQL Parser, AST & Architectural Compliance

  • Zero-Regex SQL Parsing in Production Paths ([RULE:ModularAbstraction], Invariants 7 & 8):
    • Completely removed fragile regular expressions from nested SQL condition parsing, standardizing on deterministic tokenizers in pkg/sqlutil.
  • Immutable Rule Slugs Adoption:
    • Transitioned from volatile numeric rule indexes to immutable semantic slugs ([RULE:RadicalKISS], [RULE:PipelineVerification]).
  • Function Distinct Parsing Support:
    • Added FunctionCallExpr.Distinct AST representation and parser support for COUNT(DISTINCT col) and SUM(DISTINCT col).
  • SQL Three-Valued Logic (3VL) Compliance:
    • Corrected NULL = NULL, val = NULL, NOT BETWEEN, and NOT IN (including NULL list elements) evaluations to strictly return UNKNOWN (falsy) per SQL standards.
  • AST Deparser Operator Precedence Guard:
    • Guaranteed preservation of parenthesized grouping in (A OR B) AND C during AST deparsing.
  • AST and Catalog-Driven Routing (ROUTING-AST-CATALOG-1):
    • Replaced 45 legacy string heuristics with AST and catalog symbol resolution.

2.4. Third-Party Ecosystem Integrations

  • Grafana (GRAFANA-INTEGRATION-1): Dual-protocol PostgreSQL & TDS (MSSQL) dashboard integration.
  • Casdoor (CASDOOR-INTEGRATION-1): Composite primary key support (\x1f unit separator) and TDS varbinary round-trip for global IAM/SSO.
  • Apache Answer (INTEG-ANSWER-1): Developer Q&A platform PGWire E2E integration.
  • Vaultwarden (INTEG-VAULTWARDEN-1): Rust Diesel ORM-backed password manager integration.
  • Metabase (INTEG-METABASE-1): Enterprise BI analytics platform dual-protocol integration.
  • Umami (INTEG-UMAMI-1): Privacy-preserving web analytics platform Prisma ORM integration.
  • Woodpecker CI & Vikunja (INTEG-WOODPECKER-1, INTEG-VIKUNJA-1): Lightweight CI/CD and task management integration.
  • IOTA-SDK: Go ERP framework integration and PostgreSQL DDL/protocol compatibility.

2.5. Performance & Benchmarking

  • TPC-C Scaled Saturation: Reached 732,956 tpmC / 27,118.95 TPS (P50 0.30ms, P99 2.79ms) on In-Memory OCC.
  • PGWire Prepared Statement Direct VDBE Binding (PERF-PGWIRE-PREP-TX): Bypassed redundant re-parsing within transactions.
  • PGWire 32KB Output Buffer Pooling & SELECT 1 Fast-Path Ping.
  • Permanent Lightweight Performance Gate (PERF-REGRESSION-RECOVERY-1): Automated 1,000-transaction ACID transfer regression gate running in ~0.12s.

3. Official Verification Scorecard

Area Suite Workload Scale Key Metrics / Verdict Status
SQLLogicTest Standard cmd/sqllogictest 90 standard test files 0 mismatches / 100% coverage across core SQL 100% PASS
Crash Durability cmd/crashtest (WiredTiger) 20 rounds (SIGKILL/DOUBLE/TORN) 63,310 tx recovered / 0 data loss / 4-point oracle pass 100% PASS
Crash Durability cmd/crashtest (In-Memory OCC) 20 rounds (SIGKILL/DOUBLE/TORN) 62,844 tx recovered / 0 data loss / 4-point oracle pass 100% PASS
Soak Stability cmd/soaktest (In-Memory OCC) 8 workers continuous (200 ops/s) Invariant intact / RSS·FD·Goroutines ratio 1.00 (0 leaks) / 0 latency degradation 100% PASS
TPC-C OLTP Throughput cmd/tpcc_runtime (OCC) 10W / 10 Workers / 15s 732,956 tpmC / 27,118.95 TPS (P50 0.30ms, P99 2.79ms) 100% PASS
13-Domain Smoke Test scripts/verify_0.90.sh (Stage 4) In-Memory & WiredTiger dual engines 13/13 domains passed 100% PASS
Race-Free Concurrency go test -race -short ./... 22 packages 0 data races / 100% pass 100% PASS

Detailed Changelog Delta

Added

  • complete IOTA-SDK Go ERP integration and PostgreSQL DDL/protocol compatibility (56e7471)
  • complete Grafana dual-protocol integration across PGWire and TDS (GRAFANA-INTEGRATION-1) (7e4731b)
  • support composite primary keys and TDS varbinary round-trip for Casdoor integration (CASDOOR-INTEGRATION-1) (a318fc2)
  • add phase 1 soak test tool with resource trend and integrity verdict (DURA-SOAK-1) (78def58)
  • add /metrics heap and goroutine scraping and verify 1h WiredTiger soak (DURA-SOAK-F2) (95f563b)
  • implement kill-9 crash recovery tool and fix durability edge cases (DURA-KILL-1) (1a3b0e5)
  • complete Apache Answer Q&A platform integration over PGWire (a62e71e)
  • integrate Vaultwarden with Diesel ORM over PGWire (INTEG-VAULTWARDEN-1) (ff859eb)
  • integrate Metabase BI over PGWire and TDS (INTEG-METABASE-1) (5dd6b8c)
  • integrate Umami web analytics over PGWire (INTEG-UMAMI-1) (b6eb6f4)
  • integrate Woodpecker CI and Vikunja over PGWire (INTEG-WOODPECKER-1, INTEG-VIKUNJA-1) (93ea4d1)
  • formalize table-driven compatibility rules (COMPAT-TABLE-DRIVEN-1) (aac22e1)
  • support arbitrary whitespace between words in multi-word keyword scanner (6c1b90a)
  • integrate sqllogictest runner and complete phase 1-2 oracle verification (b5a4243)
  • add case, distinct, and subquery test suites and fix function distinct parsing (6c1a22a)
  • expand conformance suites to 20 files and fix NOT IN/BETWEEN 3VL NULL evaluation (89ded2a)
  • expand conformance suites to 26 files and fix modulo, 3VL NOT/AND/OR, self-join alias, and CROSS JOIN (4c1880a)
  • add PGWire TCP executor, WiredTiger backend, parallel worker pool, and stress suites (DURA-SLT-2) (6117300)
  • complete Phase 1 critical domain stress suites and fix string concat and NOT IN NULL semantics (DURA-SLT-2) (ea0c0bb)
  • initiate Phase 2 SQLite select1/select2 suites and fix ordinal sorting and subquery aggregate planner (DURA-SLT-2) (c003365)
  • achieve 100% pass on official SQLite select1 through select5 suites (DURA-SLT-2) (119df98)
  • implement dynamic root selection and zero-cardinality fast-fail for large joins (OPT-LARGE-JOIN-1) (4caaf4b)
  • default to pure AST planner, isolate legacy regex option, and lazy-evaluate merge join cost (PLANNER-SLIM-1, PLANNER-AST-1) (befec21)
  • achieve 100% pass across full 107-file sqllogictest suite and add verify_0.90.sh (142e894)
  • support PostgreSQL Writable CTEs and ::type[] AST casts (a9590dd)
  • implement session ephemeral CTE buffer to eliminate DDL lock serialization (7edecc8)
  • implement key-striped latch, PGWire binary numeric, and SP insert-select (6f79682)
  • implement in-kernel transparent write-conflict retry safety net (9bce177)
  • implement multi-table TRUNCATE, CASCADE support, and Stored Procedure OpTruncate (2e5be34)
  • lower linear writable CTE to VDBE bytecode VM (PERF-CTE-LOWERING-VDBE-1) (dcc6eb2)
  • zero-map slot indexing for local variables and parameters in VDBE VM (PERF-VDBE-SLOT-1) (8791bbc)
  • zero-allocation DML record pooling and AST expression bypass in VDBE VM (PERF-VDBE-DML-POOL-1) (00ac704)
  • flatten VDBE VM Execute dispatch into single inline loop (PERF-VDBE-FLATTEN-1) (56111fb)
  • add all-in-one comprehensive benchmark suite and SP wire fast-path tuning (28f0475)

Fixed

  • report write conflicts as serialization failure (SQLSTATE 40001) (DURA-SOAK-F1) (c8d2288)
  • make HasDisjunction whitespace-invariant and literal-safe (f270a5f)
  • ensure wiredtiger cross-session cursor eviction thread safety and in-memory occ atomic scan snapshots (793dd80)
  • resolve 4 extended suite gaps across parser, planner, proc, and aggregate engine (4a13bd7)
  • resolve parser and runner technical debt for SQLLogicTest (30b309d)
  • support GENERATED AS IDENTITY in DDL and prevent DEFAULT AS clash (d46acc1)
  • add timeout to key latch to prevent deadlocks and decouple session cancel lock (d86518a)
  • prevent compound condition hijacking in parseInCondition with regression test (bf94625)
  • restore wiredtiger transaction_sync and isolate sqlite fuzz suites from release gate (64e2942)

Performance

  • implement direct transaction fast-path binding for pgwire and tds listeners (b713b99)
  • pool pgwire 32kb output writers and add zero-parsing fast-path ping for select 1 (b0a4608)
  • consolidate tpcc-client into 1-roundtrip stored procedure rpc calls (10w/20w/32w full metrics updated) (1aed764)
  • bind PGWire prepared statements directly to VDBE in transaction (PERF-PGWIRE-PREP-TX) (8511c52)
  • optimize Selinger DP join threshold to 6 with greedy fallback (OPT-LARGE-JOIN-1) (b85ed2a)
  • eliminate string allocations in cleanSymbol and fast-path slot lookup in getVar (PERF-VDBE-VAR-FAST-1) (75e2c2d)

Documentation

  • permanently spec-out legacy mysql tabular explain grid (42ef8b9)
  • format 3.6 client benchmark with 3.5-symmetrical OCC vs WiredTiger tables across all workloads (287fd1d)
  • add bilingual integration guides and milestone reports for IOTA-SDK and Gitea (6ebf2bd)
  • add Grafana dual-protocol integration plan (GRAFANA-INTEGRATION-1) (7db6458)
  • add bilingual Grafana integration guides and milestone reports (GRAFANA-INTEGRATION-1) (7195509)
  • add Casdoor IAM/SSO dual-protocol integration plan (CASDOOR-INTEGRATION-1) (2d433fd)
  • add bilingual integration guides and milestone reports (CASDOOR-INTEGRATION-1) (143ca7b)
  • add Gitea, IOTA-SDK, Grafana, and Casdoor integration guides to doc index (756ca42)
  • register next integration candidates (Answer, NocoDB, Metabase, Umami, Vaultwarden, Mattermost) (4fbd3cd)
  • add Apache Answer PGWire integration plan (INTEG-ANSWER-1) (0a6b237)
  • add app integration expansion master plan for six candidates (626b68d)
  • add durability verification plan and register DURA-KILL/SOAK/DOGFOOD tasks (faadf73)
  • add 1-hour soak test verification and memory analysis report (DURA-SOAK-1) (e09b656)
  • document PostgreSQL wire protocol TPC-C benchmark results (PERF-PGWIRE-PREP-TX) (93bd7bd)
  • update 1:1 dual-protocol benchmark comparison matrix in BENCHMARK.md (31556bd)
  • expand PGWire TPC-C benchmark to multi-workload scalability matrix (32, 100, 200 workers) (910fb28)
  • restructure TODO dashboard to align with Grove 3-tier backlog model (e2999e8)
  • remove 6 completed and superseded PLAN files (38ec615)
  • archive 13 completed architectural plans and update index (7e155cc)
  • rename integration milestones to *_REPORT.ko.md and remove duplicate English summaries (7022dc8)
  • streamline MODULAR_ARCHITECTURE_API_CONTRACT_PLAN into 3-phase lean plan (c11d088)
  • simplify plan filename to MODULAR_ARCHITECTURE_PLAN.ko.md and update index (146389f)
  • register ARCH-VDBE-ROUTING-1 anti-heuristic refactoring task (518bcd5)
  • add structural AST and catalog routing refactoring plan (ROUTING-AST-CATALOG-1) (99c36c4)
  • consolidate implementation and benchmark results into AST catalog routing plan (448a5df)
  • add Metabase BI dual-protocol integration plan (INTEG-METABASE-1) (0e7171d)
  • realign macro milestones and 6 quality gates for 1.0 LTS (DOC-ROADMAP-REORG-1) (b5e4723)
  • restore full Phase 1-5 architectural milestones and execution details (3f91df2)
  • record completion of anti-pattern keyword matching elimination in compat-table-driven-1 (7bbd12d)
  • add sqllogictest 1.0 full suite certification plan and todo (9182562)
  • remove superseded integration plans 1-5 and mark completed plans (ca2aa68)
  • add standard status badges to README and README.ko.md (0a2d2f2)
  • update documentation and plans to reflect select1 through select5 completion (DURA-SLT-2) (0725fa7)
  • persist large join optimization plan adopting dynamic root selection and fast-fail pruning (OPT-LARGE-JOIN-1) (7915268)
  • update Unreleased section with planner slimming, pure AST transition, and large join optimizations (6e4a937)
  • add 1.0 GA validation handoff plan and register GA-VALID tasks (f89a72b)
  • record GA-VALID-1 and GA-VALID-2 test execution results (b4f7f05)
  • provide 1.0 GA full suite fetcher and user verification guide (0825e99)
  • add 107-file sqllogictest 100% verification report (74f2ba5)
  • add cross-dbms execution speed comparison appendix to slt report (009f0fe)
  • move 20 application integration guides from doc/ to root guide/ directory (e12f476)
  • update SQL_COMPATIBILITY_AND_ERGONOMICS_GUIDE status to production ready (e52f53a)
  • add english versions for release automation, sql compatibility, and transaction performance guides (c0dfdfa)
  • establish zero-regex SQL parsing invariant for production paths in GEMINI.md (95f0390)
  • add English SLT 100% verification report and link from homepages and Web Studio (41b60d4)
  • clean up architecture ASCII diagram for CJK font rendering in README.ko.md (6909752)
  • document Phase 5 in-kernel write conflict retry loop in plan and TODO (a6596e6)
  • update verification results and benchmark comparison in grove plan (578bf46)
  • register PERF-CTE-LOWERING-VDBE-1 architecture plan and backlog (0e752f5)
  • register PERF-VDBE-EXTREME-OPT-1 5-point extreme optimization plan and backlog (45370b7)
  • record grove 7,100 TPS benchmark breakthrough and hybrid micro-latching invariants (73803e3)
  • sync 1.0 scope lock, 7.1k TPS baseline, and VDBE optimization roadmap (b114e8f)
  • document grove hotspot architectural postmortem and 4 core assets (f407d23)
  • add proactive modular abstraction invariant and Rule 20 to golden rules (f9d884e)
  • introduce immutable rule slugs and deprecate volatile sequence numbers (0a69f11)
  • reorganize documentation index and add English user/reference manuals (cd12a11)

Changed

  • encapsulate session state (R-6-P1) and normalize coordinator injection (R-6-P3) (91589f2)
  • replace 45 string heuristics with AST and catalog symbol resolution (ROUTING-AST-CATALOG-1) (03d0457)
  • centralize ContainsKeyword and simplify HasDisjunction (KISS) (3bafc63)
  • unify keyword scanning into scanKeyword SSOT primitive (bb5c92c)
  • replace naive strings keyword matching with sqlutil primitives across engine, planner, virtual catalog, and wire (6bdf3f0)
  • eliminate remaining naive keyword matching anti-patterns in ddl, dml, filter, and planner (620fc15)
  • replace condition and command regexes with pure AST and lexer parsers (PLANNER-AST-2, ENGINE-AST-3) (84c74f8)
  • eliminate legacy regular expressions in DDL, eval, and explain paths (3c21ad5)
  • revert(ui): remove unintended report links from Web Studio dashboard HTML (f6cd033)
  • remove key latch and retry sleep to restore 108k/160k tps and establish perf gate (PERF-REGRESSION-RECOVERY-1) (9e0663b)
  • eliminate regexes for nested SQL structures and modularize predicate parsing in sqlutil (e17a4ab)
  • eliminate volatile rule numbers, abstract predicate parser, and resolve REV-ENG-011 and REV-WIRE-010 (ecadff2)

Added

  • add all-in-one comprehensive benchmark and verification suite (scripts/run_all_benchmarks.sh) automating multi-target compilation, banking SP, storage kernel TPC-C, VDBE runtime TPC-C, client-server PGWire/TDS TPC-C, SQLLogicTest conformance, and SIGKILL cold restart crash durability with automated markdown report generation (BENCH-ALL-IN-ONE-1)
  • implement zero-allocation Stored Procedure FastPath in PGWire and TDS prepared statements, statically analyzing procedure parameters at compile time to bind directly to VDBE VM bytecode without string serialization, achieving 236,399 burst tpmC (8,746 TPS) and 178,355 sustained tpmC on 16W/32Workers (TUNE-SP-FASTPATH-1)
  • establish permanent lightweight in-line performance regression gate test (TestRegression_TransferFundsPerfGate) in pkg/proc/runtime, executing 1,000 multi-worker ACID transactions in ~40ms to intercept locking convoys and latency regressions prior to commit (PERF-REGRESSION-RECOVERY-1)
  • implement zero-map slot indexing for local variables and parameters in T-SQL VDBE virtual machine, bypassing map lookups and string lowercasing on OpStoreVar/OpLoadVar and lazily syncing evalCtx only on AST row expressions, achieving 54,858 tpmC on TPC-C and 7,957 TPS on TransferFunds (PERF-VDBE-SLOT-1)
  • lower linear Writable CTE mutations directly into anonymous T-SQL VDBE bytecode programs with zero #temp tables and zero catalog locks, compressing lock hold duration from 15ms to ~60µs (250x reduction) (PERF-CTE-LOWERING-VDBE-1)
  • support OUTPUT ... INTO and RETURNING ... INTO in Pratt DML parser and wire values directly into VDBE variable slots (DMLData.IntoVars)
  • achieve 7,100 TPS (+145% vs PostgreSQL 16 2,890 TPS) on hotspot inventory reservation and 5,913 TPS (+73% vs PostgreSQL 16 3,419 TPS) on 3-way order creation under 50 concurrent workers with 0.00% error rate on Apple Silicon
  • support multi-table TRUNCATE TABLE t1, t2, ... and TRUNCATE t1, t2, CASCADE, RESTRICT, RESTART IDENTITY, and CONTINUE IDENTITY options across In-Memory OCC and WiredTiger storage engines, resolving mutual foreign key dependencies without false-positive dependency violations (DDL-TRUNCATE-MULTI-CASCADE-1)
  • support native TRUNCATE TABLE statements in T-SQL Stored Procedure VM (OpTruncate) with full ACID transactional rollback support (PROC-SP-TRUNCATE-1)
  • implement in-kernel transparent write-conflict retry safety net with exponential backoff and random micro-jitter (50µs~250µs) for WiredTiger WT_ROLLBACK (-31800) and OCC serialization failures, achieving 100% success rate under high write contention (PERF-RETRY-WT-ROLLBACK-1)
  • implement 2,048-stripe in-memory key latch manager (KeyLatchManager) in storage layer to serialize concurrent updates on identical primary keys, preventing OCC abort storms and achieving 16,000+ TPS across 50 concurrent workers (PERF-HOTSPOT-LATCH-1)
  • support INSERT INTO ... SELECT statements natively in T-SQL Stored Procedure VM via query execution coordinator bridge (PROC-SP-INSERT-SELECT-1)
  • implement PGWire Describe statement metadata caching (ColumnOIDs, ColumnNames, NoData) to bypass redundant Writable CTE executions in extended query pipeline (PERF-PGWIRE-PIPELINE-1)
  • support PostgreSQL binary NUMERIC / DECIMAL (OID 1700) Base-10,000 encoder and decoder in PGWire server, resolving pgx driver scan errors (BUG-WIRE-OID-1700)
  • implement session-local ephemeral in-memory CTE buffer and SliceCursor to eliminate storage DDL and catalog lock serialization, achieving linear scale under 50+ concurrent workers (PERF-CTE-EPHEMERAL-1)
  • support PostgreSQL standard GENERATED { ALWAYS | BY DEFAULT } AS IDENTITY clauses in DDL parser and prevent reColDefault from mistakenly extracting "AS" as column default, fixing goose migration history table PK duplication errors (SQL-GOOSE-IDENTITY-1)
  • support PostgreSQL Writable CTEs (WITH ... INSERT/UPDATE/DELETE ... RETURNING) with multi-CTE chaining, post-update RETURNING consumption in INSERT ... SELECT, 0-row schema preservation, unreferenced CTE materialization, single-statement atomic rollback (23505), PGWire SELECT $N target-column parameter OID inference, and ::type[] array cast AST parsing (SQL-WRITABLE-CTE-1)
  • default to pure AST planner (proc.ParseSQL -> FromAST) and isolate legacy regex planner fallback behind --enable-legacy-regex-planner option (default false) (PLANNER-AST-1)
  • support OVER (...) window clauses on FunctionCallExpr and WithinGroupExpr, as well as WITH RECURSIVE CTE column lists and UNION ALL bodies natively in the Pratt AST parser (PLANNER-AST-1)
  • implement dynamic root table selection (optimizeRootTable) and zero-cardinality fast-fail pruning (NodeEmpty / EmptyIterator) for large multi-table joins (OPT-LARGE-JOIN-1)
  • achieve 100% pass across official SQLite SQLLogicTest suites (select1.test through select5.test, 10,706 queries in 22.69s) verifying complex multi-table joins, aggregations, and subqueries with zero hash mismatches (DURA-SLT-2)
  • add BFS spanning tree join inference in predicate pushdown to eliminate Cartesian cross-join explosions in implicit comma joins (DURA-SLT-2)
  • add greedy join ordering filter prioritization in CBO planner to collapse intermediate join cardinalities early on large join trees up to 64 tables (DURA-SLT-2)
  • add PGWire TCP socket executor (PGWireExecutor), WiredTiger isolated disk backend (-engine wiredtiger), parallel worker pool (-parallel), and expression/aggregation stress suites (expr_stress.slt, agg_stress.slt) to SQLLogicTest harness (cmd/sqllogictest, pkg/slt) (DURA-SLT-2)
  • formalize table-driven compatibility architecture across PGWire parameter OID inference, command completion tag formatting, and virtual catalog view routing (COMPAT-TABLE-DRIVEN-1)
  • complete Woodpecker CI integration over PGWire (:5433, Go, XORM, 19 tables, 32 migrations, UI and healthcheck) (INTEG-WOODPECKER-1)
  • complete Vikunja project Kanban and task management integration over PGWire (:5433, Go, XORM, 80+ migrations, recursive CTEs, user registration, JWT auth, and task CRUD) (INTEG-VIKUNJA-1)
  • support PostgreSQL recursive common table expressions (WITH RECURSIVE) with iterative convergence evaluation loop up to depth 100
  • complete Umami web analytics platform integration over PGWire (:5433, Next.js 16.4.2, Prisma ORM, 26 migrations, event tracking, and stats aggregation) (INTEG-UMAMI-1)
  • complete Metabase BI dual-protocol integration across PGWire (:5433, org.postgresql.Driver) and MSSQL TDS (:1433, com.microsoft.sqlserver.jdbc.SQLServerDriver) with 100% schema sync, MBQL visual query builder, multi-table derived joins, and native SQL support (INTEG-METABASE-1)
  • support Microsoft JDBC Driver metadata temp table multi-statement batches (DECLARE @temp TABLE / INSERT INTO @temp EXEC sp_columns_170 / SELECT ... FROM @temp) and implement sp_pkeys, sp_fkeys stored procedures
  • complete Vaultwarden password manager integration over PGWire (:5433, 47 Diesel ORM migrations, composite primary keys, cipher and folder CRUD, sync API) (INTEG-VAULTWARDEN-1)
  • complete Apache Answer Q&A knowledge-base platform integration over PGWire (:5433, 35 tables, XORM migrations, admin login, question/answer E2E) (INTEG-ANSWER-1)
  • implement IdentityProvider SPI in VDBE execution runtime and synchronize table identity counters across VDBE and SQL coordinator
  • add cmd/crashtest kill-9 crash recovery harness with 4-point durability oracle (D1~D4) and DOUBLE_KILL, TORN_WAL_TAIL, and DDL variants across In-Memory OCC and WiredTiger (DURA-KILL-1)
  • add cmd/soaktest long-duration soak test tool (steady mixed PGWire workload, RSS/FD/disk/p99 trend verdict, balance invariant and acknowledged-commit integrity check) (DURA-SOAK-1)
  • complete Casdoor IAM/SSO/OIDC dual-protocol integration across PGWire (:5433) and MSSQL TDS (:1433) (44 XORM tables, Casbin policies, OIDC/JWKS, and Auth REST API) (CASDOOR-INTEGRATION-1)
  • support composite primary keys (PRIMARY KEY (col1, col2, ...)) in WiredTiger and In-Memory OCC storage engines via \x1f unit-separator physical key composition
  • tokenize unquoted 0x hex binary literals in T-SQL Pratt lexer and decode "0x..." strings as TDS TypeBigVarBin (0xA5) raw bytes for XORM VARBINARY JSON serialization
  • complete Grafana dual-protocol integration across PGWire (:5433 internal backend, 785 migrations) and dual data source plugins (PostgreSQL :5433 & MSSQL TDS :1433) (GRAFANA-INTEGRATION-1)
  • support PostgreSQL advisory lock built-ins (pg_try_advisory_lock, pg_advisory_unlock, etc.) with boolean OID 16 metadata
  • support SQL-standard EXTRACT(field FROM source), EPOCH datepart, and statement_timestamp() / clock_timestamp() built-ins
  • implement PostgreSQL BYTEA hex escape (\x) round-trip serialization for envelope encryption
  • complete IOTA-SDK Go ERP integration over PGWire (:5433, 47 schema migrations, pgx/v5 binary UUID & TIMESTAMPTZ encoding) (IOTA-ERP-1)

Removed

  • remove KeyLatchManager and RunContext sleep-retry loops to resolve multi-key deadlock convoys and latency stalls, restoring WiredTiger throughput to 108,000+ TPS and In-Memory OCC to 160,000+ TPS (PERF-REGRESSION-RECOVERY-1)

Fixed

  • fix cross-transaction key latch deadlock during multi-worker transfer soak tests by introducing 250ms safety timeout (storage.DefaultKeyLatchTimeout) returning ErrSerializationFailure (SQLSTATE 40001) and decoupling ClientSession cancelMu from cs.Mu
  • prevent non-ephemeral multi-statement transactions from prematurely locking key latches during DML updates, preserving OCC snapshot isolation and write-conflict detection
  • bypass static PK lookup optimization for temporary and session ephemeral tables (#...) in query coordinator to prevent ErrTableNotFound false positives on CTE queries (PERF-CTE-EPHEMERAL-1)
  • prevent implicit comma joins from degrading into multi-million row Cartesian cross joins via BFS spanning tree equijoin discovery in pkg/planner/pushdown.go (DURA-SLT-2)
  • verify unqualified outer column references against available table schemas in checkAndOrientJoinKeys before reordering join keys (DURA-SLT-2)
  • treat space-containing string cells in official SQLite .test suites as single values per line in SLT parser (DURA-SLT-2)
  • optimize join key formatting in pkg/execution/join.go via fast-path strconv.Itoa/strconv.FormatInt and direct storage.Value serialization (DURA-SLT-2)
  • distinguish subquery-internal aggregates in SELECT projections from outer group aggregates in CBO planner via hasTopLevelAggregateExpr, preventing single-row truncation of scalar subquery projections (DURA-SLT-2)
  • support 1-based ordinal position sorting (e.g. ORDER BY 1, 2) in Volcano CBO planner and SortIterator by placing SortNode post-projection and indexing tuple slots (DURA-SLT-2)
  • support standard SQL string concatenation operator (||) across T-SQL Pratt parser and Volcano/row expression evaluators (DURA-SLT-2)
  • enforce SQL 3-valued logic UNKNOWN semantics for NOT IN (SELECT ...) subqueries containing NULL keys in HashJoinIterator ANTI joins, suppressing all outer rows when inner null exists (DURA-SLT-2)
  • extract and accumulate nested aggregate calls inside composite SELECT expressions (such as COALESCE(SUM(col), 0) or SUM(a) + SUM(b)) via two-pass group materialization in AggregateIterator (DURA-SLT-2)
  • preserve right-hand parentheses for non-associative binary operators (-, /, %) in BinaryExpr.String() AST deparser
  • replace hardcoded switch statements in pkg/wire and pkg/catalog/virtual with immutable O(1) table-driven hash maps, eliminating ad-hoc branching and ensuring zero-allocation lookups (COMPAT-TABLE-DRIVEN-1)
  • normalize PGWire VDBE command completion tags from generic "DML" to "UPDATE/INSERT/DELETE " so XORM and pgx accurately track RowsAffected
  • guard index seek candidate selection against conditions containing " OR " to prevent multi-predicate disjunctions from being misparsed as single-column equality bounds
  • assign globally unique sequence identifiers to CTE materialized tables and bind explicit primary keys to prevent temporary table key collision during concurrent CTE executions
  • isolate CTE prefix from outer queries in Describe row descriptions to prevent intermediate derived table column names from corrupting main query metadata
  • bind dollar parameters ($1, $2, ...) in VM variable binding loop for complex WHERE expressions
  • support PostgreSQL ::type casting and EXISTS expressions in scalar SELECT statements to prevent Prisma can-connect-to-database recursion
  • infer array parameter types for = ANY($1) clauses (OID 1009 _text, OID 1028 _oid) and decode binary array payloads in PGWire
  • allow USING clause in parseDeleteBody and export planner.ParseCTEs for complex CTE-based DELETE statements
  • extract aliases upfront in AggregateIterator.parseFields using sqlutil.SplitAsAlias to preserve aliases with special characters (_count$_all) and evaluate COUNT(*) accurately
  • strip table prefixes in virtual catalog FormatResult to correctly report existing tables to Prisma introspection queries
  • support quoted identifiers (double quotes, backticks, brackets) in derived table subquery alias extraction to prevent AS keyword duplication
  • map hyphenated aliases (table-name, database-type, database-position, pk?) in virtual catalog projection mapping to underlying schema fields
  • support bracketed aliases and HAS_PERMS_BY_NAME in sys.objects metadata queries for official SQL Server JDBC drivers
  • split and execute multi-action compound ALTER TABLE statements sequentially
  • handle whitespace and newline formatting in ADD COLUMN DDL statements
  • support parenthesized ANSI joins in FROM clauses (e.g. FROM ("t1" JOIN "t2"))
  • resolve composite primary key values in ON CONFLICT DO UPDATE upsert statements
  • evaluate DEFAULT keyword in INSERT column values to default expressions or NULL
  • bind datetime columns to PostgreSQL OID 1114 (TIMESTAMP) with 8-byte binary microsecond encoding and resolve DISTINCT column types
  • auto-generate sequential identity values in VDBE OpInsert during multi-row batch inserts when identity column is omitted
  • bypass VDBE compilation for virtual catalog queries (pg_tables, sys.*) and DML statements with RETURNING, OUTPUT, or ON CONFLICT clauses
  • prevent session variable substitution from textually replacing @@ system variables in SQL queries
  • enable WiredTiger transaction_sync=(enabled=true,method=none) to flush commit records to OS buffers on commit and make saveCatalog atomic via rename so committed transactions survive SIGKILL (DURA-KILL-1)
  • prevent CREATE TABLE from swallowing columns prefixed with CHECK, UNIQUE, or CONSTRAINT (e.g. checksum, unique_id) as table-level constraints
  • guard PGWire untyped text parameter decoding with strict integer/float lexical checks so hex strings containing 'e' or leading zeros are not misparsed as numbers
  • map WiredTiger WT_ROLLBACK write conflicts to storage.ErrSerializationFailure and report them as PGWire SQLSTATE 40001 instead of XX000 so clients can retry (DURA-SOAK-F1)
  • decouple logical table rename from physical WiredTiger table URI via TableSchema.PhysicalName to prevent EBUSY errors during active transactions
  • evict cached WiredTiger cursors in DropTable and assign unique PhysicalName for temporary tables to eliminate derived table key collisions
  • expand wildcard projections (alias.*, *) in PGWire handleDescribe and evaluate bare literal projections in Volcano ProjectionIterator
  • fix TDS inferColumnInfos colSize for 'type' columns to prevent go-mssqldb slice bounds panic

Refactored

  • eliminate 22 condition, selectivity, partition pruning, and subquery regular expressions across pkg/planner (bounds.go, pruning.go, stats.go, decorrelate.go, explain.go, planner.go) in favor of Pratt AST (proc.Expr) and lexical token scanners (PLANNER-AST-2)
  • eliminate 40 administrative and utility command regular expressions across pkg/engine (sqlexec.go, sqlexec_auth.go, sqlexec_pivot.go, sqlexec_apply.go, sqlexec_string_split.go, sqlexec_procs.go) in favor of lexical token stream parsers (ENGINE-AST-3)
  • unify keyword scanning into pkg/sqlutil SSOT tokenizer primitives (ContainsKeyword, HasPrefixKeyword, FindTopLevelKeyword), eliminating naive strings.Contains keyword matching anti-patterns across planner, engine, wire, and virtual catalog (COMPAT-TABLE-DRIVEN-1)
  • replace 45 fragile raw string substring and regexp heuristics across query routing, virtual catalog interception, and PGWire OID resolution with structural AST characteristics (proc.ExtractQueryCharacteristics) and catalog symbol table resolution (virtual.IsVirtualTable) (ROUTING-AST-CATALOG-1)
  • encapsulate ClientSession internal mutex and transaction state with high-level methods (ExecuteProgram, IsTxFailed, MarkTxFailed, InTransaction, GetActiveTx, TxStatus) and eliminate raw cs.Mu.Lock across PGWire and TDS listeners (R-6-P1)
  • normalize server coordinator lifecycle by instantiating engine.Coordinator as core instance and directly injecting it into PGWire, TDS, and HTTP listeners (R-6-P3)

Performance

  • defer Sort-Merge Join cost evaluation (O(N log N)) in CBO buildSingleJoinNode to lazy evaluation, bypassing string slicing and math.Log2 computations unless a MERGE JOIN hint or pre-sorted input is present (PLANNER-SLIM-1)
  • optimize Selinger dynamic programming join threshold from 12 to 6 (MaxJoinDPThreshold = 6), bounding exponential state space to 64 subsets (<2µs) and switching 7+ table joins immediately to greedy join ordering (OPT-LARGE-JOIN-1)
  • bind PGWire prepared statements directly to VDBE bytecode VM in active transactions via ExecuteInTxWithSessionContext, eliminating substituteParams string reconstruction and SQL re-parsing overhead (PERF-PGWIRE-PREP-TX)
  • implement zero-overhead transaction fast-path binding for BEGIN, COMMIT, and ROLLBACK in PGWire and TDS listeners (PERF-PGWIRE-1)
  • reuse 5-byte message header buffer across connection message loop in PGWire server

Documentation

  • author planner slimming and pure AST transition design plan (notes/PLANNER_SLIMMING_AND_PURE_AST_TRANSITION_PLAN.ko.md) and large join optimization plan (notes/LARGE_JOIN_OPTIMIZATION_PLAN.ko.md)
  • author Woodpecker CI and Vikunja integration reports (notes/WOODPECKER_AND_VIKUNJA_INTEGRATION_REPORT.ko.md) and bilingual guides (doc/WOODPECKER_INTEGRATION_GUIDE*.md, doc/VIKUNJA_INTEGRATION_GUIDE*.md)
  • author Metabase BI and Umami integration reports (notes/METABASE_INTEGRATION_REPORT.ko.md, notes/UMAMI_INTEGRATION_REPORT.ko.md) and bilingual guides (doc/METABASE_INTEGRATION_GUIDE*.md, doc/UMAMI_INTEGRATION_GUIDE*.md)
  • author Vaultwarden integration report (notes/VAULTWARDEN_INTEGRATION_REPORT.ko.md) and bilingual deployment guides (doc/VAULTWARDEN_INTEGRATION_GUIDE.ko.md, doc/VAULTWARDEN_INTEGRATION_GUIDE.md)
  • author Apache Answer integration report (notes/ANSWER_INTEGRATION_REPORT.ko.md) and bilingual deployment guides (doc/ANSWER_INTEGRATION_GUIDE.ko.md, doc/ANSWER_INTEGRATION_GUIDE.md)
  • author bilingual Casdoor IAM/SSO integration guides (doc/CASDOOR_INTEGRATION_GUIDE.ko.md, doc/CASDOOR_INTEGRATION_GUIDE.md) and milestone reports (notes/CASDOOR_INTEGRATION_MILESTONE.ko.md, notes/CASDOOR_INTEGRATION_MILESTONE.md)
  • author bilingual Grafana integration guides (doc/GRAFANA_INTEGRATION_GUIDE.ko.md, doc/GRAFANA_INTEGRATION_GUIDE.md) and milestone reports (notes/GRAFANA_INTEGRATION_MILESTONE.ko.md, notes/GRAFANA_INTEGRATION_MILESTONE.md)
  • author bilingual IOTA-SDK Go ERP and Gitea integration guides and milestone reports