Skip to content

LPTS 1.1.0

Latest

Choose a tag to compare

@ila ila released this 06 Jul 10:41
· 51 commits to main since this release

LPTS 1.1.0 — round-trip correctness, enforced.

This release makes "LPTS never rewrites a query wrongly" an enforced, repeatable invariant: a transparent round-trip checker, all ~3,300 files of DuckDB's own sqllogic corpus run through it, every wrong translation and invalid-SQL emission that surfaced fixed, and the result locked in as a regression gate. Across the corpus (files=3346), LPTS produces 0 wrong rewrites and 0 invalid-SQL failures — every untranslatable query is a deliberate, machine-recognizable LPTS_<CODE>: refusal.

Added

  • lpts_check session setting (SET lpts_check = true). Every top-level SELECT is transparently rewritten by LPTS and its result bag compared, by an order-independent hash, against the original. Strict mode raises on a divergence; a LPTS_CHECK_LOG log mode records one verdict per query. Nondeterministic queries are detected and excused.
  • DuckDB-corpus coverage gate — lpts_corpus_gate (a standalone C++ driver, no shell/Python dependency) runs DuckDB's entire test/sql/** suite through lpts_check in parallel and diffs per-file verdict counts against a committed baseline. New make coverage-check / make coverage-baseline targets; make test now runs the unit suite plus the gate. The gate fails on any new WRONG or FAIL.

Removed

  • PRAGMA lpts_exec('query'), the boolean-returning PRAGMA lpts_check('query'), and the lpts_check_log setting — all superseded by the lpts_check session setting above.

Changed

  • DuckDB target bumped to v1.5.4; coverage-gate baseline regenerated against v1.5.4's corpus.
  • Non-DuckDB dialect hardening (chiefly Spark): pipeline fusion disabled for the SPARK dialect (stable per-operator CTE layout for downstream consumers); BROADCAST(...) join hints emitted; ambiguous set-op / anti-join conditions over un-aliased self-joins qualified with their owning CTE; set-operation output columns routed through the shared name-dedup path (no more duplicate column names from a self-join UNION/EXCEPT/INTERSECT); untyped NULL rendered as bare NULL in non-DuckDB dialects.

Fixed

  • Wrong-results eliminated (→ 0): duplicate generated column names at UNION/EXCEPT/correlated-subquery boundaries (case-insensitively); 3-valued NULL semantics for multi-condition MARK joins (correlated EXISTS / row-IN); HAVING without GROUP BY; read_parquet named-argument preservation; multi-row scalar subqueries excused as nondeterministic.
  • Invalid-SQL eliminated (→ 0): constants with no literal form (unnamed structs, MAP/UNION/GEOMETRY/ENUM, NUL bytes); lambda arity/capture/shadowing; list-slice bracket syntax; WITH ORDINALITY and LATERAL table functions; struct-field pushdown paths; window RANGE offsets; quantile/string_agg argument recovery; reserved-word and unicode identifier quoting; and more.
  • Everything untranslatable now refuses with a standardized LPTS_<CODE>: message the gate distinguishes from bugs.

Validation

  • Coverage gate: files=3346 wrong=0 fail=0 (every non-rewritable query is a deliberate UNSUPPORTED refusal).
  • Unit suite: 2131 assertions across 45 test files.
  • SQLStorm TPC-H sf=0.001 (17,036 queries under lpts_check): 16,142 success, 0 incorrect.

Compatibility

  • DuckDB target: v1.5.4
  • extension-ci-tools target: v1.5-variegata
  • Release tag: v1.1.0

See CHANGELOG.md for the full per-fix detail.