Skip to content

v1.0.27

Choose a tag to compare

@brunolau brunolau released this 03 Oct 14:08

v1.0.27

Fix: a membership condition over an empty list binds nothing for its operand

inArray / notInArray render an empty list — or a value that is not an array — as a constant: 1=0 (no row
matches, a NULL operand included) and 1=1 (every row matches). So do inArrayOpt / notInArrayOpt up to their
threshold, and with them plain inArray / notInArray under LinkgressConfig.inArrayUsesOpt, with or without a pad
ladder. They rendered their OPERAND first and only then returned the constant, so an operand that binds parameters —
coalesce(jsonbPathText(col, param('k', 'text')), ''), a sql fragment with values, lower(param(…)) — left its
parameters in the statement's list with no $N referencing them. A plain column binds nothing, which is why it went
unnoticed. PostgreSQL — and the in-memory database, which mirrors it — refuses such a statement (could not determine data type of parameter $N, or bind message supplies N parameters, but prepared statement "" requires M), wherever
the condition sits: a where(), an and() / or() / not(), a correlated exists(), a CTE body, a projected
boolean, an update() / delete() filter, a bulk update's where, a QueryBatch or MutationBatch leg — a batch
moves the orphan into the middle of the combined list.

await db.widgets
  .where(w => inArrayOpt(coalesce(jsonbPathText(w.attributes, param('tier', 'text')), ''), []))
  .toList();
// before: WHERE 1=0 — params ['tier', ''], referenced by no $N: the statement is refused
// now:    WHERE 1=0 — params []
  • The list is checked before the operand renders. The constants and what they mean are unchanged: x IN () is FALSE
    and x NOT IN () TRUE for every row, NULL x included.
  • Joins stay as they were: the operand's column refs still reach join detection, so a navigation it reads is joined
    as before — and an UPDATE / DELETE, which joins it as FROM / USING, still reaches the same rows. Only the
    operand's text and parameters are gone.
  • A sql.placeholder() inside such an operand is no longer registered by prepare(): execute() stops asking for a
    value the statement never reads.
  • A non-empty list renders exactly as before: the SQL and parameters of 6 872 non-empty cases (every builder, operand
    shape and context below) are byte-identical to the previous build's.

Fix (in-memory database): x <> ALL(<empty array>) keeps an outer join's null-extended rows

The in-memory engine turned a LEFT JOIN into an inner join whenever a WHERE qual compared a column of its nullable side
through op ANY / ALL (array). Over an empty array x <> ALL (…) is TRUE for every x, NULL included, so the
reduction dropped rows PostgreSQL returns: neAll(w.maker.name, []) lost the widgets without a maker. Like
PostgreSQL (is_strict_saop), an ALL now rejects a NULL only over an array known to hold an element — a constant
with one, ARRAY[…] listing one; = ANY (…) and NOT IN (…) reduce as before.

Tests

  • tests/queries/membership-empty-list-params.test.ts (new) — 9 646 generated cases: inArray, notInArray (each
    also under inArrayUsesOpt), inArrayOpt, notInArrayOpt, eqAny, neAll, arrayOverlaps, arrayContainsAll,
    arrayContainedBy, jsonbHasAnyKey, jsonbHasAllKeys × 13 contexts (where, with the navigation also projected,
    inside and / or / not, a correlated exists() over a collection and over a subquery, a CTE body, a projected
    boolean, a QueryBatch between two bound branches, a MutationBatch bulk-update where between two bound legs,
    update().where(), delete().where()) × operand shapes (a column, a navigation of 1 and 2 hops, expressions binding
    1 and 2 parameters, one through 2 hops, a column with a toDriver mapper) × lists (empty, not an array, 1, 2, the
    threshold, threshold + 1) × no / a pad ladder. Each asserts (a) every bound parameter of every statement it sends is
    referenced by a $N and every $N bound, (b) the statements run, (c) the rows equal a JS oracle's over the seeded
    data, with SQL's three-valued logic. Before: 912 failures on PostgreSQL, in memory and on PGlite (in memory 24 more,
    the ALL reduction); after: 0.
  • tests/memory/sql-parity-corpus.ts, "joins": ALL / ANY over an outer join's nullable side, empty and not.
  • tests/utils/param-audit.ts (new), installed by tests/setup.ts: every statement a built-in client sends with
    parameters must reference each of them and bind each $N it references — refused otherwise. Run over the whole suite
    on PostgreSQL, in memory and on PGlite, it found nothing outside the new cases.

Files Changed

  • src/query/conditions.ts (InComparison.buildSql, NotInComparison.buildSql)
  • src/memory/engine/exec/relids.ts (nonNullableRels, holdsAnElement)
  • docs/guides/querying.md ("Matching a List of Values")
  • tests/queries/membership-empty-list-params.test.ts, tests/utils/param-audit.ts, tests/setup.ts,
    tests/memory/sql-parity-corpus.ts