Skip to content

fix(sql): raise 42703 for unknown column references instead of folding to NULL - #299

Closed
EnRaiha wants to merge 1 commit into
NodeDB-Lab:mainfrom
EnRaiha:fix/issue292-unknown-column-42703
Closed

fix(sql): raise 42703 for unknown column references instead of folding to NULL#299
EnRaiha wants to merge 1 commit into
NodeDB-Lab:mainfrom
EnRaiha:fix/issue292-unknown-column-42703

Conversation

@EnRaiha

@EnRaiha EnRaiha commented Sep 6, 2026

Copy link
Copy Markdown
Contributor

fix(sql): raise 42703 for unknown column references instead of folding to NULL

Fixes #292

Problem

A column reference that names nothing in scope planned as a field lookup and evaluated to NULL on every row. On collections whose row shape is known at plan time that is silent misbehavior: projection returned NULL-filled rows, WHERE matched nothing (IS NULL matched everything), ORDER BY no-oped, and UPDATE/DELETE predicates touched zero rows — no error anywhere. This is the same folding class as the undefined-function and division-by-zero paths that now raise (see the #226 fix), on the column-resolution side, which still folded.

Fix

Closed-schema collections validate every column reference at plan time and raise SQLSTATE 42703 (undefined_column):

  • document_strict, kv, columnar, timeseries and spatial are closed by construction.
  • document (schemaless) is closed when the collection declares columns. The open/closed line is the declared column count, not a name: the catalog adapter always prepends the synthesized primary-key column (the declared PK name or the built-in id), so a one-column list means nothing was declared and dynamic fields keep resolving — the NULL fold stays the documented behavior there. Declared fields close the set even though schemaless ColumnInfo carries raw_type: None.
  • Schemaless writes stay open: INSERT column lists may still carry undeclared dynamic fields.

The gate covers the single-table SELECT path (projection, WHERE, GROUP BY, HAVING, ORDER BY) and the UPDATE/DELETE target path (SET targets and predicates). Qualified references validate against the collection they name.

The error is typed end to end: SqlError::UnknownColumn maps to a new crate::Error::UndefinedColumn, to the public NodeDbError code 1206 (msgpack tag 79), and to SQLSTATE 42703 on pgwire and the native protocol — mirroring the undefined_function path added for the same folding class.

Out of scope

Pre-existing folds, tracked separately: JOIN/lateral/derived paths, aggregate ORDER BY, UPDATE ... FROM predicates, INSERT ... SELECT.

Verified

Query (declared doc / strict / kv / columnar) Before After
SELECT ghost N rows NULL 42703
WHERE ghost = 1 / IS NULL 0 / all rows 42703
ORDER BY ghost storage order 42703
UPDATE/DELETE ... WHERE ghost 0 rows 42703
auto-id / renamed-PK-only schemaless SELECT dyn_field fold / resolves unchanged
  • Unit: cargo nextest run -p nodedb-sql --lib — 875 passed (11 validator tests: every clause, closed/open/renamed-PK, function-arg walk, GROUP BY alias, sort-key ordinals)
  • Unit: cargo nextest run -p nodedb-types --lib — 688 passed
  • Wire (new undefined_column_42703): 4 passed — all six clause shapes on the default-declared engine; strict/kv/columnar; open fold incl. renamed-PK carve-out
  • Regression sentinels: engine_surface_document_schemaless, sql_json_path_ops, ddl_numeric_width_schemaless, sql_typeguard_defaults — 49 passed
  • cargo fmt --all -- --check clean; clippy -D warnings clean on nodedb-sql, nodedb-types, nodedb

How to test

cargo nextest run -p nodedb-sql --lib
cargo nextest run -p nodedb-types --lib
cargo nextest run -p nodedb --test wire -E 'test(undefined_column_42703) or test(engine_surface_document_schemaless) or test(sql_json_path_ops) or test(ddl_numeric_width_schemaless) or test(sql_typeguard_defaults)'

Copilot AI lite review requested due to automatic review settings September 6, 2026 17:57

Copilot AI left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Copilot was unable to review this pull request because the user who requested the review has reached their quota limit.

@EnRaiha EnRaiha added sev:2-high Major functionality broken; no acceptable workaround priority:P1 Fix in the current milestone status:needs-triage Awaiting maintainer triage (severity + priority) area:sql Parser, planner, SQL semantics engine:document Document engine (schemaless + strict) engine:kv Key-Value engine labels Sep 6, 2026
…g to NULL

A column reference that names nothing in scope planned as a field lookup and
evaluated to NULL on every row. On collections whose row shape is known at
plan time that is silent misbehavior: projection returned NULL-filled rows,
WHERE matched nothing (IS NULL matched everything), ORDER BY no-oped, and
UPDATE/DELETE predicates touched zero rows — no error anywhere. The
function-existence gate already raised for undefined functions; column
resolution had no counterpart.

Closed-schema collections now validate every reference at plan time and
raise SQLSTATE 42703 (undefined_column):

- document_strict, kv, columnar, timeseries and spatial are closed by
  construction.
- document (schemaless) is closed when the user declared columns on it; a
  collection carrying only the auto-injected id stays open, so dynamic
  fields that appear only in the data keep resolving (the NULL fold remains
  the documented behavior there).

The gate covers the single-table SELECT path (projection, WHERE, GROUP BY,
HAVING, ORDER BY) and the UPDATE/DELETE target path (SET targets and
predicates). Qualified references validate against the collection they name.
Schemaless writes stay open: INSERT column lists may still carry undeclared
dynamic fields.

The error is typed end to end: SqlError::UnknownColumn maps to a new
crate::Error::UndefinedColumn, to the public NodeDbError code 1206 with
msgpack tag 79, and to SQLSTATE 42703 on pgwire and the native protocol —
mirroring the undefined_function path added for the same folding class.

Out of scope (pre-existing folds, tracked separately): JOIN/derived/lateral
paths, aggregate ORDER BY, UPDATE ... FROM predicates, and INSERT ... SELECT.

Fixes NodeDB-Lab#292
@EnRaiha
EnRaiha force-pushed the fix/issue292-unknown-column-42703 branch from 14701a0 to d17c3c4 Compare September 6, 2026 22:48
@farhan-syah farhan-syah removed sev:2-high Major functionality broken; no acceptable workaround priority:P1 Fix in the current milestone status:needs-triage Awaiting maintainer triage (severity + priority) labels Sep 6, 2026
@farhan-syah farhan-syah closed this Sep 7, 2026
@farhan-syah

farhan-syah commented Sep 7, 2026

Copy link
Copy Markdown
Member

Superseded by #304, merged as 66d0a225c. #292 is closed.

#304 threads a ColumnScope through convert_expr, the single point every clause funnels an identifier through. The gate therefore reaches JOIN, LATERAL, derived tables, aggregate arguments, UPDATE ... FROM predicates, and INSERT ... SELECT.

The two differ on one point, worth recording for anyone reading back. This PR closes a schemaless collection once it declares columns. #304 keeps document_schemaless open whatever its declared column count. docs/documents.md:43 states undeclared fields pass freely on write, so closing the read side makes written data unreadable.

#304 was written independently and did not draw on this PR.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

area:sql Parser, planner, SQL semantics engine:document Document engine (schemaless + strict) engine:kv Key-Value engine

Projects

None yet

Development

Successfully merging this pull request may close these issues.

Unknown column references resolve to NULL instead of raising 42703 — WHERE/ORDER BY/UPDATE silently misbehave, including on fixed-schema engines

3 participants