Description
Writing to a table that carries a sqlite_autoindex_* silently corrupts
the database. The write returns success; the index is left stale.
src/schema/ddl_reader.rs::index_schema recovers an index's column list
by parsing its sqlite_master.sql. Every autoindex SQLite creates for a
PRIMARY KEY/UNIQUE constraint has sql = NULL, so the reader returns
None and the index is dropped from TableSchema::indexes
(documented at ddl_reader.rs:106, locked in by
auto_index_with_null_sql_is_omitted).
Graceful degradation is right for a read — you lose the index as an
access path, answers stay correct. For a write it is data loss, because
the same list drives two jobs in src/codegen/stmt/insert.rs:
:698 — schema.indexes.iter().filter(|idx| idx.unique) feeds
emit_unique_check, so uniqueness is not enforced and a duplicate
is accepted.
:796 — emit_index_key_ops_from_regs maintains the indexes codegen
emitted for, so the index b-tree is not updated.
Measured against stock sqlite3 3.51.0: after inserting into a
stock-created composite-PK table, PRAGMA integrity_check reports
wrong # of entries in index, and count(*) answers from the stale
index and undercounts. Exit code is 0 throughout.
This blocks the SQE integration concretely: its iceberg_tables
catalog keys on (catalog_name, table_namespace, table_name) — exactly
this shape — and each row points at Iceberg metadata. SQE currently
works around it by dropping the declared primary key for a named unique
index and refusing writes to any catalog carrying a
sqlite_autoindex_*.
Oracle-derived rule
Measured on sqlite3 3.51.0 rather than assumed. Autoindexes are
numbered sqlite_autoindex_<table>_<n>, n starting at 1, in
declaration order — not primary-key-first:
| DDL |
autoindexes |
PRIMARY KEY (a, b) |
_1 -> (a, b) |
UNIQUE (a, b) |
_1 -> (a, b) |
PRIMARY KEY (a), UNIQUE (b), UNIQUE (c) |
_1 -> (a), _2 -> (b), _3 -> (c) |
a TEXT PRIMARY KEY, b TEXT UNIQUE |
_1 -> (a), _2 -> (b) — column-level counts |
UNIQUE (c), PRIMARY KEY (a, b) |
_1 -> (c), _2 -> (a, b) — declaration order wins |
a INTEGER PRIMARY KEY |
none — rowid alias |
PRIMARY KEY (a, b)) WITHOUT ROWID |
none — the PK is the table |
So: walk the CREATE TABLE text in order; every PRIMARY KEY or
UNIQUE constraint, column-level or table-level, claims the next
number, except a single INTEGER PRIMARY KEY column and the primary key
of a WITHOUT ROWID table.
Scope
src/schema/ddl_reader.rs:
- Keep an index row whose
sql is NULL and whose name matches
sqlite_autoindex_% as a deferred entry (name, table, root page)
rather than discarding it. read_schema_and_views is already
two-phase (pending_indexes), so this is the existing hook.
- In the attach phase, recover the deferred entry's columns from the
owning table's own sql using the rule above, and push a real
IndexSchema with unique: true.
- An autoindex that still cannot be resolved must not be silently
dropped — see the safety valve below.
Safety valve
Spec 010/Req 8 accepts either recovering the index or refusing the
write. Recovery is the fix, but it cannot be total: an autoindex whose
constraint this reader fails to parse must make the table read-only
rather than writable-and-corrupting. Precedent is spec 007/Req 1 —
a hot journal is detected and refused, because serving wrong pages
beats failing to open. Same argument, write side.
Non-goals
Creating sqlite_autoindex_* on CREATE TABLE — filed separately.
Both halves are needed for a complete fix (this one fixes adopting a
stock file; that one fixes producing one), but they are independent
changes with different blast radii.
Complexity
Estimate: medium
Reasoning: The oracle rule is already derived and tabulated above,
and the two-phase hook exists. The work is a constraint-list parser over
CREATE TABLE text (the reader already has column_list_span and
split_top_level_commas), the deferred-resolution wiring, the
read-only safety valve, and corpus tests against oracle-created files.
Bounded, but it touches the schema reader that every write path depends
on, so it needs real test coverage rather than a spot fix.
Acceptance Criteria
Refs: 010/Req-8, #678
Description
Writing to a table that carries a
sqlite_autoindex_*silently corruptsthe database. The write returns success; the index is left stale.
src/schema/ddl_reader.rs::index_schemarecovers an index's column listby parsing its
sqlite_master.sql. Every autoindex SQLite creates for aPRIMARY KEY/UNIQUEconstraint hassql = NULL, so the reader returnsNoneand the index is dropped fromTableSchema::indexes(documented at
ddl_reader.rs:106, locked in byauto_index_with_null_sql_is_omitted).Graceful degradation is right for a read — you lose the index as an
access path, answers stay correct. For a write it is data loss, because
the same list drives two jobs in
src/codegen/stmt/insert.rs::698—schema.indexes.iter().filter(|idx| idx.unique)feedsemit_unique_check, so uniqueness is not enforced and a duplicateis accepted.
:796—emit_index_key_ops_from_regsmaintains the indexes codegenemitted for, so the index b-tree is not updated.
Measured against stock
sqlite33.51.0: after inserting into astock-created composite-PK table,
PRAGMA integrity_checkreportswrong # of entries in index, andcount(*)answers from the staleindex and undercounts. Exit code is 0 throughout.
This blocks the SQE integration concretely: its
iceberg_tablescatalog keys on
(catalog_name, table_namespace, table_name)— exactlythis shape — and each row points at Iceberg metadata. SQE currently
works around it by dropping the declared primary key for a named unique
index and refusing writes to any catalog carrying a
sqlite_autoindex_*.Oracle-derived rule
Measured on
sqlite33.51.0 rather than assumed. Autoindexes arenumbered
sqlite_autoindex_<table>_<n>,nstarting at 1, indeclaration order — not primary-key-first:
PRIMARY KEY (a, b)_1 -> (a, b)UNIQUE (a, b)_1 -> (a, b)PRIMARY KEY (a), UNIQUE (b), UNIQUE (c)_1 -> (a),_2 -> (b),_3 -> (c)a TEXT PRIMARY KEY, b TEXT UNIQUE_1 -> (a),_2 -> (b)— column-level countsUNIQUE (c), PRIMARY KEY (a, b)_1 -> (c),_2 -> (a, b)— declaration order winsa INTEGER PRIMARY KEYPRIMARY KEY (a, b)) WITHOUT ROWIDSo: walk the
CREATE TABLEtext in order; everyPRIMARY KEYorUNIQUEconstraint, column-level or table-level, claims the nextnumber, except a single
INTEGER PRIMARY KEYcolumn and the primary keyof a
WITHOUT ROWIDtable.Scope
src/schema/ddl_reader.rs:sqlisNULLand whose name matchessqlite_autoindex_%as a deferred entry (name, table, root page)rather than discarding it.
read_schema_and_viewsis alreadytwo-phase (
pending_indexes), so this is the existing hook.owning table's own
sqlusing the rule above, and push a realIndexSchemawithunique: true.dropped — see the safety valve below.
Safety valve
Spec 010/Req 8 accepts either recovering the index or refusing the
write. Recovery is the fix, but it cannot be total: an autoindex whose
constraint this reader fails to parse must make the table read-only
rather than writable-and-corrupting. Precedent is spec 007/Req 1 —
a hot journal is detected and refused, because serving wrong pages
beats failing to open. Same argument, write side.
Non-goals
Creating
sqlite_autoindex_*onCREATE TABLE— filed separately.Both halves are needed for a complete fix (this one fixes adopting a
stock file; that one fixes producing one), but they are independent
changes with different blast radii.
Complexity
Estimate: medium
Reasoning: The oracle rule is already derived and tabulated above,
and the two-phase hook exists. The work is a constraint-list parser over
CREATE TABLEtext (the reader already hascolumn_list_spanandsplit_top_level_commas), the deferred-resolution wiring, theread-only safety valve, and corpus tests against oracle-created files.
Bounded, but it touches the schema reader that every write path depends
on, so it needs real test coverage rather than a spot fix.
Acceptance Criteria
PRAGMA integrity_checkclean andcount(*)correctCREATE UNIQUE INDEXbehaviour unchangedWITHOUT ROWIDtables gain no phantom indexmake lint,cargo fmt --check,make check-mod-filescleanRefs: 010/Req-8, #678