Skip to content

v0.51.0 — the declaration, checked against the world

Choose a tag to compare

@flyersworder flyersworder released this 05 Sep 08:05
1c7ba7a

Declared columns are finally reconciled with the live schema (#90)

A semantic source could declare a column that does not exist, and nothing reported it — not at load, not at describe_table, not anywhere. The declaration lives inside contract_digest, so a contract stayed "frozen" around a column the warehouse had renamed while every gate stayed green: the declarations had not changed, the world had. That is the failure a data contract exists to prevent, and the library had both sides in hand — describe_table already held the declarations and the live schema in one function and did not compare them.

check_schema_drift — the CI half

from agentic_data_contracts.validation import check_schema_drift

report = check_schema_drift(contract, adapter)  # or pass an explicit source
if not report.ok:
    print(report.summary())
    sys.exit(1)
Checked 3 tables (16 columns): 2 drifted, 0 unchecked.
  drift: analytics.orders: declared column 'order_date' does not exist
  drift: analytics.orders: declared column 'revenue' does not exist

Four kinds: missing_schema, missing_table, missing_column, and case_mismatch — the last is its own kind because the description overlay is an exact-match lookup, so a column declared MCC where the warehouse says mcc really does lose its authored documentation, but calling it missing sends the reader hunting for a column that is right there.

Four things to know before wiring it into CI:

  • Gate on report.ok, not has_drift. An unresolved * wildcard, an adapter that raised, a semantic source that could not be read — none are drift, and none are a pass. They land in report.unchecked, and ok is False whenever anything is there. tables_checked / columns_checked are reported for the same reason: a run that checked nothing must not read like a run that found nothing.
  • Omitting the source loads the contract's own, exactly as create_tools does. Without that, a contract carrying an inline frozen snapshot — the shape a digest-pinned contract takes — would compare zero columns and report clean.
  • It never resolves a wildcard for you. resolve_tables() rewrites allowed_tables[].tables in place, and those bytes are inside the canonical form contract_digest hashes. A preflight that quietly re-pinned the artifact it was auditing would be worse than the defect.
  • The allow-list is the scope. A dbt manifest carries every model in a project; walking all of them means hundreds of warehouse round-trips to report drift in tables the agent may not query anyway.

Table and schema names are matched case-insensitively (a catalog that upper-cases unquoted identifiers must not make a correct lower-case contract fail); only columns report a case mismatch, because every other path hands a table name to the adapter to resolve.

describe_table — the agent-facing half

{"schema": "main", "table": "orders", "columns": [...],
 "note": "The semantic source declares 2 column(s) this table does not have: revenue, order_date. Its documentation is stale — trust the column list above, not the declaration."}

A note, not an error: the live column list is still correct and the table description still applies, and failing the call would cost the agent a usable answer over a documentation defect.

Held as a property, and proven by mutation

Four review rounds found the same class repeatedly, usually inside the previous round's own fix: the check reporting clean while having compared nothing. Patching instances did not converge, so the guarantee is stated as two conservation laws and generated across how the contract spells a table, how the source spells it, how the catalog answers, where the declarations come from, and how the source and contract are shaped.

Two laws, not one, because the first cannot see the second class of defect. Never report clean while having compared nothing catches a silent pass; it says nothing about a report that invents a problem. A dead case-retry produced an honest-looking unchecked entry for a wholly correct contract and satisfied the first law completely — only a correct contract against a reachable warehouse comes back ok catches it.

Every one of the ten fixes was reverted in turn to confirm the gate goes red. Three of its axes exist only because that exercise scored zero.

Examples

None of the three examples declared a tables: section, so all three reported a clean check over zero columns — the exact defect this release fixes, sitting in the repo's own executable documentation. revenue_agent now declares 14 columns with per-table and per-column descriptions, and ships check_drift.py as the third validation verb alongside verify_examples.py and evaluate_conformance.py, golden-diffed in CI.

The phantom columns it injects are revenue and order_date — the two names that example's hand-written column_hints section already warns about, one of them asserting "Verified against the warehouse on 2026-05-15". A human typed that date; this check makes the claim automatic.

Upgrade notes

  • New CI failure mode, by design. A contract that declares a column the warehouse no longer has now has a way to fail. That is the point, but it is the line to read before wiring check_schema_drift into a pipeline.
  • Nothing existing changes behaviour. The note key is additive on describe_table, and contract_digest is untouched — the DABStep experiment's published digests recompute unchanged.
  • Type mismatches are not covered. A column declared BIGINT that the warehouse reports as VARCHAR is the same class of drift and equally silent. Names first.

Full Changelog: v0.50.0...v0.51.0