Adds a complete data-quality checking workflow to the extension: define checks once, run them all in one call, keep history — all in plain SQL.
New check primitives (each as anofox_tab_* plus a short alias):
regex_match— share of values matching a pattern vs thresholdsvalues_in_set— allowed-value membership rate, with sample violationsagg_check— avg/min/max/sum/median/stddev of a column vs boundsduplicate_count— duplicates over a column or multi-column keyoccurrence— highest/lowest frequency of any single valuematch_rate— cross-table join match rate (referential-integrity check)compliance— share of rows satisfying a custom SQL predicate; expressions are validated at bind time so multi-statement injection payloads fail the binder
Time-aware checks:
rel_count_change— daily (distinct) count vs a rolling baseline averagemetric_anomaly_iqr— daily metric outside Q1/Q3 ± k·IQR of the trailing window; point it at a persisted results table for anomaly detection on your check historyrolling_values_in_set— set membership over a trailing date window
Check-suite runner:
run_checks(checks_table)reads a SQL table of check definitions and returns one uniform result row per check (run_ts, check_name, check_type, table_name, column_name, identifier, value, lower/upper_threshold, status, message)monitor_onlyseverity (warn instead of fail), per-partition results viaidentifier_column,${today}/${yesterday±N}filter tokens, and per-check error isolation (a missing target becomes anerrorrow instead of failing the run)- Persist history with plain
INSERT INTO dq_results SELECT * FROM run_checks('dq_checks')
Python package anofox-tabular==2026.8.22 (PyPI): wrappers for all new functions plus define_checks/run_checks with optional result-table persistence.
Other changes:
- DuckDB target bumped to v1.5.5 (Python pin aligned)
- Suite-runner params are parsed with DuckDB's bundled yyjson — no dependency on the json extension at runtime
- Windows is excluded on the v1.4.x LTS line (flaky vcpkg MSYS downloads in that toolchain); Windows users are served by the v1.5.x builds
- Validated at scale: a ~2M-row seeded-defect test in CI, plus the 2.96M-row NYC yellow-taxi Jan-2024 dataset where the checks surface the real defects (124 duplicate trips, negative fares, 4.7% NULL passenger_count)