v0.15.0

What's Changed

• Added LLM-generated AI explanations for row-level anomaly detection (#1129). The has_no_row_anomalies check now attaches a plain-language ai_explanation to each flagged row under _dq_info[].anomaly, describing the likely cause, business impact, suggested action, the top contributing features, and the group's size and average severity. Explanations are generated vis Spark ai_query function against a Databricks Model Serving endpoint — no extra Python dependencies and no driver-side LLM calls — and anomalous rows are grouped by segment and top contributing features so the model is called once per group, keeping cost predictable on large datasets (bounded by max_groups). AI explanations are enabled by default and does not require additional settings. New parameters with good set of defaults include enable_ai_explanation, ai_explanation_llm_model_config, redact_columns (to keep sensitive columns out of the prompt and grouping), and max_groups. If the serving endpoint is unreachable, explanations are left null with a warning and scoring still succeeds. LLMModelConfig also gains max_tokens, temperature, timeout, and max_retries to bound LLM cost and latency and expose tuning parameters for the users if required.
• Added stratified sampling to the profiler (#1240). The profiler now accepts a sample_by option to perform stratified sampling based on column values. Users control the sampling fraction with either a single sample_fraction applied equally across all strata, or a dictionary mapping each stratum to its own fraction. When sample_by is omitted, the profiler continues to use uniform sampling across all rows.
• Added new row-level check function to validate email addresses called is_valid_email (#1158). A new is_valid_email check validates email addresses against a pragmatic, ReDoS-safe subset of RFC 5321/5322. Like the IP-address checks, it ignores null values (no violation reported).
• Added geofencing checks (#1164). Five new row-level geospatial checks validate topological relationships between a column geometry and a reference geometry: is_geo_contains, is_geo_covers, is_geo_intersects, is_geo_touches, and is_geo_within. By default they use exact, meter-level precision built on the ST_* family of functions; is_geo_covers and is_geo_intersects additionally support an approximate mode built on H3_* cell indexing with a configurable resolution for faster checks on large datasets. The reference geometry can be a literal WKT/WKB/EWKT/EWKB value or another column, with optional try_to_geometry conversion of either side. Running these checks requires Databricks serverless compute or runtime 17.1 or above.
• Added support for metrics-only writes (#1236). save_results_in_table and the corresponding workflow path can now persist summary metrics without requiring an output or quarantine table, supporting observability-focused pipelines that only need the metrics table. Batch observations are triggered before metrics are saved so the metrics table is populated correctly, and streaming and no-observer cases now raise explicit errors. Existing configurations with an output or quarantine table are unaffected.
• Allow custom check failure messages (#1092). DQRule now accepts an optional message_expr parameter that lets users define custom failure messages as a Spark Column or a SQL expression string. The same option is supported for checks defined declaratively in metadata (YAML/JSON), specified as a top-level message_expr key on the check definition alongside criticality and check. When omitted, the default message behavior is preserved; when provided, the custom message replaces the default message for failed rows.
• Added a Query Results Cookbook and aligned stored check names and fingerprints (#1193). A new reference page provides "copy-paste" SQL and PySpark recipes for querying DQX result tables (summary metrics, output, quarantine, and checks) to trace errors and warnings across runs, rows, and check definitions. To make the cookbook's fingerprint and name joins reliable, checks saved without an explicit name now store the same autogenerated name and name-inclusive rule_fingerprint that apply_checks writes to _errors/_warnings (named checks and for_each_column rules are byte-identical to before). Requesting summary metrics via metrics_config without a configured observer now fails fast with an InvalidParameterError instead of silently skipping the metrics table.
• Added in-app language switching to DQX Studio (#1172). DQX Studio now ships with four locales (English, Brazilian Portuguese, Italian, and Spanish), selectable from a new Preferences card on the user's Profile page. The choice is persisted per-browser via localStorage with no server-side or table changes, and the change is frontend-only. Non-English translations are AI-assisted and not yet reviewed by native speakers.
• DQX Studio: replaced the apx build framework with first-party build and dev scripts (#1223). The app no longer depends on the apx package. scripts/build_app.py generates the FastAPI OpenAPI schema, runs orval, builds the frontend with Vite, and produces the application wheel (with a build-tagged local-version segment so successive deploys at the same commit always reinstall fresh code). scripts/dev.py runs uvicorn with reload alongside the Vite dev server, forwarding signals and tearing down both processes together. The bundle and warehouse-grant scripts were updated to support both bundle-managed and external (reuse) SQL warehouse modes. There is no runtime behavior change in the app itself.
• DQX Studio: added Lakebase storage backend to improve app latency with declarative storage and destroy protection (#1173). Schemas, the wheels volume, and the Lakebase instance and logical database are now declared in the bundle with prevent_destroy lifecycle protection, and make app-bind adopts pre-existing resources. OLTP tables (rules, settings, RBAC, comments, schedules) move to Postgres via a migration runner, while analytical tables (validation runs, profiling, quarantine, metrics) stay on Delta. Error, warning, and input row counts from the DQX observer are now persisted and surfaced in the UI, label badges and label filtering were added to rule selection and scheduling, and a Spark Connect Observation.get mutability bug that overwrote total row counts was fixed.
• Fixed quarantine-only writes when no output table is configured (#1183). apply_checks_and_save_in_table and apply_checks_by_metadata_and_save_in_table previously raised AttributeError when called with output_config=None and a quarantine_config. output_config is now optional and skipped when unset, so quarantine-only runs write just the invalid records; passing neither configuration raises a clear InvalidParameterError.
• Allow special characters in catalog and schema names (#1232). The validation regex for storage locations now accepts catalog and schema names that contain characters such as hyphens, which were previously rejected.
• Fixed installation when the anomaly-detection workflow is absent (#1194). Installation no longer fails when the Anomaly Trainer workflow is not present; its presence is now checked before it is appended to the workflow.
• Fixed data contract rule generation without the [llm] extra (#1191). DQLLMEngine was imported unconditionally in contract_rules_generator.py purely for a type annotation, causing an ImportError when the [llm] extra was not installed and producing a misleading "install datacontract-cli" error. The import is now guarded behind TYPE_CHECKING, so generate_rules_from_contract(..., process_text_rules=False) works without the [llm] extra.
• Added an installation wizard reference and promoted DQX Studio as the recommended no-code option (#1229).
• Added a data drift detection guide to the profiling documentation (#1205).
• Renamed user to client_id in the LakebaseChecksStorageConfig documentation to match the actual configuration field (#1201).

BREAKING CHANGES!

• Row-level anomaly detection rule now computes SHAP feature contributions by default — enable_contributions defaults to True (was False), adding scoring cost (requires the shap library, already included in the [anomaly] extra). Set enable_contributions=False to restore the previous behaviour. (#1129)
• Row-level anomaly detection now generates AI explanations by default — enable_ai_explanation defaults to True, so existing anomaly checks will make LLM calls against a Databricks Model Serving endpoint (default databricks-claude-sonnet-4-5) and incur cost. This requires Foundation Model APIs to be available in the workspace; if the endpoint is unreachable, explanations are skipped (null) with a warning rather than failing. Set enable_ai_explanation=False to opt out entirely. (#1129)
• The _dq_info[].anomaly output now contains an additional ai_explanation struct. Downstream consumers that assert on the exact anomaly struct schema should account for the new field. (#1129)
• Requesting summary metrics via metrics_config without a configured observer now raises InvalidParameterError instead of silently skipping the metrics table. (#1193)

Full Changelog: v0.14.0...v0.15.0

Contributors:
@mwojtyczka, @fedeflowers, @ghanse, @mvanhorn, @IvannKurchenko, @lfbraz, @aarushisingh04, @berrybluecode, @GewoonMaarten, @SAY-5, @cornzyblack, @acreese11, @Vsatyam013, @ruslan-basyrov,