Skip to content

DataQ v1.2.0

Latest
Latest

Choose a tag to compare

View all tags
@github-actions github-actions released this 04 Oct 14:37
· 107 commits to main since this release
v1.2.0
a81dba2
This commit was created on GitHub.com and signed with GitHub’s verified signature.
GPG key ID: B5690EEEBB952194
Verified
Learn about vigilant mode.

The DQ-intelligence and operability cycle (2026-08-22 → 2026-10-04). Full entries are on the changelog page; headlines:

Added

  • Six SQL datasources on one engine-generic base (ADR 0045): PostgreSQL, MySQL / MariaDB,
    Trino, Amazon Athena and Amazon Redshift, plus SQL Server / Azure SQL / Synapse / Fabric SQL
    (ADR 0044). ADLS Gen2 connections can sign in as a service principal, which also covers
    Fabric OneLake.
  • Per-check engines (ADR 0036): Snowflake DMF, including custom DMFs, and Databricks DQX
    run in the user's own workspace, beside Great Expectations.
  • LLM assistance, off by default (ADR 0042): SQL generation, check suggestions and
    root-cause narratives through a provider seam (Anthropic, any OpenAI-compatible endpoint,
    Snowflake Cortex), each validated before anything is saved.
  • Automated coverage (ADR 0047): system-owned suites over the asset inventory, a review
    queue of suggested rules, and coverage and false-positive figures on the Dashboard.
  • Suites as code: YAML suite files, validate, drift and apply, and a generated Python
    client with the dataq CLI, attached to each release, under an API compatibility policy.
  • A pipeline gate (ADR 0046): POST /orchestration/gate lets a pipeline stage start its
    bound suites and wait for the verdict.
  • Operating a workspace: reusable notification channels; the admin area as six pages
    with in-app membership (ADR 0043), guided offboarding and a workspace-health view; a hash
    chain over the audit log.
  • Scores and bulk work: health scores for assets, connections and dimensions; bulk snooze,
    delete and thresholds for checks; a first-run path and empty states that say what to do next.
  • MCP: 46 → 54 tools, including column lineage tracing, notification channels, curated
    docs, coverage figures and suggested rules.
  • A production-like local stack: docker-compose.ghcr.yml serves only the UI, over HTTPS
    from a CA it generates, with TLS on every internal hop and data and a persistent vault in a
    directory. A --profile demo sample warehouse shows real results on first sign-in.
  • Accessibility: every route passes the automated axe checks in both themes.
  • Docs: rebuilt on a task-first layout, versioned per release, with an upgrade guide and
    a support policy.

Changed

  • AWS stack — Slack alerts are off unless slack_webhook_secret_name is set (#2292). The
    stack used to hardcode SLACK_WEBHOOK_SECRET_NAME=channel-slack-webhook without creating that
    secret. An AWS deployment that created <prefix>/channel-slack-webhook itself must set
    slack_webhook_secret_name = "channel-slack-webhook" before its next task-definition replace,
    or Slack alerts stop without a log line.

  • Breaking — connections are tested before they are saved (#1927). POST /connections,
    a PATCH that changes config or a credential, and POST /connections/{id}/reauth refuse a
    config that fails its test with 422 connection_test_failed_on_save and write nothing (a
    re-auth keeps the old credential). An Admin can send skip_test: true to save anyway; the
    audit event records it. PUT /admin/llm tests an enabled provider first
    (422 llm_test_failed_on_save).

  • Breaking — Unity Catalog DATE columns on the frame lane (sampled suites,
    UC_SQL_PUSHDOWN=false, and the frame-only types such as to_be_of_type) are now
    evaluated as dates, as on Parquet and Iceberg. Before, a date bound errored and a set
    of ISO dates failed every row. A to_be_of_type check on a UC DATE column that
    names datetime64[ns] / datetime64[s] must be changed to type_: date. Datetime
    bounds at midnight keep working; a bound with a time of day on a DATE column errors.

  • Breaking — the prebuilt-image stack publishes only the UI, over HTTPS, and keeps its data
    in a directory     (DATAQ_DATA_DIR, default ./dataq-data). Take the compose file and the
    images from the same release.

  • This release adds 33 database migrations. They run before the application starts. Back
    up first; see the upgrade guide for rolling back.

Security

  • The custom-SQL guard lexes each engine's own quoting.
  • Sensitive columns stay masked in incident evidence, alerts and LLM prompts.

Upgrading

Back up first, then follow the upgrade guide. The Python client and the dataq CLI are attached below (pip install dataq_client-1.2.0-py3-none-any.whl).

Full changelog: v1.1.0...v1.2.0

Assets 4
Loading