Skip to content

v3.7.0

Choose a tag to compare

@github-actions github-actions released this 25 Jun 00:31
· 128 commits to main since this release

Added

  • Runtime Lineage (Scheduled Queries) — a new Runtime Lineage sub-tab that shows observed-runtime data lineage for a selected job. Reads system.query_log (every run is tagged with its job_id) to graph the tables a job actually reads and writes — chaining jobs together when one job's destination table is another job's source — and reveals the columns observed flowing through each table/job when a node is selected (column level). The graph expands on demand — each card shows one level upstream/downstream and a + reveals the next hop in that direction — and uses the same searchable job filter as Runs.
  • Scheduled Queries (DataOps) — a new top-level DataOps page with a Scheduled Queries feature (Overview / Jobs / Runs). Schedule any read-only SELECT on a daily/weekly/monthly preset or a custom UTC cron expression, with deterministic time windows ({{slot_start}}/{{slot_end}}/{{prev_run_at}}), bounded result snapshots, and an optional engine-generated, idempotent materialize write-back (append / replace-partition / upsert) into a destination table. Failure-based alerting notifies the linked notification channels when a run fails (and once on recovery), transition-based to avoid flapping. The Overview is a real summary of the feature (health KPIs, success rate, cadence / output-mode / last-run breakdowns, upcoming runs, top failing jobs); the Jobs list is filterable (enabled/disabled, last-run state, name search) and the Jobs and Runs lists are paginated. Includes an in-process per-job-lease scheduler that is correct under multiple replicas with no leader election, a crash-only reaper + bounded retry, and a transactional notification outbox for at-least-once delivery. Gated by new scheduled_queries:view|edit|delete|run|write RBAC permissions. Opt-out via SCHEDULED_QUERIES_ENABLED=false.

Fixed

  • SSO no longer silently escalates privileges (#270) — when an IdP claim resolved to more than one mapped role, the previous behaviour collapsed to the highest-privilege match, so a user in multiple groups could land in an unexpectedly powerful role. Role sync now fails closed: an ambiguous claim assigns no role and keeps the user's existing one, logging a warning so the misconfiguration can be fixed. Multi-group role mappings remain supported — only genuine overlap (one user resolving to several roles) is rejected.