Skip to content

microsoftexpert/tf-mod-databricks-sql-alert

Folders and files

NameName
Last commit message
Last commit date

Latest commit

 

History

3 Commits
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 

Repository files navigation

🧱 Databricks SQL Alert Terraform Module

Provisions a single Databricks SQL alert (databricks_alert) that evaluates a saved query's result against a threshold condition and notifies on trigger/recovery, against the databricks/databricks provider ~> 1.117.0.

Terraform Provider Module Type Resources Posture

🧩 Overview

  • 🔔 Creates one databricks_alert that evaluates an existing databricks_query's result column against a threshold condition.
  • 🎯 Models the provider's condition block exactly — a required op/operand.column.name pair plus an optional threshold.value oneof (bool_value / double_value / string_value).
  • 🚫 Does not create, own, or modify the query it watches — query_id is a reference, not a managed resource.
  • 🚫 No tags variable — databricks_alert has no tags attribute of any shape in the pinned schema; this module does not invent one.
  • 🚫 Never accepts a credential, host, or account ID.
  • 🌍 Workspace-plane only.

💡 Why it matters: an alert's threshold and notification settings change independently of the query it watches, and one query can back several alerts — keeping alert and query as separate modules lets each evolve on its own schedule instead of forcing a query edit every time a threshold changes.


❤️ Support this project

If these Terraform modules have been helpful to you or your organization, I'd appreciate your support in any of the following ways:

Whether it's a star, a professional connection, or a coffee, every gesture helps keep these modules actively maintained and continually improving. Thank you for being part of the community!


🗺️ Where this fits

flowchart LR
 WAREHOUSE["tf-mod-databricks-sql-warehouse"]
 style WAREHOUSE fill:#F2F2F2,color:#1B3139,stroke:#CCCCCC,stroke-width:1px

 QUERY["tf-mod-databricks-sql-query"]
 style QUERY fill:#1B3139,color:#fff,stroke:#1B3139,stroke-width:1px

 THIS["tf-mod-databricks-sql-alert"]
 style THIS fill:#FF3621,color:#fff,stroke:#1B3139,stroke-width:1px

 PERMISSIONS["tf-mod-databricks-permissions"]
 style PERMISSIONS fill:#F2F2F2,color:#1B3139,stroke:#CCCCCC,stroke-width:1px

 WAREHOUSE -->|"id becomes warehouse_id"| QUERY
 QUERY -->|"id becomes query_id"| THIS
 THIS -->|"id becomes workspace object id"| PERMISSIONS
Loading

tf-mod-databricks-sql-query is this module's direct upstream dependency — its id output feeds this module's query_id input. tf-mod-databricks-sql-warehouse is further upstream (a query needs a warehouse to run against); tf-mod-databricks-permissions is a plausible downstream consumer of this module's id for workspace-object ACLs, once authored. This diagram deliberately omits tf-mod-databricks-notification-destinationdatabricks_alert's schema has no field that references a notification-destination object, so that composition would not reflect anything the code actually does.

ℹ️ None of tf-mod-databricks-sql-warehouse, tf-mod-databricks-sql-query, or tf-mod-databricks-permissions are authored yet in this library as of this module's authoring session — this diagram reflects the planned catalog contract from this library's module catalog, not a verified cross-module terraform plan.

🧬 What this builds

flowchart TB
 subgraph INPUTS["var.*"]
 BASIC["display_name / query_id"]
 MSG["custom_body / custom_subject / notify_on_ok / seconds_to_retrigger"]
 META["owner_user_name / parent_path"]
 COND["condition (op / empty_result_state / operand.column.name / threshold.value)"]
 end

 KEYSTONE["databricks_alert.this"]
 style KEYSTONE fill:#1B3139,color:#fff,stroke:#1B3139,stroke-width:1px

 subgraph INTERNAL["condition block (single, non-dynamic)"]
 OPERAND["operand.column.name (required)"]
 THRESH["threshold.value oneof: bool_value / double_value / string_value"]
 end

 subgraph OUTPUTS["outputs"]
 ID["id / display_name"]
 end

 BASIC --> KEYSTONE
 MSG --> KEYSTONE
 META --> KEYSTONE
 COND --> KEYSTONE
 KEYSTONE --> OPERAND
 KEYSTONE --> THRESH
 KEYSTONE --> ID
Loading

Resource inventory: one resource, databricks_alert.this. No child collection — the condition block is internal structure on the keystone, not a separate resource.

✅ Provider / Versions

Requirement Value
Terraform >= 1.12.0
databricks/databricks ~> 1.117.0
Provider block None — the caller's root module configures provider "databricks" {}
tags / custom_tags Not supported by databricks_alert — none added
timeouts Not supported by databricks_alert — none added

Schema notes that bite:

  • databricks_alert (not databricks_sql_alert) is the correct, current resource this module wraps. databricks_sql_alert is this library's designated legacy/superseded predecessor per house guidance — the compiled terraform providers schema -json for this pinned version carries no machine-readable deprecated: true flag on any resource (verified by scanning the full schema dump), so this distinction is house-standard guidance, not a schema-verified fact. Concretely, databricks_sql_alert's required attributes are name/query_id with an options block, a visibly different, older shape from databricks_alert's display_name/query_id/condition.
  • FYI only — databricks_alert_v2 exists in the pinned schema, with required attributes display_name, evaluation, query_text, schedule, warehouse_id — a self-contained shape that embeds the query text and warehouse directly rather than referencing a separate databricks_query resource. This module deliberately stays on databricks_alert for design continuity with tf-mod-databricks-sql-query's query_id-based model. Not adopted for v1.0.0.
  • No tags attribute exists on databricks_alert at all — confirmed absent from the pinned schema. This is a deliberate contrast with databricks_job and databricks_pipeline (both map(string) tags) and databricks_query (list(string) tags, a genuinely different shape from the other two). Three SQL/BI-adjacent sibling resources model "tags" three different ways, and databricks_alert models it not at all — this module does not invent a tags variable the resource doesn't support.
  • condition is schema-required with exactly one instance (min_items = 1, max_items = 1) — var.condition is required and non-nullable, rendered as a single plain block, not dynamic. Its threshold.value.{bool_value,double_value,string_value} trio is a oneof the provider expects exactly one member of, but this module does not enforce that via a validation {} block — see Architecture Notes.
  • id, create_time, update_time, lifecycle_state, state, and trigger_time are all computed-only; none are settable inputs.

🔑 Required Databricks Permissions & Scopes

  • CAN_RUN (minimum) on the referenced query (query_id) — an alert cannot evaluate a query it is not permitted to execute.
  • Workspace-level SQL object creation entitlement for the applying identity.
  • Unverifiable against schema.json: the compiled provider schema carries no IAM/RBAC metadata — the permission claims above come from Databricks' documented SQL-object permission model, not the machine schema. Confirm current permission-level names against live workspace documentation before treating this as authoritative.

Databricks Prerequisites

  • Workspace-level provider context (not account-level).
  • The referenced query_id must already exist and be readable by the applying identity — this module does not create the query.

📁 Module Structure

tf-mod-databricks-sql-alert/
├── providers.tf # required_providers only — no provider {} block
├── variables.tf # display_name, query_id, condition (all required), messaging/throttle/owner fields
├── main.tf # databricks_alert.this, with a plain condition block + dynamic threshold
├── outputs.tf # id first, then display_name
├── SCOPE.md # cross-module contract
├── README.md # this file
└── examples/
 └── basic/
 └── main.tf # smallest real, runnable call

⚙️ Quick Start

module "row_count_alert" {
  source = "git::https://github.com/microsoftexpert/tf-mod-databricks-sql-alert.git?ref=v1.0.0"

  display_name = "daily-row-count-threshold"
  query_id     = "01ef8a3b-1234-4abc-9def-0123456789ab"

  condition = {
    op = "GREATER_THAN"
    operand = {
      column = {
        name = "row_count"
      }
    }
    threshold = {
      value = {
        double_value = 1000000
      }
    }
  }
}

The caller's root module configures provider "databricks" {} and its auth; this module accepts neither.

🔌 Cross-Module Contract

Consumes:

Input Type Source module
query_id string tf-mod-databricks-sql-query output id

Emits:

Output Description Consumed by
id ID of this alert Auditing / drift-detection tooling; tf-mod-databricks-permissions (once authored)
display_name Display name of the alert Auditing

📚 Example Library

1 · Minimal numeric-threshold alert
module "row_count_alert" {
  source = "git::https://github.com/microsoftexpert/tf-mod-databricks-sql-alert.git?ref=v1.0.0"

  display_name = "daily-row-count-threshold"
  query_id     = "01ef8a3b-1234-4abc-9def-0123456789ab"

  condition = {
    op = "GREATER_THAN"
    operand = {
      column = {
        name = "row_count"
      }
    }
    threshold = {
      value = {
        double_value = 1000000
      }
    }
  }
}
2 · String-comparison alert
module "pipeline_status_alert" {
  source = "git::https://github.com/microsoftexpert/tf-mod-databricks-sql-alert.git?ref=v1.0.0"

  display_name = "pipeline-status-not-healthy"
  query_id     = "01ef8a3b-2222-4abc-9def-0123456789ab"

  condition = {
    op = "NOT_EQUAL"
    operand = {
      column = {
        name = "status"
      }
    }
    threshold = {
      value = {
        string_value = "HEALTHY"
      }
    }
  }
}

💡 Set exactly one of bool_value / double_value / string_value to match the query result column's actual type — see Architecture Notes.

3 · Boolean-condition alert
module "sla_breach_alert" {
  source = "git::https://github.com/microsoftexpert/tf-mod-databricks-sql-alert.git?ref=v1.0.0"

  display_name = "sla-breach-flag"
  query_id     = "01ef8a3b-3333-4abc-9def-0123456789ab"

  condition = {
    op = "EQUAL"
    operand = {
      column = {
        name = "sla_breached"
      }
    }
    threshold = {
      value = {
        bool_value = true
      }
    }
  }
}
4 · Alert with no threshold (existence/empty-result check)
module "no_data_alert" {
  source = "git::https://github.com/microsoftexpert/tf-mod-databricks-sql-alert.git?ref=v1.0.0"

  display_name = "ingestion-produced-no-rows"
  query_id     = "01ef8a3b-4444-4abc-9def-0123456789ab"

  condition = {
    op                 = "GREATER_THAN"
    empty_result_state = "TRIGGERED"
    operand = {
      column = {
        name = "row_count"
      }
    }
    # threshold intentionally omitted — the alert relies on empty_result_state alone.
  }
}

ℹ️ threshold is optional at the block level (max_items = 1, no min_items) — this module renders it as a dynamic block precisely so it can be omitted like this.

5 · notify_on_ok enabled (recovery notifications)
module "recoverable_alert" {
  source = "git::https://github.com/microsoftexpert/tf-mod-databricks-sql-alert.git?ref=v1.0.0"

  display_name = "queue-depth-with-recovery-notice"
  query_id     = "01ef8a3b-5555-4abc-9def-0123456789ab"
  notify_on_ok = true

  condition = {
    op = "GREATER_THAN"
    operand = {
      column = {
        name = "queue_depth"
      }
    }
    threshold = {
      value = {
        double_value = 5000
      }
    }
  }
}

⚠️ Secure/quiet default is notify_on_ok = false. Only enable this for alerts wired into an on-call or ticketing pipeline that wants an explicit "all clear" signal.

6 · seconds_to_retrigger throttling (alert-storm control)
module "throttled_disk_alert" {
  source = "git::https://github.com/microsoftexpert/tf-mod-databricks-sql-alert.git?ref=v1.0.0"

  display_name         = "disk-usage-throttled"
  query_id             = "01ef8a3b-6666-4abc-9def-0123456789ab"
  seconds_to_retrigger = 3600

  condition = {
    op = "GREATER_THAN"
    operand = {
      column = {
        name = "disk_pct_used"
      }
    }
    threshold = {
      value = {
        double_value = 90
      }
    }
  }
}

💡 This module's own default is 900 (15 minutes). Raise it further for a noisy query that re-evaluates on a short schedule.

7 · No-throttle alert (notify on every evaluation)
module "immediate_repeat_alert" {
  source = "git::https://github.com/microsoftexpert/tf-mod-databricks-sql-alert.git?ref=v1.0.0"

  display_name         = "critical-immediate-repeat"
  query_id             = "01ef8a3b-7777-4abc-9def-0123456789ab"
  seconds_to_retrigger = 0

  condition = {
    op = "GREATER_THAN"
    operand = {
      column = {
        name = "error_rate"
      }
    }
    threshold = {
      value = {
        double_value = 0.5
      }
    }
  }
}

⚠️ seconds_to_retrigger = 0 disables this module's default throttle explicitly — expect a notification on every evaluation cycle the condition remains true.

8 · custom_body / custom_subject templated message
module "templated_alert" {
  source = "git::https://github.com/microsoftexpert/tf-mod-databricks-sql-alert.git?ref=v1.0.0"

  display_name   = "revenue-drop-templated"
  query_id       = "01ef8a3b-8888-4abc-9def-0123456789ab"
  custom_subject = "[] Revenue drop detected"
  custom_body    = "Daily revenue fell below threshold. Review the linked query for details."

  condition = {
    op = "LESS_THAN"
    operand = {
      column = {
        name = "daily_revenue"
      }
    }
    threshold = {
      value = {
        double_value = 100000
      }
    }
  }
}
9 · Explicit owner
module "governed_alert" {
  source = "git::https://github.com/microsoftexpert/tf-mod-databricks-sql-alert.git?ref=v1.0.0"

  display_name    = "governed-threshold-alert"
  query_id        = "01ef8a3b-9999-4abc-9def-0123456789ab"
  owner_user_name = "uc-admins"

  condition = {
    op = "GREATER_THAN"
    operand = {
      column = {
        name = "row_count"
      }
    }
    threshold = {
      value = {
        double_value = 500000
      }
    }
  }
}
10 · parent_path-targeted workspace folder
module "team_folder_alert" {
  source = "git::https://github.com/microsoftexpert/tf-mod-databricks-sql-alert.git?ref=v1.0.0"

  display_name = "team-owned-threshold-alert"
  query_id     = "01ef8a3b-aaaa-4abc-9def-0123456789ab"
  parent_path  = "/Workspace/Shared/analytics-team/alerts"

  condition = {
    op = "GREATER_THAN"
    operand = {
      column = {
        name = "row_count"
      }
    }
    threshold = {
      value = {
        double_value = 250000
      }
    }
  }
}

ℹ️ Leaving parent_path at its default (null) places the alert in the creating identity's home folder rather than a shared, open-ACL location — this module's secure default.

11 · Alert on a parameterized query
module "region_threshold_alert" {
  source = "git::https://github.com/microsoftexpert/tf-mod-databricks-sql-alert.git?ref=v1.0.0"

  display_name = "region-scoped-row-count"
  # query_id references a databricks_query configured with a `region` parameter — see
  # tf-mod-databricks-sql-query's parameterized-query example. This module has no awareness of the
  # query's own parameters; it only evaluates the result column named in `condition`.
  query_id = "01ef8a3b-bbbb-4abc-9def-0123456789ab"

  condition = {
    op = "GREATER_THAN"
    operand = {
      column = {
        name = "row_count"
      }
    }
    threshold = {
      value = {
        double_value = 750000
      }
    }
  }
}
12 · for_each-driven multi-alert creation at scale
locals {
  table_row_count_queries = {
    orders    = "01ef8a3b-c001-4abc-9def-0123456789ab"
    customers = "01ef8a3b-c002-4abc-9def-0123456789ab"
    payments  = "01ef8a3b-c003-4abc-9def-0123456789ab"
  }
}

module "table_row_count_alerts" {
  for_each = local.table_row_count_queries
  source   = "git::https://github.com/microsoftexpert/tf-mod-databricks-sql-alert.git?ref=v1.0.0"

  display_name = "${each.key}-row-count-floor"
  query_id     = each.value

  condition = {
    op = "LESS_THAN"
    operand = {
      column = {
        name = "row_count"
      }
    }
    threshold = {
      value = {
        double_value = 1
      }
    }
  }
}

ℹ️ for_each is applied at the caller's root-module level — this module itself has no child collection to iterate; each instance creates exactly one alert.

13 · Minimal least-privilege baseline (recommended starting point)
module "baseline_alert" {
  source = "git::https://github.com/microsoftexpert/tf-mod-databricks-sql-alert.git?ref=v1.0.0"

  display_name = "baseline-threshold-alert"
  query_id     = "01ef8a3b-dddd-4abc-9def-0123456789ab"

  condition = {
    op = "GREATER_THAN"
    operand = {
      column = {
        name = "row_count"
      }
    }
    threshold = {
      value = {
        double_value = 1
      }
    }
  }
  # notify_on_ok left at its secure default: false
  # seconds_to_retrigger left at its secure default: 900
  # parent_path left at its secure default: null (creating identity's home folder)
}
🏗️ 14 · End-to-end composition — warehouse → query → alert
module "reporting_warehouse" {
  source = "git::https://github.com/microsoftexpert/tf-mod-databricks-sql-warehouse.git?ref=v1.0.0"

  name         = "reporting-small"
  cluster_size = "Small"
}

module "daily_row_count_query" {
  source = "git::https://github.com/microsoftexpert/tf-mod-databricks-sql-query.git?ref=v1.0.0"

  display_name = "daily-orders-row-count"
  warehouse_id = module.reporting_warehouse.id
  query_text   = "SELECT COUNT(*) AS row_count FROM analytics.orders WHERE order_date = current_date"
}

module "daily_row_count_alert" {
  source = "git::https://github.com/microsoftexpert/tf-mod-databricks-sql-alert.git?ref=v1.0.0"

  display_name = "daily-orders-row-count-floor"
  query_id     = module.daily_row_count_query.id

  condition = {
    op = "LESS_THAN"
    operand = {
      column = {
        name = "row_count"
      }
    }
    threshold = {
      value = {
        double_value = 1
      }
    }
  }
}

ℹ️ tf-mod-databricks-sql-warehouse and tf-mod-databricks-sql-query are not yet authored in this library as of this module's authoring session — this composition reflects their planned contracts from this library's catalog table (databricks_sql_endpoint's name/cluster_size and databricks_query's display_name/warehouse_id/query_text), not a verified cross-module terraform plan.

📥 Inputs

Variable Type Default Notes
display_name string — (required) Not force-new
query_id string — (required) References an existing databricks_query
condition object({...}) — (required, non-nullable) See full schema below
custom_body string null
custom_subject string null
notify_on_ok bool false Secure/quiet default
seconds_to_retrigger number 900 Secure default (throttle)
owner_user_name string null
parent_path string null Secure default (home folder, not /Shared)
Full variable declarations
variable "display_name" {
  type = string
}

variable "query_id" {
  type = string
}

variable "condition" {
  type = object({
    op                 = string
    empty_result_state = optional(string)
    operand = object({
      column = object({
        name = string
      })
    })
    threshold = optional(object({
      value = object({
        bool_value   = optional(bool)
        double_value = optional(number)
        string_value = optional(string)
      })
    }))
  })
}

variable "custom_body" {
  type    = string
  default = null
}

variable "custom_subject" {
  type    = string
  default = null
}

variable "notify_on_ok" {
  type    = bool
  default = false
}

variable "seconds_to_retrigger" {
  type    = number
  default = 900
  # validation: must be >= 0
}

variable "owner_user_name" {
  type    = string
  default = null
}

variable "parent_path" {
  type    = string
  default = null
}

🧾 Outputs

Output Description Sensitive?
id ID of this alert No
display_name Display name of the alert No

🧠 Architecture Notes

  • condition is required and non-nullable, rendered as a plain block. Since the provider marks condition schema-required with exactly one instance (min_items = 1, max_items = 1), there is no meaningful "absent" state to guard with a dynamic block — var.condition has no default and main.tf renders it as a single, always-present content {}-equivalent plain block.
  • threshold.value's three-way oneof is not enforced by a validation {} block. A caller can set zero, one, two, or all three of bool_value/double_value/string_value and still pass terraform validate — Databricks' API is the actual enforcement point. This mirrors the autoscale/num_workers precedent established elsewhere in this library: the "set exactly one" constraint is documented (here, and in variables.tf's description) rather than encoded, because a hard validation would need to correctly special-case the "no threshold at all" path (example 4 above) without becoming an unreadable compound condition.
  • No tags attribute anywhere in this module, by design. databricks_alert has no tagging concept in the pinned schema — this is a deliberate contrast with databricks_job/ databricks_pipeline (map(string) tags) and databricks_query (list(string) tags), not an oversight.
  • query_id is a reference, never a managed resource. This module cannot detect whether the referenced query still exists with a compatible result-column shape — that only surfaces at apply/plan time against a live workspace, or at alert-evaluation time in the workspace UI.
  • No force-new fields identified in the pinned schema for databricks_alert — unlike tf-mod-databricks-schema's catalog_name/name/storage_root, this resource's top-level attributes (including display_name) are documented as in-place updatable.

🧱 Design Principles

Concern Secure default Opt-out (caller must set explicitly)
Recovery notification noise notify_on_ok = false Explicit true for callers wiring recovery signals into on-call/ticketing
Notification-storm throttling seconds_to_retrigger = 900 (15 min) Explicit 0 for no throttle, or a higher value for a noisier query
Workspace object visibility parent_path = null (creating identity's home folder) Explicit /Shared-style path required for a shared location

The notify_on_ok and seconds_to_retrigger defaults above are this module's own choices, not values the pinned databricks_alert schema documents as its own API-level default — these are candidates for promotion into this library's master Secure-by-default table. parent_path's default follows the existing "Notebook / workspace object visibility" row already established for this library.

🚀 Runbook

cd tf-mod-databricks-sql-alert
terraform init -backend=false
terraform validate
terraform fmt -check

Pin consumers to an immutable tag — ?ref=v1.0.0 — never a branch. This module is plan-only; a human applies from CI after review.

🧪 Testing

terraform validate / terraform fmt -check catch: missing display_name/query_id/condition, the condition.op/condition.operand.column.name non-empty validations, the seconds_to_retrigger >= 0 validation, and malformed HCL. They do not catch: whether the referenced query_id actually exists or is readable by the applying identity, whether condition.operand.column.name matches an actual column in the query's result set, or whether exactly one of threshold.value's three fields is set in a way the Databricks API accepts. Those require an actual plan/apply against a live workspace, out of scope for this authoring process.

💬 Example Output

$ terraform output
display_name = "daily-row-count-threshold"
id = "01ef9c4d-5678-4bcd-8901-234567890abc"

🔍 Troubleshooting

Symptom Cause Fix
Apply fails with a permissions error even though terraform validate passed Applying identity lacks CAN_RUN on the referenced query Confirm the identity holds the required permission on query_id
Alert never triggers, or always evaluates as false/null More than one of bool_value/double_value/string_value set, or none set Set exactly one, matching the query result column's actual type
You find databricks_sql_alert in older documentation or examples It is this library's designated legacy/superseded predecessor of databricks_alert, with a different name/options-based shape Use this module (databricks_alert) instead; do not mix the two resources in the same alert definition
Notifications arrive far more often than expected seconds_to_retrigger set too low, or explicitly 0 Raise it, or accept this module's default of 900
No recovery notification sent when the condition clears notify_on_ok is at its secure default of false Set notify_on_ok = true explicitly
Apply fails because the query doesn't exist query_id is wrong or the query hasn't been created yet Confirm tf-mod-databricks-sql-query (or the query's manual creation) applied successfully first

🔗 Related Docs

💙 "Infrastructure as Code should be standardized, consistent, and secure."

About

No description or website provided.

Topics

Resources

License

Stars

0 stars

Watchers

0 watching

Forks

Packages

 
 
 

Contributors

Languages