🚀 Azure Data Explorer (Kusto) Terraform Module

The single deeply-typed boundary for an Azure Data Explorer estate — the cluster, its databases, follower attachments, RBAC, managed private endpoints, KQL setup scripts, and every ingestion data connection (Cosmos DB · Event Grid · Event Hub · IoT Hub). Built for AzureRM v4.x.

🧩 Overview

This composite module provisions an Azure Data Explorer (Kusto) cluster and every tightly-coupled child resource that has no independent lifecycle, behind one module boundary:

🟣 One cluster (this) — SKU, managed identity, optimized auto-scale, language extensions, network posture.
🗄️ Databases — for_each map, with hot-cache and soft-delete retention.
🔗 Attached (follower) database configurations — cross-cluster data sharing.
🔐 Customer-managed key (CMK) — optional Key Vault / Managed HSM encryption (platform-managed by default).
👤 Cluster- and database-level principal assignments — Entra ID RBAC.
🛡️ Managed private endpoints — outbound private-link to Storage / Event Hub / IoT Hub / SQL / ADX.
📜 KQL scripts — schema/table/mapping bootstrap (inline or blob-backed; marked sensitive).
📥 Data connections — Cosmos DB, Event Grid, Event Hub, and IoT Hub ingestion.
🔎 Read-only lookups — data.azurerm_kusto_cluster / data.azurerm_kusto_database for resolving leader/sibling IDs.

💡 Why it matters: one terraform apply stands up a query-ready Data Explorer estate with secure-by-default networking and encryption — and because every collection is a for_each map, adding a database or data connection never re-plans the cluster.

❤️ Support this project

If these Terraform modules have been helpful to you or your organization, I'd appreciate your support in any of the following ways:

⭐ Star this repository to help others discover this Terraform module.
🤝 Connect with me on LinkedIn: linkedin.com/in/microsoftexpert
☕ Buy me a coffee: buymeacoffee.com/microsoftexpert

Whether it's a star, a professional connection, or a coffee, every gesture helps keep these modules actively maintained and continually improving. Thank you for being part of the community!

🗺️ Where this fits in the family

flowchart LR
 RG["tf-mod-azure-resource-group"]
 MI["tf-mod-azure-managed-identity"]
 KV["tf-mod-azure-key-vault"]
 ADX["tf-mod-azure-data-explorer<br/>(this module)"]
 PE["tf-mod-azure-private-endpoint"]
 RA["tf-mod-azure-role-assignment"]
 DIAG["tf-mod-azure-monitor-diagnostic-setting"]
 SRC["tf-mod-azure-event-hub · tf-mod-azure-iot-hub<br/>tf-mod-azure-storage-account · tf-mod-azure-cosmosdb"]

 RG -->|"resource_group_name"| ADX
 MI -->|"identity_ids · user_identity"| ADX
 KV -->|"key_vault_id · key_name"| ADX
 SRC -->|"eventhub_id · iothub_id · storage_account_id · cosmosdb_container_id"| ADX
 ADX -->|"id"| PE
 ADX -->|"identity_principal_id"| RA
 ADX -->|"id"| DIAG

 style ADX fill:#8957E5,color:#fff
 style RG fill:#0078D4,color:#fff

This module consumes resource_group_name, managed-identity IDs, Key Vault key references, and ingestion-source resource IDs from its siblings, and emits id, identity_principal_id, and per-collection ID maps for tf-mod-azure-private-endpoint, tf-mod-azure-role-assignment, and diagnostic settings — see the Cross-Module Contract.

🧬 What this module builds

flowchart TD
 CLUSTER["azurerm_kusto_cluster<br/>(this)"]

 CMK["customer_managed_key<br/>(0-or-1)"]
 DB["database<br/>(for_each)"]
 ADC["attached_database_configuration<br/>(for_each)"]
 CPA["cluster_principal_assignment<br/>(for_each)"]
 MPE["managed_private_endpoint<br/>(for_each)"]

 DPA["database_principal_assignment<br/>(for_each)"]
 SCRIPT["script<br/>(for_each · sensitive)"]
 COSMOS["cosmosdb_data_connection<br/>(for_each)"]
 EG["eventgrid_data_connection<br/>(for_each)"]
 EH["eventhub_data_connection<br/>(for_each)"]
 IOT["iothub_data_connection<br/>(for_each)"]

 CLUSTER --> CMK
 CLUSTER --> DB
 CLUSTER --> ADC
 CLUSTER --> CPA
 CLUSTER --> MPE
 DB --> DPA
 DB --> SCRIPT
 DB --> COSMOS
 DB --> EG
 DB --> EH
 DB --> IOT

 style CLUSTER fill:#8957E5,color:#fff

Resource inventory (12 managed resource types + 2 data sources):

🟣 azurerm_kusto_cluster.this — primary: the Data Explorer cluster.
🔐 azurerm_kusto_cluster_customer_managed_key.customer_managed_key — optional CMK encryption (0-or-1).
🗄️ azurerm_kusto_database.this — databases.
🔗 azurerm_kusto_attached_database_configuration.attached_database_configuration — follower/data-sharing attachments.
👤 azurerm_kusto_cluster_principal_assignment.cluster_principal_assignment — cluster-scope RBAC.
👤 azurerm_kusto_database_principal_assignment.database_principal_assignment — database-scope RBAC.
🛡️ azurerm_kusto_cluster_managed_private_endpoint.managed_private_endpoint — outbound managed private endpoints.
📜 azurerm_kusto_script.script — KQL setup scripts.
📥 azurerm_kusto_cosmosdb_data_connection.cosmosdb_data_connection — Cosmos DB ingestion.
📥 azurerm_kusto_eventgrid_data_connection.eventgrid_data_connection — Event Grid (blob event) ingestion.
📥 azurerm_kusto_eventhub_data_connection.eventhub_data_connection — Event Hub ingestion.
📥 azurerm_kusto_iothub_data_connection.iothub_data_connection — IoT Hub ingestion.
🔎 data.azurerm_kusto_cluster.lookup / data.azurerm_kusto_database.lookup — read-only lookups.

✅ Provider / Versions

Terraform >= 1.12.0
AzureRM >= 4.0, < 5.0
The module declares the provider requirement only — it configures no provider {} block, so it composes cleanly under any root.

ℹ️ Schema notes that bite (azurerm 4.x):

The cluster sku is a required block; capacity is required unless optimized_auto_scale is set.

azurerm_kusto_cluster_principal_assignment and azurerm_kusto_database_principal_assignment expose no update timeout — every field forces replacement.

The provider exposes both a language_extension (current) and a deprecated language_extensions block; this module renders only the current language_extension.

azurerm_kusto_cosmosdb_data_connection and azurerm_kusto_iothub_data_connection are fully immutable — any change forces a new connection.

📁 Module Structure

tf-mod-azure-data-explorer/
├── providers.tf # Terraform & azurerm version pins (no provider block)
├── variables.tf # Deeply-typed inputs, validation, secure defaults
├── main.tf # Total renderer: cluster + for_each child collections
├── outputs.tf # id, identity_principal_id, per-collection ID maps
├── SCOPE.md # Cross-module contract (consumes / emits / gotchas)
└── README.md # This file

⚙️ Quick Start

module "data_explorer" {
  source = "git::https://github.com/microsoftexpert/tf-mod-azure-data-explorer?ref=v1.0.0"

  name                = "adxcaseydev01"
  resource_group_name = module.resource_group.name
  location            = "eastus2"

  sku = {
    name     = "Dev(No SLA)_Standard_D11_v2"
    capacity = 1
  }

  databases = {
    telemetry = {
      name             = "telemetry"
      hot_cache_period = "P7D"
    }
  }

  tags = { environment = "dev", workload = "analytics" }
}

🔒 With no networking inputs supplied, the cluster is created with public_network_access_enabled = false and disk_encryption_enabled = true — the secure default.

🔌 Cross-Module Contract

Consumes

Input	Type	Source
`resource_group_name`	`string`	`tf-mod-azure-resource-group.name`
`location`	`string`	`tf-mod-azure-resource-group.location` (or root)
`identity.identity_ids`	`list(string)`	`tf-mod-azure-managed-identity.id`
`customer_managed_key.key_vault_id` / `key_name`	`string`	`tf-mod-azure-key-vault.id` / Key Vault key name
`customer_managed_key.user_identity`	`string`	`tf-mod-azure-managed-identity.id`
`managed_private_endpoints[].private_link_resource_id`	`string`	`tf-mod-azure-storage-account` / `tf-mod-azure-event-hub` / `tf-mod-azure-iot-hub` `.id`
`eventhub_data_connections[].eventhub_id`	`string`	`tf-mod-azure-event-hub.id`
`eventgrid_data_connections[].storage_account_id` / `eventhub_id`	`string`	`tf-mod-azure-storage-account.id` / `tf-mod-azure-event-hub.id`
`iothub_data_connections[].iothub_id`	`string`	`tf-mod-azure-iot-hub.id`
`cosmosdb_data_connections[].cosmosdb_container_id` / `managed_identity_id`	`string`	`tf-mod-azure-cosmosdb.container_id` / `tf-mod-azure-managed-identity.id`

Emits

Output	Description	Consumed by
`id`	Kusto Cluster resource ID	`tf-mod-azure-private-endpoint` (resource_id), `tf-mod-azure-role-assignment` (scope), `tf-mod-azure-monitor-diagnostic-setting`
`name`	Cluster name	Any module referencing the cluster by name
`resource_group_name`	Resource group (passthrough)	Sibling modules in the same RG
`location`	Cluster region	Sibling modules that colocate resources with the cluster
`uri` / `data_ingestion_uri`	Query / ingestion FQDNs	KQL clients, ingestion pipelines
`identity_principal_id`	System-identity object ID (`null` if none)	`tf-mod-azure-role-assignment` (principal_id), Key Vault access policies
`identity_tenant_id`	System-identity tenant ID (`null` if none)	Cross-tenant trust configuration
`database_ids` / `database_names`	Map of database key → ID / name	Data-connection wiring, downstream RBAC
`_data_connection_ids`, `_principal_assignment_ids`, `managed_private_endpoint_ids`, `script_ids`, `attached_database_configuration_ids`	Per-collection ID maps	Monitoring, dependency wiring
`cluster_lookup_ids` / `cluster_lookup_uris` / `database_lookup_ids`	Resolved IDs/URIs of looked-up existing resources	Follower attachments, cross-cluster queries

📚 Example Library

1️⃣ Minimal — smallest call that produces a real cluster

module "data_explorer" {
  source = "git::https://github.com/microsoftexpert/tf-mod-azure-data-explorer?ref=v1.0.0"

  name                = "adxcaseydev01"
  resource_group_name = "rg-analytics-dev"
  location            = "eastus2"

  sku = {
    name     = "Dev(No SLA)_Standard_D11_v2"
    capacity = 1
  }
}

2️⃣ Production SKU with optimized auto-scale

module "data_explorer" {
  source = "git::https://github.com/microsoftexpert/tf-mod-azure-data-explorer?ref=v1.0.0"

  name                = "adxcaseyprod01"
  resource_group_name = "rg-analytics-prod"
  location            = "eastus2"

  sku = {
    name = "Standard_E8ads_v5" # capacity omitted — set by auto-scale
  }

  optimized_auto_scale = {
    minimum_instances = 2
    maximum_instances = 10
  }

  zones = ["1", "2", "3"] # availability zones (immutable; not on Dev SKUs)
}

⚠️ zones is immutable — changing it forces a new cluster. Set it at creation time.

3️⃣ Databases with retention

databases = {
  telemetry = {
    name               = "telemetry"
    hot_cache_period   = "P31D" # 31 days hot cache
    soft_delete_period = "P365D"
  }
  audit = {
    name               = "audit"
    hot_cache_period   = "P7D"
    soft_delete_period = "P2555D" # ~7 years
  }
}

4️⃣ System-assigned managed identity

identity = {
  type = "SystemAssigned"
}

The system identity's object ID is exported as identity_principal_id for downstream role assignments (e.g. granting the cluster read access to a Cosmos DB account).

5️⃣ User-assigned managed identity

identity = {
  type         = "SystemAssigned, UserAssigned"
  identity_ids = [module.managed_identity.id]
}

💡 identity_ids is required whenever type includes UserAssigned — the module enforces this with a validation block.

6️⃣ Cluster & database RBAC (principal assignments)

cluster_principal_assignments = {
  platform_admin = {
    name           = "platform-admin"
    principal_id   = "00000000-0000-0000-0000-000000000000"
    principal_type = "Group"
    role           = "AllDatabasesViewer"
    tenant_id      = "11111111-1111-1111-1111-111111111111"
  }
}

database_principal_assignments = {
  telemetry_ingestor = {
    name           = "telemetry-ingestor"
    database_name  = "telemetry" # matches databases["telemetry"].name
    principal_id   = "22222222-2222-2222-2222-222222222222"
    principal_type = "App"
    role           = "Ingestor"
    tenant_id      = "11111111-1111-1111-1111-111111111111"
  }
}

ℹ️ Cluster roles: AllDatabasesAdmin · AllDatabasesViewer · AllDatabasesMonitor. Database roles: Admin · Ingestor · Monitor · UnrestrictedViewer · User · Viewer.

7️⃣ KQL setup scripts (inline & blob-backed)

scripts = {
  create_table = {
    name           = "create-telemetry-table"
    database_key   = "telemetry" # resolves to databases["telemetry"].id
    script_content = <<-KQL
.create table Events (Timestamp:datetime, DeviceId:string, Payload:dynamic)
 KQL
  }
  blob_script = {
    name        = "bootstrap-from-blob"
    database_id = "/subscriptions/.../databases/telemetry"
    url         = "https://sa.blob.core.windows.net/scripts/bootstrap.kql"
    sas_token   = var.script_sas_token
  }
}

🔒 The scripts variable is marked sensitive = true (it can carry script_content and sas_token). The module iterates it with for_each = toset(nonsensitive(keys(var.scripts))) so keys can drive resources without leaking secret values. 💡 Provide exactly one of database_key (a key from databases) or database_id (an explicit ID).

8️⃣ Event Hub data connection

eventhub_data_connections = {
  telemetry_stream = {
    name              = "telemetry-stream"
    database_name     = "telemetry"
    eventhub_id       = module.event_hub.id
    consumer_group    = "adx"
    data_format       = "JSON"
    table_name        = "Events"
    mapping_rule_name = "EventsMapping"
    compression       = "GZip"
    identity_id       = module.managed_identity.id # trusted-service auth (recommended)
  }
}

9️⃣ Event Grid & IoT Hub data connections

eventgrid_data_connections = {
  blob_drops = {
    name                         = "blob-drops"
    database_name                = "telemetry"
    storage_account_id           = module.storage_account.id
    eventhub_id                  = module.event_hub.id
    eventhub_consumer_group_name = "adx-eg"
    blob_storage_event_type      = "Microsoft.Storage.BlobCreated"
    data_format                  = "JSON"
  }
}

iothub_data_connections = {
  devices = {
    name                      = "device-telemetry"
    database_name             = "telemetry"
    iothub_id                 = module.iot_hub.id
    consumer_group            = "adx"
    shared_access_policy_name = "adx-read"
    data_format               = "JSON"
  }
}

🔟 Cosmos DB data connection

identity = { type = "SystemAssigned" }

cosmosdb_data_connections = {
  orders = {
    name                  = "orders-cdc"
    database_key          = "telemetry"
    cosmosdb_container_id = module.cosmosdb.container_id
    managed_identity_id   = module.data_explorer.id # cluster ID for system identity
    table_name            = "Orders"
    mapping_rule_name     = "OrdersMapping"
  }
}

⚠️ Cosmos DB connections are fully immutable — every field forces a new connection. The cluster's identity needs the Cosmos DB Built-in Data Reader role on the account.

1️⃣1️⃣ Attached (follower) database configuration — cross-cluster sharing

# Look up the leader cluster by name…
cluster_lookups = {
  leader = { name = "adxleaderprod", resource_group_name = "rg-leader" }
}

# …then follow one of its databases.
attached_database_configurations = {
  shared = {
    name          = "follow-sales"
    cluster_id    = module.data_explorer.cluster_lookup_ids["leader"]
    database_name = "sales"
    sharing = {
      tables_to_include    = ["FactSales"]
      functions_to_exclude = ["InternalOnly"]
    }
  }
}

⚠️ A leader cluster supports at most 100 follower clusters. If streaming ingestion is enabled on the leader, it must also be enabled on this follower to receive streamed data.

1️⃣2️⃣ for_each at scale — many databases from a map(object)

locals {
  tenants = ["acme", "globex", "initech", "umbrella"]
}

module "data_explorer" {
  source = "git::https://github.com/microsoftexpert/tf-mod-azure-data-explorer?ref=v1.0.0"

  name                = "adxcaseyprod01"
  resource_group_name = "rg-analytics-prod"
  location            = "eastus2"
  sku                 = { name = "Standard_E8ads_v5", capacity = 2 }

  databases = {
    for t in local.tenants : t => {
      name             = "db-${t}"
      hot_cache_period = "P14D"
    }
  }
}

💡 The map key (acme, globex, …) is the resource's stable identity. Choose keys that never change — renaming a key destroys and recreates that database. See Architecture Notes.

1️⃣3️⃣ Private endpoint wiring — feed `id` into tf-mod-azure-private-endpoint

module "data_explorer" {
  source              = "git::https://github.com/microsoftexpert/tf-mod-azure-data-explorer?ref=v1.0.0"
  name                = "adxcaseyprod01"
  resource_group_name = "rg-analytics-prod"
  location            = "eastus2"
  sku                 = { name = "Standard_E8ads_v5", capacity = 2 }

  public_network_access_enabled = false # inbound only via private endpoint
}

module "adx_private_endpoint" {
  source                         = "git::https://github.com/microsoftexpert/tf-mod-azure-private-endpoint?ref=v1.0.0"
  name                           = "pe-adxcaseyprod01"
  resource_group_name            = "rg-analytics-prod"
  location                       = "eastus2"
  subnet_id                      = module.subnet.id
  private_connection_resource_id = module.data_explorer.id # ← the keystone wire
  subresource_names              = ["cluster"]
}

⚠️ Private endpoints are not supported on VNet-injected clusters — do not combine virtual_network_configuration with an inbound private endpoint.

1️⃣4️⃣ CMK encryption with Key Vault

identity = { type = "SystemAssigned" } # required for CMK

customer_managed_key = {
  key_vault_id = module.key_vault.id
  key_name     = "adx-cmk"
  # key_version omitted → Azure auto-rotates to the latest version
}

🔒 null (the default) = platform-managed key. Exactly one of key_vault_id or managed_hsm_key_id must be set. Switching to/from CMK or rotating the key reference is an in-place update — it does not recreate the cluster. The cluster identity needs Get / WrapKey / UnwrapKey on the key.

1️⃣5️⃣ Security / hardening — locked-down, -compliant variant

module "data_explorer" {
  source              = "git::https://github.com/microsoftexpert/tf-mod-azure-data-explorer?ref=v1.0.0"
  name                = "adxcaseyprod01"
  resource_group_name = "rg-analytics-prod"
  location            = "eastus2"
  sku                 = { name = "Standard_E8ads_v5", capacity = 3 }
  zones               = ["1", "2", "3"]

  # Network — inbound private only, egress locked down
  public_network_access_enabled      = false
  outbound_network_access_restricted = true
  allowed_fqdns                      = ["mystorage.blob.core.windows.net"]
  trusted_external_tenants           = [] # this tenant only

  # Encryption
  disk_encryption_enabled   = true
  double_encryption_enabled = true # immutable — set at creation
  identity                  = { type = "SystemAssigned" }
  customer_managed_key = {
    key_vault_id = module.key_vault.id
    key_name     = "adx-cmk"
  }

  # Egress via managed private endpoint (no public storage exposure)
  managed_private_endpoints = {
    ingest_storage = {
      name                     = "mpe-ingest-sa"
      group_id                 = "blob"
      private_link_resource_id = module.storage_account.id
    }
  }
}

🔒 double_encryption_enabled is immutable — it can only be set at create time.

🏗️ End-to-end composition (mandatory) — full suite wired via outputs → inputs

module "resource_group" {
  source   = "git::https://github.com/microsoftexpert/tf-mod-azure-resource-group?ref=v1.0.0"
  name     = "rg-analytics-prod"
  location = "eastus2"
}

module "managed_identity" {
  source              = "git::https://github.com/microsoftexpert/tf-mod-azure-managed-identity?ref=v1.0.0"
  name                = "id-adx-prod"
  resource_group_name = module.resource_group.name
  location            = module.resource_group.location
}

module "key_vault" {
  source              = "git::https://github.com/microsoftexpert/tf-mod-azure-key-vault?ref=v1.0.0"
  name                = "kv-adx-prod"
  resource_group_name = module.resource_group.name
  location            = module.resource_group.location
}

module "event_hub" {
  source              = "git::https://github.com/microsoftexpert/tf-mod-azure-event-hub?ref=v1.0.0"
  name                = "evh-telemetry"
  resource_group_name = module.resource_group.name
  location            = module.resource_group.location
}

module "data_explorer" {
  source              = "git::https://github.com/microsoftexpert/tf-mod-azure-data-explorer?ref=v1.0.0"
  name                = "adxcaseyprod01"
  resource_group_name = module.resource_group.name
  location            = module.resource_group.location

  sku   = { name = "Standard_E8ads_v5", capacity = 3 }
  zones = ["1", "2", "3"]

  identity = {
    type         = "SystemAssigned, UserAssigned"
    identity_ids = [module.managed_identity.id]
  }

  public_network_access_enabled = false
  disk_encryption_enabled       = true

  customer_managed_key = {
    key_vault_id  = module.key_vault.id
    key_name      = "adx-cmk"
    user_identity = module.managed_identity.id
  }

  databases = {
    telemetry = { name = "telemetry", hot_cache_period = "P31D" }
  }

  eventhub_data_connections = {
    telemetry_stream = {
      name           = "telemetry-stream"
      database_name  = "telemetry"
      eventhub_id    = module.event_hub.id
      consumer_group = "adx"
      data_format    = "JSON"
      table_name     = "Events"
      identity_id    = module.managed_identity.id
    }
  }
}

module "adx_private_endpoint" {
  source                         = "git::https://github.com/microsoftexpert/tf-mod-azure-private-endpoint?ref=v1.0.0"
  name                           = "pe-adxcaseyprod01"
  resource_group_name            = module.resource_group.name
  location                       = module.resource_group.location
  subnet_id                      = module.subnet.id
  private_connection_resource_id = module.data_explorer.id
  subresource_names              = ["cluster"]
}

module "adx_reader_role" {
  source               = "git::https://github.com/microsoftexpert/tf-mod-azure-role-assignment?ref=v1.0.0"
  scope                = module.data_explorer.id
  principal_id         = module.data_explorer.identity_principal_id
  role_definition_name = "Reader"
}

📥 Inputs

Core (required): name, resource_group_name, location, sku.

Cluster config (optional): identity, public_network_access_enabled, disk_encryption_enabled, double_encryption_enabled, streaming_ingestion_enabled, purge_enabled, auto_stop_enabled, outbound_network_access_restricted, public_ip_type, allowed_fqdns, allowed_ip_ranges, trusted_external_tenants, zones, optimized_auto_scale, virtual_network_configuration, language_extensions, customer_managed_key.

Child collections (maps, default {}): databases, attached_database_configurations, cluster_principal_assignments, database_principal_assignments, managed_private_endpoints, scripts (sensitive), cosmosdb_data_connections, eventgrid_data_connections, eventhub_data_connections, iothub_data_connections.

Read-only lookups: cluster_lookups, database_lookups.

Universal tail: tags, timeouts.

Full nested object schemas

sku = object({
  name     = string           # closed set — see validation in variables.tf
  capacity = optional(number) # required unless optimized_auto_scale is set
})

identity = object({
  type         = string                 # SystemAssigned | UserAssigned | "SystemAssigned, UserAssigned"
  identity_ids = optional(list(string)) # required when type includes UserAssigned
})

optimized_auto_scale = object({
  minimum_instances = number # 0-1000
  maximum_instances = number # 0-1000, >= minimum
})

customer_managed_key = object({         # null = platform-managed (default)
  key_vault_id       = optional(string) # XOR managed_hsm_key_id
  key_name           = optional(string)
  key_version        = optional(string) # omit for auto-rotation
  managed_hsm_key_id = optional(string)
  user_identity      = optional(string)
})

databases = map(object({
  name                = string
  location            = optional(string) # defaults to cluster location
  resource_group_name = optional(string) # defaults to cluster RG
  hot_cache_period    = optional(string) # ISO 8601, e.g. "P7D"
  soft_delete_period  = optional(string)
}))

scripts = map(object({ # SENSITIVE
  name                               = string
  database_id                        = optional(string) # XOR database_key
  database_key                       = optional(string)
  script_content                     = optional(string) # XOR url/sas_token
  sas_token                          = optional(string)
  url                                = optional(string)
  continue_on_errors_enabled         = optional(bool, false)
  force_an_update_when_value_changed = optional(string)
  script_level                       = optional(string, "Database") # Database | Cluster
  principal_permissions_action       = optional(string)
}))

eventhub_data_connections = map(object({
  name                    = string
  database_name           = string
  eventhub_id             = string
  consumer_group          = string
  location                = optional(string)
  resource_group_name     = optional(string)
  compression             = optional(string, "None")   # GZip | None
  data_format             = optional(string)           # APACHEAVRO | AVRO | CSV | JSON | …
  database_routing_type   = optional(string, "Single") # Single | Multi
  event_system_properties = optional(list(string))
  identity_id             = optional(string)
  mapping_rule_name       = optional(string)
  table_name              = optional(string)
  retrieval_start_date    = optional(string)
}))

ℹ️ See variables.tf for every collection's complete schema, defaults, and validation rules.

🧾 Outputs

Output	Description
`id`	Kusto Cluster resource ID (feeds private endpoints, role assignments, diagnostics)
`name`	Cluster name
`resource_group_name`	Resource group (passthrough)
`location`	Cluster region
`uri`	Cluster query URI (FQDN)
`data_ingestion_uri`	Cluster data-ingestion URI
`identity_principal_id`	System-identity object ID — `null` when no system identity
`identity_tenant_id`	System-identity tenant ID — `null` when no system identity
`database_ids` / `database_names`	Map of database key → ID / name
`attached_database_configuration_ids`	Map of key → attached config ID
`cluster_principal_assignment_ids`	Map of key → cluster RBAC assignment ID
`database_principal_assignment_ids`	Map of key → database RBAC assignment ID
`managed_private_endpoint_ids`	Map of key → managed private endpoint ID
`script_ids`	Map of key → script ID
`cosmosdb_data_connection_ids`	Map of key → Cosmos DB connection ID
`eventgrid_data_connection_ids`	Map of key → Event Grid connection ID
`eventhub_data_connection_ids`	Map of key → Event Hub connection ID
`iothub_data_connection_ids`	Map of key → IoT Hub connection ID
`cluster_lookup_ids` / `cluster_lookup_uris`	Map of lookup key → resolved cluster ID / URI
`database_lookup_ids`	Map of lookup key → resolved database ID

ℹ️ No output is marked sensitive — IDs and URIs are non-secret. Secret-bearing inputs (scripts) are sensitive; the module never emits script content.

🧠 Architecture Notes

Cluster → database → table hierarchy. Per Microsoft's logical architecture, the cluster is the top-level compute/storage container; databases live inside it, tables inside databases. This module owns the cluster and database layers; tables/mappings are bootstrapped via scripts.
Private endpoint integration. Set public_network_access_enabled = false and wire module.data_explorer.id into tf-mod-azure-private-endpoint (subresource_names = ["cluster"]). The cluster firewall governs only the public endpoint; private endpoints use a consent flow. ⚠️ Private endpoints are not supported on VNet-injected clusters — virtual_network_configuration and inbound private endpoints are mutually exclusive. For outbound private connectivity to ingestion sources, use managed_private_endpoints (Microsoft recommends managed-identity + trusted-service access over MPE for Storage/Event Hubs, except the Event Grid flow).
for_each map(object) lifecycle. Every child collection is keyed by a caller-supplied string. The key is the resource's identity — renaming a key (e.g. telemetry → telemetry_v2) makes Terraform plan a destroy + recreate of that one element, while the rest of the map is untouched. Choose keys that outlive the values they describe. Because collections are independent maps, you can safely target a single element during incident response: terraform apply -target='module.data_explorer.azurerm_kusto_database.this["telemetry"]'. Partial applies never force a cluster re-plan.
Secrets in for_each keys. scripts is sensitive = true (it can carry script_content/sas_token). Terraform forbids a sensitive value in for_each, so the module uses for_each = toset(nonsensitive(keys(var.scripts))) — only the keys are de-sensitised for iteration; the secret values stay sensitive and are read back per-key.
SKU gating. Dev(No SLA)_* SKUs are single-node and do not support availability zones or optimized_auto_scale (and have no SLA). Production engine SKUs (*_v4/*_v5, EC*-series) are required for those, and for the largest node counts. Scaling SKU/capacity is an in-place update; changing zones or double_encryption_enabled is immutable (recreate). The module validates the SKU name against the full provider-supported set.
CMK vs platform-managed key. customer_managed_key = null (default) uses Microsoft-managed encryption. Supplying the object switches to CMK — in place, no recreate — and rotating the key reference is also in place. Omit key_version to let Azure auto-rotate. CMK requires a cluster managed identity with Get/WrapKey/UnwrapKey on the key; exactly one of key_vault_id or managed_hsm_key_id is allowed (enforced by validation).
Capacity limits to design around (per-subscription / per-cluster): 20 clusters per region per subscription, 1,000 instances per cluster, 10,000 databases per cluster, 100 follower clusters per leader, and a control-plane write budget of ~1,000 ops/hour — large for_each maps applied in one run can approach the write budget.

🧱 Design Principles

🔒 Secure by default — public_network_access_enabled = false, disk_encryption_enabled = true, purge_enabled = false, own-tenant trust.
🧱 Type is the contract — deeply-typed object schemas, optional with baked-in defaults, validation {} on every enum and closed set; malformed input fails at plan time.
🧩 One primary this, children by role — composite boundary with for_each maps, never count.
🔁 main.tf is a total renderer — dynamic blocks for every optional/repeating block, try(x, null) on every optional nested field.
🤝 Consumes-not-creates — ingestion sources, identities, and Key Vault keys arrive by ID; the module never creates them.
🙈 Secrets stay sensitive — secret-bearing inputs are sensitive = true; no secret is emitted as an output.

🚀 Runbook

terraform init -backend=false
terraform validate
terraform fmt -check
terraform plan -out tfplan
terraform apply tfplan
terraform output

⚠️ Always pin the module by tag — ?ref=v1.0.0, never a branch. Tags are immutable; a branch can change under you and silently alter a plan.

🧪 Testing

The offline proof gate (no Azure credentials required):

terraform fmt -check # zero formatting drift
terraform validate # schema-valid configuration
tflint # lint for provider-specific issues

This module passes terraform validate and terraform fmt -check clean.

💬 Example Output

id = "/subscriptions/.../providers/Microsoft.Kusto/clusters/adxcaseyprod01"
name = "adxcaseyprod01"
uri = "https://adxcaseyprod01.eastus2.kusto.windows.net"
data_ingestion_uri = "https://ingest-adxcaseyprod01.eastus2.kusto.windows.net"
identity_principal_id = "aaaaaaaa-bbbb-cccc-dddd-eeeeeeeeeeee"
database_ids = {
 "telemetry" = "/subscriptions/.../clusters/adxcaseyprod01/databases/telemetry"
}
eventhub_data_connection_ids = {
 "telemetry_stream" = "/subscriptions/.../dataConnections/telemetry-stream"
}

🔍 Troubleshooting

sku.capacity required error. The provider requires capacity unless optimized_auto_scale is set. Supply one or the other.
zones / double_encryption_enabled won't change. Both are immutable — Terraform will plan a full cluster replacement. Set them at creation; to change them on an existing cluster, plan a deliberate recreate (and migrate data first).
Availability zones / auto-scale rejected on a Dev SKU. Dev(No SLA)_* SKUs don't support zones or optimized auto-scale. Move to a production engine SKU (*_v4/*_v5).
Private endpoint creation fails on a VNet-injected cluster. Private endpoints are not supported for clusters that use virtual_network_configuration. Choose one connectivity model; virtual_network_configuration is also a deprecated pattern — prefer private endpoints + managed private endpoints.
Public access still open after apply. Setting public_network_access_enabled = false blocks the public endpoint, but inbound private access requires a separate tf-mod-azure-private-endpoint. The cluster firewall only governs the public endpoint.
Data connection applies before the table exists. Event Hub/Grid/IoT/Cosmos connections require the target table (and mapping rule) to already exist. Bootstrap them with a script first — the module already orders data connections depends_on the databases, but it cannot order them after table creation inside a script; sequence table-creating scripts ahead of connections.
Streamed data not reaching a follower. If streaming ingestion is enabled on the leader, it must also be enabled on this follower cluster (streaming_ingestion_enabled = true) to receive streamed extents.
AllowedFqdnList blocks ingestion. When outbound_network_access_restricted = true, you must list every egress FQDN in allowed_fqdns (e.g. your storage account blob endpoint) or ingestion/callouts fail silently.
Cosmos DB / IoT Hub connection re-creates on every plan diff. These resource types are fully immutable — any attribute change forces replacement. Treat their inputs as set-once.
Forbidden writing CMK key. The cluster identity lacks Key Vault permissions. Grant the system (or user_identity) Get/WrapKey/UnwrapKey on the key before applying customer_managed_key.
Throttling / "aborted due to throttling" on small SKUs. Dev/Extra-small SKUs throttle under concurrent query/management load. Scale to the next SKU. Note the control-plane budget of ~1,000 write ops/hour when applying very large for_each maps in a single run.
language_extensions shows disk pressure when idle. The language-extension sandbox reserves disk even with no query running — size the SKU accordingly when enabling PYTHON/R.

🔗 Related Docs

Terraform Registry — azurerm_kusto_cluster and the azurerm_kusto_* resource family
Microsoft Learn — What is Azure Data Explorer? and How Azure Data Explorer works
Microsoft Learn — Network security for Azure Data Explorer (private endpoints, managed private endpoints, public access)
Microsoft Learn — Reliability in Azure Data Explorer (availability zones, storage redundancy)
Microsoft Learn — Azure subscription and service limits (Azure Data Explorer limits)
Microsoft Learn — Customer-managed keys for Azure Data Explorer

💙 "Infrastructure as Code should be standardized, consistent, and secure."

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Repository files navigation

🚀 Azure Data Explorer (Kusto) Terraform Module

🧩 Overview

❤️ Support this project

🗺️ Where this fits in the family

🧬 What this module builds

✅ Provider / Versions

📁 Module Structure

⚙️ Quick Start

🔌 Cross-Module Contract

Consumes

Emits

📚 Example Library

📥 Inputs

🧾 Outputs

🧠 Architecture Notes

🧱 Design Principles

🚀 Runbook

🧪 Testing

💬 Example Output

🔍 Troubleshooting

🔗 Related Docs

About

Uh oh!

Releases

Packages

Uh oh!

Contributors

Uh oh!

Languages

Name		Name	Last commit message	Last commit date
Latest commit History 3 Commits
.gitignore		.gitignore
LICENSE		LICENSE
README.md		README.md
SCOPE.md		SCOPE.md
main.tf		main.tf
outputs.tf		outputs.tf
providers.tf		providers.tf
variables.tf		variables.tf

Folders and files

Latest commit

History

Repository files navigation

🚀 Azure Data Explorer (Kusto) Terraform Module

🧩 Overview

❤️ Support this project

🗺️ Where this fits in the family

🧬 What this module builds

✅ Provider / Versions

📁 Module Structure

⚙️ Quick Start

🔌 Cross-Module Contract

Consumes

Emits

📚 Example Library

📥 Inputs

🧾 Outputs

🧠 Architecture Notes

🧱 Design Principles

🚀 Runbook

🧪 Testing

💬 Example Output

🔍 Troubleshooting

🔗 Related Docs

About

Topics

Resources

License

Uh oh!

Stars

Watchers

Forks

Releases

Packages 0

Uh oh!

Contributors

Uh oh!

Languages

Packages