🧱 Databricks Schema Terraform Module

Provisions a Unity Catalog schema — the second level of the three-level namespace, contained within a catalog — against the databricks/databricks provider ~> 1.117.0.

🧩 Overview

• 🗂️ Creates one databricks_schema beneath an existing catalog.
• 🚫 No isolation_mode equivalent — that knob is catalog-level only; this module does not invent one.
• 🚫 Never accepts a credential, host, or account ID.
• 🌍 Workspace-plane only — this resource cannot be used with an account-level provider.

💡 Why it matters: a schema is where tables and views actually live. Its catalog_name and name are both immutable identity fields — renaming or reparenting a schema is a destroy/recreate event, not an in-place update.

---

❤️ Support this project

If these Terraform modules have been helpful to you or your organization, I'd appreciate your support in any of the following ways:

• ⭐ Star this repository to help others discover this Terraform module.
• 🤝 Connect with me on LinkedIn: linkedin.com/in/microsoftexpert
• ☕ Buy me a coffee: buymeacoffee.com/microsoftexpert

Whether it's a star, a professional connection, or a coffee, every gesture helps keep these modules actively maintained and continually improving. Thank you for being part of the community!

---

🗺️ Where this fits

flowchart LR
 CATALOG["tf-mod-databricks-catalog"]
 style CATALOG fill:#1B3139,color:#fff,stroke:#1B3139,stroke-width:1px

 THIS["tf-mod-databricks-schema"]
 style THIS fill:#FF3621,color:#fff,stroke:#1B3139,stroke-width:1px

 VOLUME["tf-mod-databricks-volume"]
 style VOLUME fill:#F2F2F2,color:#1B3139,stroke:#CCCCCC,stroke-width:1px

 CATALOG -->|"name becomes catalog_name"| THIS
 THIS -->|"name becomes schema_name"| VOLUME

Loading

tf-mod-databricks-catalog is this module's keystone/target sibling — its name output feeds this module's catalog_name input directly. tf-mod-databricks-volume is downstream, consuming this module's name output as its schema_name input.

🧬 What this builds

flowchart TB
 subgraph INPUTS["var.*"]
 NAME["catalog_name / name / storage_root / comment / properties"]
 MODE["enable_predictive_optimization / owner / metastore_id / force_destroy"]
 end

 KEYSTONE["databricks_schema.this"]
 style KEYSTONE fill:#1B3139,color:#fff,stroke:#1B3139,stroke-width:1px

 subgraph OUTPUTS["outputs"]
 ID["id (catalog_name.name) / name"]
 META2["catalog_name / schema_id / metastore_id"]
 end

 NAME --> KEYSTONE
 MODE --> KEYSTONE

 KEYSTONE --> ID
 KEYSTONE --> META2

Loading

Resource inventory: one resource, databricks_schema.this. No child collection.

✅ Provider / Versions

Requirement	Value
Terraform	>= 1.12.0
databricks/databricks	~> 1.117.0
Provider block	None — the caller's root module configures provider "databricks" {}
tags / custom_tags	Not supported by databricks_schema — none added
timeouts	Not supported by databricks_schema — none added

Schema notes that bite:

• catalog_name and name are both force-new per the live provider documentation — changing either destroys and recreates the schema. storage_root is force-new as well.
• No isolation_mode argument exists on this resource — it is a databricks_catalog-only concept. This module deliberately does not carry the catalog module's secure default down to the schema level, because there is no equivalent knob to set.
• metastore_id and owner show the same docs-vs-schema ambiguity as tf-mod-databricks-catalog's metastore_id — this module follows the machine-readable schema (settable, nullable) for consistency with its siblings.
• provider_config { workspace_id } is deliberately not exposed, applied consistently across the whole Unity Catalog chain. Like databricks_catalog, this resource has no top-level api attribute at all.
• id is a composite string, <catalog_name>.<name> — not a separately generated identifier.

🔑 Required Databricks Permissions & Scopes

• CREATE_SCHEMA privilege on the parent catalog (or catalog owner / metastore admin).
• Confirmed against the live provider documentation: "This resource can only be used with a workspace-level provider!"

Databricks Prerequisites

• Workspace-level provider context — confirmed via the provider's own documentation.
• The referenced catalog_name must already exist — created by tf-mod-databricks-catalog or equivalent.

📁 Module Structure

tf-mod-databricks-schema/
├── providers.tf # required_providers only — no provider {} block
├── variables.tf # catalog_name, name (both required), storage_root, comment, properties,...
├── main.tf # databricks_schema.this
├── outputs.tf # id first, then name and computed schema metadata
├── SCOPE.md # cross-module contract
├── README.md # this file
└── examples/
 └── basic/
 └── main.tf # smallest real, runnable call

⚙️ Quick Start

module "raw_schema" {
  source = "git::https://github.com/microsoftexpert/tf-mod-databricks-schema.git?ref=v1.0.0"

  catalog_name = "analytics"
  name         = "raw"
  comment      = "Raw ingestion schema, managed by Terraform"
}

🔌 Cross-Module Contract

Consumes:

Input	Type	Source module
catalog_name	string	tf-mod-databricks-catalog output name

Emits:

Output	Description	Consumed by
id	ID of this schema, <catalog_name>.<name>	Auditing / drift-detection tooling
name	Schema name	tf-mod-databricks-volume (schema_name input)
catalog_name	Echo of the parent catalog_name input	Auditing
schema_id	Unique identifier of the schema	Auditing
metastore_id	ID of the metastore the parent catalog belongs to	Auditing

📚 Example Library

1 · Minimal schema

module "raw_schema" {
  source = "git::https://github.com/microsoftexpert/tf-mod-databricks-schema.git?ref=v1.0.0"

  catalog_name = "analytics"
  name         = "raw"
}

2 · Schema with comment and properties

module "curated_schema" {
  source = "git::https://github.com/microsoftexpert/tf-mod-databricks-schema.git?ref=v1.0.0"

  catalog_name = "analytics"
  name         = "curated"
  comment      = "Cleaned, curated analytics tables"

  properties = {
    layer = "curated"
  }
}

3 · Explicit managed storage root

module "large_tables_schema" {
  source = "git::https://github.com/microsoftexpert/tf-mod-databricks-schema.git?ref=v1.0.0"

  catalog_name = "analytics"
  name         = "large-tables"
  storage_root = "abfss://analytics-large-tables@caseyuc.dfs.core.windows.net/"
}

⚠️ storage_root is force-new — changing it destroys and recreates the schema.

4 · Predictive optimization explicitly enabled

module "ml_features_schema" {
  source = "git::https://github.com/microsoftexpert/tf-mod-databricks-schema.git?ref=v1.0.0"

  catalog_name                   = "ml"
  name                           = "features"
  enable_predictive_optimization = "ENABLE"
}

5 · Explicit owner

module "governed_schema" {
  source = "git::https://github.com/microsoftexpert/tf-mod-databricks-schema.git?ref=v1.0.0"

  catalog_name = "analytics"
  name         = "governed"
  owner        = "uc-admins"
}

6 · Explicit metastore override

module "cross_region_schema" {
  source = "git::https://github.com/microsoftexpert/tf-mod-databricks-schema.git?ref=v1.0.0"

  catalog_name = "cross-region"
  name         = "reporting"
  metastore_id = "aaaaaaaa-bbbb-cccc-dddd-eeeeeeeeeeee"
}

7 · Destroy protection explicitly disabled (deliberate teardown)

module "decommission_schema" {
  source = "git::https://github.com/microsoftexpert/tf-mod-databricks-schema.git?ref=v1.0.0"

  catalog_name  = "sandbox"
  name          = "decommission-me"
  force_destroy = true
}

⚠️ Secure default is false. Only set true immediately before a reviewed, deliberate teardown.

8 · for_each-driven multi-schema creation at scale

locals {
  layer_schemas = {
    raw    = "Raw ingestion layer"
    bronze = "Bronze/cleaned layer"
    silver = "Silver/conformed layer"
    gold   = "Gold/aggregated layer"
  }
}

module "medallion_schemas" {
  for_each = local.layer_schemas
  source   = "git::https://github.com/microsoftexpert/tf-mod-databricks-schema.git?ref=v1.0.0"

  catalog_name = "analytics"
  name         = each.key
  comment      = each.value
}

ℹ️ for_each is applied at the caller's root-module level — this module itself has no child collection to iterate; each instance creates exactly one schema.

9 · Multiple catalogs, one schema naming convention each

locals {
  domain_catalogs = ["analytics", "finance", "ml"]
}

module "raw_schemas" {
  for_each = toset(local.domain_catalogs)
  source   = "git::https://github.com/microsoftexpert/tf-mod-databricks-schema.git?ref=v1.0.0"

  catalog_name = each.key
  name         = "raw"
}

10 · Minimal least-privilege baseline (recommended starting point)

module "baseline_schema" {
  source = "git::https://github.com/microsoftexpert/tf-mod-databricks-schema.git?ref=v1.0.0"

  catalog_name = "analytics"
  name         = "baseline"
  # force_destroy left at its secure default: false
}

🏗️ 11 · End-to-end composition — catalog → schema → volume

module "analytics_catalog" {
  source = "git::https://github.com/microsoftexpert/tf-mod-databricks-catalog.git?ref=v1.0.0"

  name = "analytics"
}

module "raw_schema" {
  source = "git::https://github.com/microsoftexpert/tf-mod-databricks-schema.git?ref=v1.0.0"

  catalog_name = module.analytics_catalog.name
  name         = "raw"
}

module "raw_landing_volume" {
  source = "git::https://github.com/microsoftexpert/tf-mod-databricks-volume.git?ref=v1.0.0"

  catalog_name = module.analytics_catalog.name
  schema_name  = module.raw_schema.name
  name         = "landing"
}

ℹ️ tf-mod-databricks-volume is a seeded module in this same catalog batch — this composition reflects its planned contract, not yet a verified cross-module terraform plan.

📥 Inputs

Variable	Type	Default	Notes
catalog_name	string	— (required)	Force-new
name	string	— (required)	Force-new
storage_root	string	null	Force-new
comment	string	null
properties	map(string)	{}
enable_predictive_optimization	string	null	ENABLE | DISABLE | INHERIT
owner	string	null
metastore_id	string	null
force_destroy	bool	false	Secure default

Full variable declarations

variable "catalog_name" {
  type = string
}

variable "name" {
  type = string
}

variable "storage_root" {
  type    = string
  default = null
}

variable "comment" {
  type    = string
  default = null
}

variable "properties" {
  type    = map(string)
  default = {}
}

variable "enable_predictive_optimization" {
  type    = string
  default = null
  # validation: must be "ENABLE", "DISABLE", or "INHERIT" when non-null
}

variable "owner" {
  type    = string
  default = null
}

variable "metastore_id" {
  type    = string
  default = null
}

variable "force_destroy" {
  type    = bool
  default = false
}

🧾 Outputs

Output	Description	Sensitive?
id	ID of this schema, <catalog_name>.<name>	No
name	Schema name	No
catalog_name	Echo of the parent catalog_name input	No
schema_id	Unique identifier of the schema	No
metastore_id	ID of the metastore the parent catalog belongs to	No

🧠 Architecture Notes

• catalog_name, name, and storage_root are all force-new. Changing any of them destroys and recreates the schema, cascading to every table registered underneath (unless force_destroy is set).
• No isolation_mode field exists on databricks_schema. A schema inherits its parent catalog's isolation posture; there is no schema-level override in this provider version.
• id is a composite string, <catalog_name>.<name> — not a separately generated identifier.
• No for_each, no child resources. Example breadth comes from configuration variants and for_each-at-scale composition at the caller level, not from anything internal to this module.

🧱 Design Principles

Concern	Secure default	Opt-out (caller must set explicitly)
Destroy protection	force_destroy = false	Explicit true required for a deliberate, reviewed teardown

This module has no isolation-mode-equivalent secure default to set — see Architecture Notes for why. force_destroy follows the same destructive-action-requires-opt-in pattern established by tf-mod-databricks-metastore and tf-mod-databricks-catalog.

🚀 Runbook

cd tf-mod-databricks-schema
terraform init -backend=false
terraform validate
terraform fmt -check

Pin consumers to an immutable tag — ?ref=v1.0.0 — never a branch. This module is plan-only; a human applies from CI after review.

🧪 Testing

terraform validate / terraform fmt -check catch: missing catalog_name/name, the enable_predictive_optimization enum validation, and malformed HCL. They do not catch: whether the applying identity actually holds CREATE_SCHEMA rights, whether the referenced catalog_name actually exists, or any real Unity Catalog API-side constraint. Those require an actual plan/apply against a live workspace, out of scope for this authoring process.

💬 Example Output

$ terraform output
catalog_name = "analytics"
id = "analytics.raw"
metastore_id = "12345678-90ab-cdef-1234-567890abcdef"
name = "raw"
schema_id = "87654321-fedc-ba09-8765-432109876543"

🔍 Troubleshooting

Symptom	Cause	Fix
Apply fails with a permissions error even though terraform validate passed	Applying identity lacks CREATE_SCHEMA on the parent catalog	Confirm the identity holds the required Unity Catalog privilege
Apply attempts to replace the schema unexpectedly	catalog_name, name, or storage_root was changed	These are force-new; treat any change as a deliberate migration, not a routine edit
terraform destroy fails with a "schema not empty" style error	force_destroy is false (the secure default) and the schema still has registered tables	Confirm the teardown is intentional, then set force_destroy = true explicitly for that run only
Apply fails because the parent catalog doesn't exist	catalog_name is wrong or the catalog hasn't been created yet	Confirm tf-mod-databricks-catalog applied successfully first

🔗 Related Docs

• databricks_schema provider resource
• tf-mod-databricks-catalog (upstream, provides catalog_name)
• tf-mod-databricks-volume (downstream, consumes this module's name)
• This module's SCOPE.md

💙 "Infrastructure as Code should be standardized, consistent, and secure."