🧱 Databricks Catalog Terraform Module

Provisions a Unity Catalog catalog — the top level of the three-level namespace beneath a metastore — against the databricks/databricks provider ~> 1.117.0.

🧩 Overview

🗂️ Creates one databricks_catalog — a standard managed catalog, a Foreign Catalog, or a Delta Sharing Catalog, all modeled by the same provider resource.
🔐 Defaults to isolation_mode = "ISOLATED" — a catalog is not open to every workspace by default.
🚫 Never accepts a credential, host, or account ID — the caller's workspace-level provider supplies those.
🌍 Workspace-plane only — this resource cannot be used with an account-level provider.

💡 Why it matters: every schema, volume, and table beneath this catalog inherits its isolation posture and encryption configuration. Getting isolation_mode and managed_encryption_settings right here sets the boundary for everything created underneath.

❤️ Support this project

If these Terraform modules have been helpful to you or your organization, I'd appreciate your support in any of the following ways:

⭐ Star this repository to help others discover this Terraform module.
🤝 Connect with me on LinkedIn: linkedin.com/in/microsoftexpert
☕ Buy me a coffee: buymeacoffee.com/microsoftexpert

Whether it's a star, a professional connection, or a coffee, every gesture helps keep these modules actively maintained and continually improving. Thank you for being part of the community!

🗺️ Where this fits

flowchart LR
 META["tf-mod-databricks-metastore"]
 ASSIGN["tf-mod-databricks-metastore-assignment"]
 style META fill:#F2F2F2,color:#1B3139,stroke:#CCCCCC,stroke-width:1px
 style ASSIGN fill:#F2F2F2,color:#1B3139,stroke:#CCCCCC,stroke-width:1px

 THIS["tf-mod-databricks-catalog"]
 style THIS fill:#FF3621,color:#fff,stroke:#1B3139,stroke-width:1px

 SCHEMA["tf-mod-databricks-schema"]
 style SCHEMA fill:#1B3139,color:#fff,stroke:#1B3139,stroke-width:1px

 META -->|"id becomes metastore_id"| ASSIGN
 ASSIGN -.->|"binds metastore to workspace (no direct reference)"| THIS
 THIS -->|"name becomes catalog_name"| SCHEMA

This module sits between the metastore chain (upstream, not directly referenced by ID — a metastore must simply be assigned to the calling workspace) and tf-mod-databricks-schema (downstream, consuming this module's name output as its catalog_name input).

🧬 What this builds

flowchart TB
 subgraph INPUTS["var.*"]
 NAME["name / metastore_id / comment / properties / options"]
 MODE["isolation_mode / enable_predictive_optimization / owner / force_destroy / browse_only"]
 FLAVOR["storage_root -or- connection_name -or- provider_name+share_name (mutually exclusive)"]
 ENC["managed_encryption_settings (Azure CMK, optional)"]
 end

 KEYSTONE["databricks_catalog.this"]
 style KEYSTONE fill:#1B3139,color:#fff,stroke:#1B3139,stroke-width:1px

 subgraph OUTPUTS["outputs"]
 ID["id / name (identical)"]
 META2["metastore_id / full_name / catalog_type / storage_location / securable_type"]
 end

 NAME --> KEYSTONE
 MODE --> KEYSTONE
 FLAVOR --> KEYSTONE
 ENC --> KEYSTONE

 KEYSTONE --> ID
 KEYSTONE --> META2

Resource inventory: one resource, databricks_catalog.this. No child collection.

✅ Provider / Versions

Requirement	Value
Terraform	`>= 1.12.0`
`databricks/databricks`	`~> 1.117.0`
Provider block	None — the caller's root module configures `provider "databricks" {}`
`tags` / `custom_tags`	Not supported — `properties` (a generic map) is the closest analog, not a tagging contract
`timeouts`	Not supported by `databricks_catalog` — none added

Schema notes that bite:

metastore_id is optional/computed, not required. Omitting it defaults catalog creation to the metastore already assigned to the calling workspace — only set it explicitly for a workspace with access to more than one metastore. The provider's prose documentation lists metastore_id only under "Attribute Reference" (implying output-only), while the machine-readable schema marks it settable; this module follows the machine-readable schema, the same source terraform validate uses.
Foreign Catalog and Delta Sharing Catalog variants are supported in v1.0.0, not deferred — connection_name (Foreign Catalog) and provider_name/share_name (Delta Sharing Catalog) are typed inputs, validated mutually exclusive with each other and with storage_root (standard managed catalog) via cross-variable validation {} blocks (supported since Terraform 1.9).
provider_config { workspace_id } is deliberately not exposed, applied consistently across the whole Unity Catalog chain (metastore, metastore assignment, catalog, schema, volume) — plane targeting stays a provider-block concern. Unlike databricks_metastore, this resource has no top-level api attribute at all — a genuine inconsistency within the provider itself, not a design choice this module made.
id equals name for this resource — Databricks uses the catalog name as its identifier, not a separately generated value.

🔑 Required Databricks Permissions & Scopes

CREATE_CATALOG privilege on the parent metastore (or metastore admin / account admin).
If connection_name is set (Foreign Catalog): CREATE_FOREIGN_CATALOG and usage rights on the referenced databricks_connection.
If provider_name/share_name is set (Delta Sharing Catalog): rights to create a catalog from the referenced Delta Sharing provider/share.

Databricks Prerequisites

Workspace-level provider context — confirmed against the live provider documentation: "This resource can only be used with a workspace-level provider!"
A metastore must already be assigned to the target workspace via tf-mod-databricks-metastore-assignment (or equivalent) before this module applies, even though this module takes no direct Terraform reference to that assignment.
If metastore_id is left null, the workspace must have exactly one assigned metastore.

📁 Module Structure

tf-mod-databricks-catalog/
├── providers.tf # required_providers only — no provider {} block
├── variables.tf # name, metastore_id, storage_root/connection_name/provider+share_name,...
├── main.tf # databricks_catalog.this + dynamic managed_encryption_settings
├── outputs.tf # id first, then name and computed catalog metadata
├── SCOPE.md # cross-module contract
├── README.md # this file
└── examples/
 └── basic/
 └── main.tf # smallest real, runnable call

⚙️ Quick Start

module "analytics_catalog" {
  source = "git::https://github.com/microsoftexpert/tf-mod-databricks-catalog.git?ref=v1.0.0"

  name    = "analytics"
  comment = "Analytics domain catalog, managed by Terraform"
}

🔌 Cross-Module Contract

Consumes: none as a hard cross-module reference — metastore_id is an optional string that, when set, is not sourced from a sibling module's output in this catalog (a metastore's assignment determines the default, resolved outside Terraform's reference graph).

Emits:

Output	Description	Consumed by
`id`	ID of this catalog — identical to `name`	Auditing / drift-detection tooling
`name`	Catalog name	`tf-mod-databricks-schema` (`catalog_name` input)
`metastore_id`	ID of the parent metastore this catalog was created in	Auditing
`full_name`, `catalog_type`, `storage_location`, `securable_type`	Computed Unity Catalog metadata	Auditing / drift-detection tooling

📚 Example Library

1 · Minimal catalog

module "sandbox_catalog" {
  source = "git::https://github.com/microsoftexpert/tf-mod-databricks-catalog.git?ref=v1.0.0"

  name = "sandbox"
}

2 · Catalog with comment and properties

module "finance_catalog" {
  source = "git::https://github.com/microsoftexpert/tf-mod-databricks-catalog.git?ref=v1.0.0"

  name    = "finance"
  comment = "Finance domain catalog"

  properties = {
    domain      = "finance"
    cost_center = "fin-ops"
  }
}

3 · Explicit managed storage root

module "analytics_catalog" {
  source = "git::https://github.com/microsoftexpert/tf-mod-databricks-catalog.git?ref=v1.0.0"

  name         = "analytics"
  storage_root = "abfss://analytics@caseyuc.dfs.core.windows.net/"
}

⚠️ storage_root is force-new — changing it destroys and recreates the catalog.

4 · Explicit metastore (multi-metastore workspace)

module "cross_region_catalog" {
  source = "git::https://github.com/microsoftexpert/tf-mod-databricks-catalog.git?ref=v1.0.0"

  name         = "cross-region"
  metastore_id = "aaaaaaaa-bbbb-cccc-dddd-eeeeeeeeeeee"
}

5 · Open isolation mode (opt-in)

module "shared_reference_catalog" {
  source = "git::https://github.com/microsoftexpert/tf-mod-databricks-catalog.git?ref=v1.0.0"

  name           = "shared-reference"
  isolation_mode = "OPEN"
}

⚠️ Secure default is ISOLATED. Only set OPEN for a catalog deliberately meant to be visible from every workspace attached to the metastore.

6 · Predictive optimization explicitly enabled

module "ml_features_catalog" {
  source = "git::https://github.com/microsoftexpert/tf-mod-databricks-catalog.git?ref=v1.0.0"

  name                           = "ml-features"
  enable_predictive_optimization = "ENABLE"
}

7 · Foreign Catalog (external data source)

module "postgres_foreign_catalog" {
  source = "git::https://github.com/microsoftexpert/tf-mod-databricks-catalog.git?ref=v1.0.0"

  name            = "postgres-prod"
  connection_name = "prod-postgres-connection"

  options = {
    database = "prod_db"
  }
}

🔒 storage_root must be left null for a Foreign Catalog — this module rejects a call that sets both connection_name and storage_root.

8 · Delta Sharing Catalog

module "shared_partner_catalog" {
  source = "git::https://github.com/microsoftexpert/tf-mod-databricks-catalog.git?ref=v1.0.0"

  name          = "partner-shared-data"
  provider_name = "partner-org-provider"
  share_name    = "quarterly-financials"
}

🔒 provider_name and share_name must be set together — this module rejects a call that sets only one.

9 · Explicit owner

module "governed_catalog" {
  source = "git::https://github.com/microsoftexpert/tf-mod-databricks-catalog.git?ref=v1.0.0"

  name  = "governed"
  owner = "uc-admins"
}

10 · Azure customer-managed-key encryption

module "encrypted_catalog" {
  source = "git::https://github.com/microsoftexpert/tf-mod-databricks-catalog.git?ref=v1.0.0"

  name = "encrypted"

  managed_encryption_settings = {
    azure_encryption_settings = {
      azure_cmk_access_connector_id = "/subscriptions/.../accessConnectors/casey-uc-connector"
      azure_tenant_id               = "00000000-1111-2222-3333-444444444444"
    }
  }
}

ℹ️ Azure-specific — do not set azure_encryption_settings on AWS/GCP workspaces.

11 · Destroy protection explicitly disabled (deliberate teardown)

module "decommission_catalog" {
  source = "git::https://github.com/microsoftexpert/tf-mod-databricks-catalog.git?ref=v1.0.0"

  name          = "decommission-me"
  force_destroy = true
}

⚠️ Secure default is false. Only set true immediately before a reviewed, deliberate teardown.

12 · for_each-driven multi-catalog creation at scale

locals {
  domain_catalogs = {
    analytics = "Analytics domain catalog"
    finance   = "Finance domain catalog"
    ml        = "ML domain catalog"
  }
}

module "domain_catalogs" {
  for_each = local.domain_catalogs
  source   = "git::https://github.com/microsoftexpert/tf-mod-databricks-catalog.git?ref=v1.0.0"

  name    = each.key
  comment = each.value
}

ℹ️ for_each is applied at the caller's root-module level — this module itself has no child collection to iterate; each instance creates exactly one catalog.

13 · Minimal least-privilege baseline (recommended starting point)

module "baseline_catalog" {
  source = "git::https://github.com/microsoftexpert/tf-mod-databricks-catalog.git?ref=v1.0.0"

  name = "baseline"
  # isolation_mode left at its secure default: "ISOLATED"
  # force_destroy left at its secure default: false
  # browse_only left at its secure default: false
}

🏗️ 14 · End-to-end composition — metastore assignment → catalog → schema

module "metastore_assignment" {
  source = "git::https://github.com/microsoftexpert/tf-mod-databricks-metastore-assignment.git?ref=v1.0.0"

  metastore_id = module.metastore.id
  workspace_id = 123456789012345
}

module "analytics_catalog" {
  source = "git::https://github.com/microsoftexpert/tf-mod-databricks-catalog.git?ref=v1.0.0"

  name    = "analytics"
  comment = "Analytics domain catalog"

  depends_on = [module.metastore_assignment]
}

module "analytics_schema" {
  source = "git::https://github.com/microsoftexpert/tf-mod-databricks-schema.git?ref=v1.0.0"

  catalog_name = module.analytics_catalog.name
  name         = "raw"
}

ℹ️ tf-mod-databricks-schema is a seeded module in this same catalog batch — this composition reflects its planned contract, not yet a verified cross-module terraform plan. depends_on is required on the catalog module because it has no direct Terraform reference to the metastore assignment.

📥 Inputs

Variable	Type	Default	Notes
`name`	`string`	— (required)
`metastore_id`	`string`	`null`	Optional/computed — defaults to the workspace's assigned metastore
`storage_root`	`string`	`null`	Force-new; standard managed catalog only
`comment`	`string`	`null`
`properties`	`map(string)`	`{}`	Not a tagging contract
`options`	`map(string)`	`{}`	Foreign Catalog only
`isolation_mode`	`string`	`"ISOLATED"`	`ISOLATED` \| `OPEN`
`enable_predictive_optimization`	`string`	`null`	`ENABLE` \| `DISABLE` \| `INHERIT`
`owner`	`string`	`null`
`force_destroy`	`bool`	`false`	Secure default
`browse_only`	`bool`	`false`	Secure default
`connection_name`	`string`	`null`	Force-new; Foreign Catalog only
`provider_name`	`string`	`null`	Force-new; Delta Sharing Catalog only
`share_name`	`string`	`null`	Force-new; Delta Sharing Catalog only
`managed_encryption_settings`	`object(...)`	`null`	Azure CMK support

Full variable declarations

variable "name" {
  type = string
}

variable "metastore_id" {
  type    = string
  default = null
}

variable "storage_root" {
  type    = string
  default = null
}

variable "comment" {
  type    = string
  default = null
}

variable "properties" {
  type    = map(string)
  default = {}
}

variable "options" {
  type    = map(string)
  default = {}
}

variable "isolation_mode" {
  type    = string
  default = "ISOLATED"
  # validation: must be "ISOLATED" or "OPEN"
}

variable "enable_predictive_optimization" {
  type    = string
  default = null
  # validation: must be "ENABLE", "DISABLE", or "INHERIT" when non-null
}

variable "owner" {
  type    = string
  default = null
}

variable "force_destroy" {
  type    = bool
  default = false
}

variable "browse_only" {
  type    = bool
  default = false
}

variable "connection_name" {
  type    = string
  default = null
  # validation: mutually exclusive with provider_name/share_name and storage_root
}

variable "provider_name" {
  type    = string
  default = null
  # validation: must be set together with share_name; mutually exclusive with storage_root
}

variable "share_name" {
  type    = string
  default = null
}

variable "managed_encryption_settings" {
  type = object({
    azure_key_vault_key_id  = optional(string)
    customer_managed_key_id = optional(string)
    azure_encryption_settings = optional(object({
      azure_cmk_access_connector_id = optional(string)
      azure_cmk_managed_identity_id = optional(string)
      azure_tenant_id               = optional(string)
    }))
  })
  default = null
}

🧾 Outputs

Output	Description	Sensitive?
`id`	ID of this catalog — identical to `name`	No
`name`	Catalog name	No
`metastore_id`	ID of the parent metastore	No
`full_name`	Fully qualified Unity Catalog name	No
`catalog_type`	Type of the catalog (managed / foreign / delta sharing)	No
`storage_location`	Effective storage location URL for managed tables	No
`securable_type`	Type of Unity Catalog securable	No

🧠 Architecture Notes

storage_root, connection_name, and provider_name/share_name are all force-new. Changing any of them destroys and recreates the catalog, cascading to every schema/table registered underneath (unless force_destroy is set, in which case the destroy proceeds instead of failing on non-empty contents).
Catalog "flavor" mutual exclusivity is enforced by this module's own validation {} blocks, not by the provider schema — a standard managed catalog (storage_root), a Foreign Catalog (connection_name), and a Delta Sharing Catalog (provider_name+share_name) are mutually exclusive configurations.
id equals name. There is no separately generated identifier for this resource.
No for_each, no child resources. This is a pure standalone primitive; example breadth comes from configuration variants, not composition.

🧱 Design Principles

Concern	Secure default	Opt-out (caller must set explicitly)
Catalog isolation mode	`isolation_mode = "ISOLATED"`	`"OPEN"` requires explicit opt-in
Destroy protection	`force_destroy = false`	Explicit `true` required for a deliberate, reviewed teardown
Browse-only registration	`browse_only = false`	Explicit `true` required for a deliberate cross-metastore reference

These rows match this library's Secure-by-default convention exactly — isolation_mode = "ISOLATED" is the standard already applied to Unity Catalog objects; force_destroy/browse_only follow the same destructive-action-requires-opt-in pattern established by tf-mod-databricks-metastore.

🚀 Runbook

cd tf-mod-databricks-catalog
terraform init -backend=false
terraform validate
terraform fmt -check

Pin consumers to an immutable tag — ?ref=v1.0.0 — never a branch. This module is plan-only; a human applies from CI after review.

🧪 Testing

terraform validate / terraform fmt -check catch: missing name, the isolation_mode and enable_predictive_optimization enum validations, and the three catalog-flavor mutual-exclusivity validations. They do not catch: whether the applying identity actually holds CREATE_CATALOG rights, whether a referenced connection_name/metastore actually exists, or any real Unity Catalog API-side constraint (naming collision, quota). Those require an actual plan/apply against a live workspace, out of scope for this authoring process.

💬 Example Output

$ terraform output
catalog_type = "MANAGED"
full_name = "analytics"
id = "analytics"
metastore_id = "12345678-90ab-cdef-1234-567890abcdef"
name = "analytics"
securable_type = "CATALOG"
storage_location = "abfss://unity-catalog@caseyuc.dfs.core.windows.net/analytics"

🔍 Troubleshooting

Symptom	Cause	Fix
`terraform validate` fails with a catalog-flavor mutual-exclusivity error	More than one of `storage_root` / `connection_name` / `provider_name`+`share_name` was set	Set only the fields for the catalog flavor you intend (standard managed, Foreign, or Delta Sharing)
Apply fails with a permissions error even though `terraform validate` passed	Applying identity lacks `CREATE_CATALOG` on the metastore	Confirm the identity holds the required Unity Catalog privilege
Apply attempts to replace the catalog unexpectedly	`storage_root`, `connection_name`, or `provider_name`/`share_name` was changed	These are force-new; treat any change as a deliberate migration, not a routine edit
`terraform destroy` fails with a "catalog not empty" style error	`force_destroy` is `false` (the secure default) and the catalog still has registered schemas	Confirm the teardown is intentional, then set `force_destroy = true` explicitly for that run only
Catalog is visible from workspaces it shouldn't be	`isolation_mode` was set to `OPEN`	Revert to the secure default `ISOLATED`, or use workspace bindings to scope access explicitly

🔗 Related Docs

databricks_catalog provider resource
tf-mod-databricks-metastore-assignment (prerequisite — no direct reference)
tf-mod-databricks-schema (downstream, consumes this module's name)
This module's SCOPE.md

💙 "Infrastructure as Code should be standardized, consistent, and secure."

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Repository files navigation

🧱 Databricks Catalog Terraform Module

🧩 Overview

❤️ Support this project

🗺️ Where this fits

🧬 What this builds

✅ Provider / Versions

🔑 Required Databricks Permissions & Scopes

Databricks Prerequisites

📁 Module Structure

⚙️ Quick Start

🔌 Cross-Module Contract

📚 Example Library

📥 Inputs

🧾 Outputs

🧠 Architecture Notes

🧱 Design Principles

🚀 Runbook

🧪 Testing

💬 Example Output

🔍 Troubleshooting

🔗 Related Docs

About

Uh oh!

Releases

Packages

Uh oh!

Contributors

Uh oh!

Languages

Name		Name	Last commit message	Last commit date
Latest commit History 3 Commits
examples/basic		examples/basic
.gitignore		.gitignore
LICENSE		LICENSE
README.md		README.md
SCOPE.md		SCOPE.md
main.tf		main.tf
outputs.tf		outputs.tf
providers.tf		providers.tf
variables.tf		variables.tf

Folders and files

Latest commit

History

Repository files navigation

🧱 Databricks Catalog Terraform Module

🧩 Overview

❤️ Support this project

🗺️ Where this fits

🧬 What this builds

✅ Provider / Versions

🔑 Required Databricks Permissions & Scopes

Databricks Prerequisites

📁 Module Structure

⚙️ Quick Start

🔌 Cross-Module Contract

📚 Example Library

📥 Inputs

🧾 Outputs

🧠 Architecture Notes

🧱 Design Principles

🚀 Runbook

🧪 Testing

💬 Example Output

🔍 Troubleshooting

🔗 Related Docs

About

Topics

Resources

License

Uh oh!

Stars

Watchers

Forks

Releases

Packages 0

Uh oh!

Contributors

Uh oh!

Languages

Packages