Skip to content

microsoftexpert/tf-mod-fabric-environment

Folders and files

NameName
Last commit message
Last commit date

Latest commit

Β 

History

4 Commits
Β 
Β 
Β 
Β 
Β 
Β 
Β 
Β 
Β 
Β 
Β 
Β 
Β 
Β 
Β 
Β 
Β 
Β 

Repository files navigation

πŸ“Š Microsoft Fabric Environment Terraform Module

Manages a Fabric Environment item (fabric_environment) together with its optional, singleton Spark compute-settings companion (fabric_spark_environment_settings). Targets the microsoft/fabric provider ~> 1.12.0.

Terraform provider module type resources posture


🧩 Overview

  • πŸ“Š Creates one fabric_environment item β€” a general-purpose Fabric Environment scoped to a workspace (and, optionally, a folder).
  • ⚑ Optionally creates one fabric_spark_environment_settings record β€” the Environment's Spark compute-settings companion β€” as a singleton (0 or 1 instance), never a for_each-keyed collection.
  • 🏷️ Supports the genuine tags (Set-of-String, fabric_tag GUIDs) attribute on fabric_environment.
  • πŸ”’ Ships secure by default: no Spark settings record, no tags, and workspace-root folder placement unless the caller explicitly opts in.

πŸ’‘ Why it matters: this module is a merge of what earlier planning (this module suite's fabric convention module suite, the original module-catalog planning notes' production-names table) guessed would be two separate modules β€” a standalone tf-mod-fabric-environment and a composite tf-mod-fabric-spark-environment. Reconciling against the live microsoft/fabric v1.12.0 schema confirmed no fabric_spark_environment resource exists in the provider at all β€” the real shape is one Environment item with one optional Spark-settings companion keyed 1:1 to it. See this module's SCOPE.md for the full "MERGED CORRECTION" writeup, and Β§ Architecture Notes below for how that shapes main.tf.


❀️ Support this project

If these Terraform modules have been helpful to you or your organization, I'd appreciate your support in any of the following ways:

Whether it's a star, a professional connection, or a coffee, every gesture helps keep these modules actively maintained and continually improving. Thank you for being part of the community!


πŸ—ΊοΈ Where this fits

graph TD
 workspace["tf-mod-fabric-workspace"]
 folder["tf-mod-fabric-folder"]
 tag["tf-mod-fabric-tag"]
 pool["tf-mod-fabric-spark-custom-pool"]
 environment["tf-mod-fabric-environment"]

 workspace -->|"workspace_id"| environment
 folder -->|"folder_id (optional)"| environment
 tag -->|"tags (optional GUIDs)"| environment
 pool -->|"pool.name / pool.type (informational)"| environment

 style environment fill:#0F6CBD,color:#ffffff
 style workspace fill:#143551,color:#ffffff
 style folder fill:#EDEDED,color:#111111
 style tag fill:#EDEDED,color:#111111
 style pool fill:#EDEDED,color:#111111
Loading

tf-mod-fabric-workspace is the keystone/target every Environment is scoped under. tf-mod-fabric-folder and tf-mod-fabric-tag are consumed by GUID, never owned. tf-mod-fabric-spark-custom-pool is referenced informationally by name/type inside spark_settings.pool β€” this module does not create or manage custom pools.


🧬 What this builds

graph TD
 env["fabric_environment.this"]
 settings["fabric_spark_environment_settings.this[0] 0..1 singleton (count, not for_each)"]

 env -->|"environment_id"| settings

 style env fill:#0F6CBD,color:#ffffff
 style settings fill:#143551,color:#ffffff
Loading
Resource Cardinality Notes
fabric_environment.this 1 (keystone) Always created
fabric_spark_environment_settings.this 0..1 (count, singleton) Created only when var.spark_settings is non-null

βœ… Provider / Versions

Requirement Value
Terraform >= 1.12.0
microsoft/fabric provider ~> 1.12.0 (confirm still current β€” the provider ships roughly every two weeks)
Provider block None β€” the caller configures provider "fabric" {} and auth

Schema notes that bite (confirmed against the live v1.12.0 schema via the terraform-registry MCP and a terraform providers schema -json pull against the real provider binary):

  • tags is a genuine, schema-confirmed Set of String on fabric_environment β€” a set of fabric_tag GUIDs, not a key/value map. This directly contradicts this module suite's naming convention' blanket "no tags argument anywhere in this provider's schema" framing and this module suite's tags-naming rule's original "no tags variable is ever added" instruction β€” both predate this discovery. Never pass a literal string; only GUIDs from tf-mod-fabric-tag.
  • workspace_id, environment_id, and publication_status are all ForceNew on fabric_spark_environment_settings β€” changing publication_status between "Published" and "Staging" after creation destroys and recreates the settings record, it is never an in-place update.
  • Every nested structure on both resources β€” timeouts, dynamic_executor_allocation, pool β€” is a Terraform Plugin Framework Attribute, not a classic block. Confirmed empirically via terraform providers schema -json against the real v1.12.0 binary: attempting dynamic "timeouts" { content {... } } fails terraform validate with Unsupported block type. All of these are assigned directly with = as object literals in main.tf.
  • The dynamic_executor_allocation.min_executors/.max_executors pairing is enforced independently of enabled. This module's original scaffold assumed both fields were required when enabled = true and forbidden when enabled = false. A live terraform validate against the real v1.12.0 provider binary shows the provider's actual attribute validator only enforces "both set or both omitted" β€” enabled = true with no min_executors/max_executors validates cleanly, as does enabled = false with both set. variables.tf models only the confirmed constraint.
  • driver_cores/executor_cores/driver_memory/executor_memory are closed enums (4/8/16/ 32/64 and "28g"/"56g"/"112g"/"224g"/"400g" respectively), but the provider does not statically cross-validate that a chosen core/memory pair is a legal combination for the underlying Spark pool β€” an invalid pairing can still surface only as an API-side error at apply.
  • pool.name and pool.type are required together by the provider's own schema β€” this module models pool as an object with both fields required (not independently optional), so the type system enforces the pairing for free; there is no dedicated validation{} needed for it.
  • properties.publish_details is read-only and reflects the state of the last publish attempt (Cancelled/Cancelling/Failed/Running/Success/Waiting) β€” do not treat a Waiting/Running state surfaced in publish_state as a plan-time failure.
  • No delegated-auth exception applies to either resource as of v1.12.0 β€” both provider docs explicitly state "This resource supports Service Principal authentication." Reconfirm at authoring time per this module suite's authentication-model convention.

πŸ”‘ Required Fabric / Entra Permissions

  • Fabric Workspace Contributor (minimum) on the target workspace β€” required to create, update, and delete an Environment item and its Spark settings sub-resource. <VERIFY: confirm the exact minimum Fabric workspace role against Microsoft Learn's current Fabric permissions model β€” Fabric RBAC is workspace-role-derived rather than a separate item-level ACL system, and role names/grants have shifted across recent Fabric admin-portal releases.>
  • Entra: the calling Service Principal must be enabled under the tenant's "Service principals can use Fabric APIs" setting (see this module suite's authentication-model convention) β€” a blanket Fabric-API prerequisite, not specific to Environment items.
  • No delegated-auth-only APIs are known to apply to fabric_environment or fabric_spark_environment_settings as of v1.12.0.

Microsoft Fabric Prerequisites

  • The target workspace's backing capacity must be Active (not paused) for Environment creation and publish operations to succeed reliably.
  • Tenant-level SPN/MSI Fabric API access must already be enabled in the Fabric Admin Portal (see Β§ Required Permissions above) β€” this module cannot enable it and cannot detect that it's missing from a plan.
  • Environment "publish" is an asynchronous, potentially long-running operation β€” a caller applying spark_settings changes should expect a publish cycle; generous spark_settings_timeouts are recommended (see Β§ Architecture Notes).

πŸ“ Module Structure

tf-mod-fabric-environment/
β”œβ”€β”€ providers.tf # required_providers block (fabric ~> 1.12.0); no provider {} block
β”œβ”€β”€ variables.tf # display_name, workspace_id, description, folder_id, tags, spark_settings, timeouts, spark_settings_timeouts
β”œβ”€β”€ main.tf # fabric_environment.this (keystone) + fabric_spark_environment_settings.this (count-based singleton)
β”œβ”€β”€ outputs.tf # id, display_name, spark_environment_settings_id, publish_state
β”œβ”€β”€ README.md # this file
β”œβ”€β”€ SCOPE.md # cross-module contract (written before the .tf files were authored, per the composite-module rule)
└── examples/
 β”œβ”€β”€ basic/ # Environment only, no spark_settings
 └── complete/ # Environment + full spark_settings + folder + tags

βš™οΈ Quick Start

# Caller's root module configures the fabric provider/auth β€” never this module.
provider "fabric" {
  # tenant_id / client_id / client_secret, or use_oidc / use_msi, per environment
}

module "env_claims_default" {
  source = "git::https://github.com/microsoftexpert/tf-mod-fabric-environment.git?ref=v1.0.0"

  display_name = "env-claims-default-dev"
  workspace_id = module.workspace_claims.id
}

πŸ”Œ Cross-Module Contract

Consumes

Input Type Source module
workspace_id string tf-mod-fabric-workspace
folder_id string (optional) tf-mod-fabric-folder
tags (tag IDs) set(string) (optional) tf-mod-fabric-tag
spark_settings.pool.name/.type string (optional, informational) tf-mod-fabric-spark-custom-pool

Emits

Output Description Consumed by
id Environment item GUID Any item module attaching a default Spark environment
display_name Environment display name Informational
spark_environment_settings_id Settings child GUID, null if unset Informational
publish_state Computed properties.publish_details.state Informational / pipeline gating

πŸ“š Example Library

1 Β· Minimal Environment β€” no Spark settings

The safe "empty call": no fabric_spark_environment_settings record is created.

module "env_claims_default" {
  source = "git::https://github.com/microsoftexpert/tf-mod-fabric-environment.git?ref=v1.0.0"

  display_name = "env-claims-default-dev"
  workspace_id = module.workspace_claims.id
}

ℹ️ This Environment runs under default Fabric-managed Spark compute.

2 Β· Environment with a description and explicit folder placement
module "env_claims_raw" {
  source = "git::https://github.com/microsoftexpert/tf-mod-fabric-environment.git?ref=v1.0.0"

  display_name = "env-claims-raw-dev"
  description  = "Default-compute Environment for raw-zone Claims notebooks."
  workspace_id = module.workspace_claims.id
  folder_id    = module.folder_raw.id
}

πŸ’‘ folder_id defaults to null (workspace root) per this module suite's conventions Β§ Secure-by-default β€” supply it explicitly to nest under a governed folder hierarchy.

3 Β· Environment with tags from tf-mod-fabric-tag
module "env_claims_tagged" {
  source = "git::https://github.com/microsoftexpert/tf-mod-fabric-environment.git?ref=v1.0.0"

  display_name = "env-claims-tagged-prod"
  workspace_id = module.workspace_claims.id

  tags = [
    module.tag_data_classification_internal.id
  ]
}

πŸ”’ tags on fabric_environment is a Set-of-String of fabric_tag GUIDs β€” never a literal string.

4 Β· Minimal Spark settings (Published, no sizing)
module "env_claims_spark_minimal" {
  source = "git::https://github.com/microsoftexpert/tf-mod-fabric-environment.git?ref=v1.0.0"

  display_name = "env-claims-spark-minimal-dev"
  workspace_id = module.workspace_claims.id

  spark_settings = {
    publication_status = "Published"
  }
}
5 Β· Small driver/executor sizing (4 cores / 28g)
module "env_claims_spark_small" {
  source = "git::https://github.com/microsoftexpert/tf-mod-fabric-environment.git?ref=v1.0.0"

  display_name = "env-claims-spark-small-dev"
  workspace_id = module.workspace_claims.id

  spark_settings = {
    publication_status = "Published"
    driver_cores       = 4
    driver_memory      = "28g"
    executor_cores     = 4
    executor_memory    = "28g"
  }
}
6 Β· Large driver/executor sizing (64 cores / 400g)
module "env_claims_spark_large" {
  source = "git::https://github.com/microsoftexpert/tf-mod-fabric-environment.git?ref=v1.0.0"

  display_name = "env-claims-spark-large-prod"
  workspace_id = module.workspace_claims.id

  spark_settings = {
    publication_status = "Published"
    driver_cores       = 64
    driver_memory      = "400g"
    executor_cores     = 64
    executor_memory    = "400g"
  }
}

⚠️ The provider does not statically validate that a core/memory pair is a legal combination for the underlying capacity SKU β€” an invalid pairing surfaces only as an API-side error at apply.

7 Β· Dynamic executor allocation enabled
module "env_claims_spark_dynamic" {
  source = "git::https://github.com/microsoftexpert/tf-mod-fabric-environment.git?ref=v1.0.0"

  display_name = "env-claims-spark-dynamic-prod"
  workspace_id = module.workspace_claims.id

  spark_settings = {
    publication_status = "Published"
    dynamic_executor_allocation = {
      enabled       = true
      min_executors = 2
      max_executors = 8
    }
  }
}
8 Β· Static allocation (dynamic_executor_allocation omitted)
module "env_claims_spark_static" {
  source = "git::https://github.com/microsoftexpert/tf-mod-fabric-environment.git?ref=v1.0.0"

  display_name = "env-claims-spark-static-prod"
  workspace_id = module.workspace_claims.id

  spark_settings = {
    publication_status = "Published"
    driver_cores       = 8
    executor_cores     = 8
  }
}

ℹ️ Omitting dynamic_executor_allocation entirely leaves executor allocation under the provider's own computed default β€” this module never invents a default for a field the provider itself leaves computed.

9 Β· Workspace-scoped custom pool by name/type
module "env_claims_spark_pool_workspace" {
  source = "git::https://github.com/microsoftexpert/tf-mod-fabric-environment.git?ref=v1.0.0"

  display_name = "env-claims-spark-pool-ws-prod"
  workspace_id = module.workspace_claims.id

  spark_settings = {
    publication_status = "Published"
    pool = {
      name = module.spark_custom_pool_claims.name
      type = "Workspace"
    }
  }
}
10 Β· Capacity-scoped starter pool
module "env_claims_spark_pool_capacity" {
  source = "git::https://github.com/microsoftexpert/tf-mod-fabric-environment.git?ref=v1.0.0"

  display_name = "env-claims-spark-pool-cap-prod"
  workspace_id = module.workspace_claims.id

  spark_settings = {
    publication_status = "Published"
    pool = {
      name = "Starter Pool"
      type = "Capacity"
    }
  }
}
11 Β· Staging publication_status
module "env_claims_spark_staging" {
  source = "git::https://github.com/microsoftexpert/tf-mod-fabric-environment.git?ref=v1.0.0"

  display_name = "env-claims-spark-staging-dev"
  workspace_id = module.workspace_claims.id

  spark_settings = {
    publication_status = "Staging"
  }
}

πŸ”’ publication_status is ForceNew β€” flipping this value on a later apply destroys and recreates the settings record rather than updating it in place.

12 Β· Published publication_status (contrast pair to #11)
module "env_claims_spark_published" {
  source = "git::https://github.com/microsoftexpert/tf-mod-fabric-environment.git?ref=v1.0.0"

  display_name = "env-claims-spark-published-prod"
  workspace_id = module.workspace_claims.id

  spark_settings = {
    publication_status = "Published"
  }
}
13 Β· Spark properties
module "env_claims_spark_properties" {
  source = "git::https://github.com/microsoftexpert/tf-mod-fabric-environment.git?ref=v1.0.0"

  display_name = "env-claims-spark-props-prod"
  workspace_id = module.workspace_claims.id

  spark_settings = {
    publication_status = "Published"
    spark_properties = {
      "spark.acls.enable"            = "true"
      "spark.sql.shuffle.partitions" = "200"
    }
  }
}

πŸ”’ Every key must start with "spark.", contain no whitespace, and not end with "." β€” enforced by a validation{} block at terraform validate time.

14 Β· Custom spark_settings_timeouts for a long-running publish
module "env_claims_spark_long_publish" {
  source = "git::https://github.com/microsoftexpert/tf-mod-fabric-environment.git?ref=v1.0.0"

  display_name = "env-claims-spark-longpub-prod"
  workspace_id = module.workspace_claims.id

  spark_settings = {
    publication_status = "Published"
    driver_cores       = 16
    executor_cores     = 16
  }

  spark_settings_timeouts = {
    create = "45m"
    update = "45m"
  }
}
15 Β· πŸ—οΈ End-to-end composition

Wires tf-mod-fabric-workspace, tf-mod-fabric-folder, tf-mod-fabric-tag, and tf-mod-fabric-spark-custom-pool outputs into this module's inputs.

module "workspace_claims" {
  source = "git::https://github.com/microsoftexpert/tf-mod-fabric-workspace.git?ref=v1.0.0"

  display_name = "ws-claims-bronze-prod"
  capacity_id  = azurerm_fabric_capacity.claims.id
}

module "folder_curated" {
  source = "git::https://github.com/microsoftexpert/tf-mod-fabric-folder.git?ref=v1.0.0"

  display_name = "curated"
  workspace_id = module.workspace_claims.id
}

module "tag_data_classification_internal" {
  source = "git::https://github.com/microsoftexpert/tf-mod-fabric-tag.git?ref=v1.0.0"

  display_name = "data-classification:internal"
}

module "spark_custom_pool_claims" {
  source = "git::https://github.com/microsoftexpert/tf-mod-fabric-spark-custom-pool.git?ref=v1.0.0"

  display_name = "pool-claims-spark-prod"
  workspace_id = module.workspace_claims.id
}

module "env_claims_spark_prod" {
  source = "git::https://github.com/microsoftexpert/tf-mod-fabric-environment.git?ref=v1.0.0"

  display_name = "env-claims-spark-prod"
  description  = "Spark-tuned Environment for Claims production data-engineering workloads."
  workspace_id = module.workspace_claims.id
  folder_id    = module.folder_curated.id

  tags = [
    module.tag_data_classification_internal.id
  ]

  spark_settings = {
    publication_status = "Published"
    driver_cores       = 8
    driver_memory      = "56g"
    executor_cores     = 8
    executor_memory    = "56g"

    dynamic_executor_allocation = {
      enabled       = true
      min_executors = 2
      max_executors = 6
    }

    pool = {
      name = module.spark_custom_pool_claims.name
      type = "Workspace"
    }

    spark_properties = {
      "spark.acls.enable" = "true"
    }
  }

  spark_settings_timeouts = {
    create = "30m"
    update = "30m"
  }
}

πŸ“₯ Inputs

Summary: display_name (required), workspace_id (required), description/folder_id (optional, default null), tags (optional, default []), spark_settings (optional, default null), timeouts/spark_settings_timeouts (optional, default null).

Full object schemas
variable "display_name" {
  type = string
}

variable "workspace_id" {
  type = string
}

variable "description" {
  type    = string
  default = null
}

variable "folder_id" {
  type    = string
  default = null
}

variable "tags" {
  type    = set(string)
  default = []
}

variable "spark_settings" {
  type = object({
    publication_status = string
    driver_cores       = optional(number)
    driver_memory      = optional(string)
    executor_cores     = optional(number)
    executor_memory    = optional(string)
    runtime_version    = optional(string)
    dynamic_executor_allocation = optional(object({
      enabled       = optional(bool)
      min_executors = optional(number)
      max_executors = optional(number)
    }))
    pool = optional(object({
      name = string
      type = string
    }))
    spark_properties = optional(map(string), {})
  })
  default = null
}

variable "timeouts" {
  type = object({
    create = optional(string)
    read   = optional(string)
    update = optional(string)
    delete = optional(string)
  })
  default = null
}

variable "spark_settings_timeouts" {
  type = object({
    create = optional(string)
    read   = optional(string)
    update = optional(string)
    delete = optional(string)
  })
  default = null
}

🧾 Outputs

Output Description Sensitive/Conditional
id Environment item GUID (primary output) β€”
display_name Environment display name, echoed back β€”
spark_environment_settings_id Settings child GUID Conditional β€” null when spark_settings was not supplied
publish_state Computed publish state from the last publish attempt β€”

🧠 Architecture Notes

  • The count-based conditional singleton. fabric_spark_environment_settings.this uses count = var.spark_settings != null ? 1: 0 β€” a deliberate, documented exception to "always for_each, never count" (this module suite's code-standard convention). The rule against count exists to protect keyed collections from reordering when one entry is added/removed; a true 0-or-1 singleton with no plural relationship carries no such risk, so count is the correct, minimal tool.
  • workspace_id/environment_id are always derived, never caller-supplied inside spark_settings. main.tf sets workspace_id = var.workspace_id and environment_id = fabric_environment.this.id directly β€” the spark_settings object intentionally has no workspace_id/environment_id fields, so the settings record can never drift to reference a different workspace/environment than the one this module just created.
  • Nested Attributes, not blocks. timeouts, dynamic_executor_allocation, and pool are all Terraform Plugin Framework Attributes on the live v1.12.0 schema (confirmed via terraform providers schema -json), so main.tf assigns each with a plain = object expression (timeouts = var.timeouts != null ? {... }: null), never a dynamic "..." { content {... } } block β€” the latter fails terraform validate with Unsupported block type against this provider.
  • try throughout spark_settings reads. Because the settings resource only exists when count = 1, every field read from var.spark_settings inside fabric_spark_environment_settings.this is wrapped in try(..., null) so a null var.spark_settings never throws a "attempt to get attribute from null value" error during evaluation.
  • The min/max executor pairing is enforced independently of enabled. See Β§ Provider / Versions above β€” variables.tf's validation{} blocks model only the confirmed "both set or both omitted" constraint, not a dependency on enabled's value.

🧱 Design Principles

Concern Safe default Opt-out
Spark compute settings No fabric_spark_environment_settings record (spark_settings = null) Supply a spark_settings object
Folder placement Workspace root (folder_id = null) Supply folder_id
Tags Empty (tags = []) Supply fabric_tag GUIDs
Item definition tokens (n/a to this resource) β€” β€”
Spark property substitution No properties (spark_properties = {}) Supply spark_properties

πŸš€ Runbook

cd C:\GitHubCode\newfabricmodules\tf-mod-fabric-environment
terraform init -backend=false
terraform validate
terraform fmt -check

Pin ?ref=v1.0.0 in every consuming root module β€” never a branch. This module is plan-only; a human applies from a reviewed CI pipeline.


πŸ§ͺ Testing

  • terraform validate/fmt -check catch: enum typos (publication_status, driver_cores, pool.type), the conditional dynamic_executor_allocation.min_executors/.max_executors pairing, malformed spark_properties keys, and missing required fields β€” all at zero cost, before any Fabric API call.
  • Only a live plan/apply against a real tenant exercises: whether the chosen core/memory pairing is actually legal for the underlying capacity SKU, actual publish-cycle timing (properties.publish_details.state), and whether runtime_version is currently supported.

πŸ’¬ Example Output

spark_environment_settings_id = null
environment_id = "9f2b1a3e-4c5d-4e6f-8a9b-0c1d2e3f4a5b"
publish_state = null

# after applying spark_settings:
spark_environment_settings_id = "1a2b3c4d-5e6f-7a8b-9c0d-1e2f3a4b5c6d"
publish_state = "Success"

πŸ” Troubleshooting

Symptom Cause Fix
publish_state stuck at Waiting/Running Environment publish is asynchronous Wait and re-plan/refresh; this is not a Terraform-level failure
Unexpected replace of the settings record publication_status, workspace_id, or environment_id changed These are ForceNew on the provider β€” expect destroy/recreate, not an in-place update
Core/memory combination rejected at apply only The provider does not statically cross-validate legal pairings Confirm the pairing against current Fabric capacity SKU documentation before applying
"Unsupported block type" on timeouts/pool/dynamic_executor_allocation if hand-editing main.tf These are Attributes, not blocks, on this provider Assign directly with =, never dynamic "..." { content {... } }
spark_properties key rejected Key doesn't start with "spark.", contains whitespace, or ends with "." Fix the key per the validation{} error message

πŸ”— Related Docs

  • fabric_environment and fabric_spark_environment_settings provider docs (microsoft/fabric v1.12.0)
  • Sibling modules: tf-mod-fabric-workspace, tf-mod-fabric-folder, tf-mod-fabric-tag, tf-mod-fabric-spark-custom-pool
  • This module's SCOPE.md

About

No description or website provided.

Topics

Resources

License

Stars

0 stars

Watchers

0 watching

Forks

Packages

 
 
 

Contributors

Languages