Manages a Fabric Environment item (
fabric_environment) together with its optional, singleton Spark compute-settings companion (fabric_spark_environment_settings). Targets themicrosoft/fabricprovider~> 1.12.0.
- π Creates one
fabric_environmentitem β a general-purpose Fabric Environment scoped to a workspace (and, optionally, a folder). - β‘ Optionally creates one
fabric_spark_environment_settingsrecord β the Environment's Spark compute-settings companion β as a singleton (0 or 1 instance), never afor_each-keyed collection. - π·οΈ Supports the genuine
tags(Set-of-String,fabric_tagGUIDs) attribute onfabric_environment. - π Ships secure by default: no Spark settings record, no tags, and workspace-root folder placement unless the caller explicitly opts in.
π‘ Why it matters: this module is a merge of what earlier planning (this module suite's fabric convention module suite, the original module-catalog planning notes' production-names table) guessed would be two separate modules β a standalone
tf-mod-fabric-environmentand a compositetf-mod-fabric-spark-environment. Reconciling against the livemicrosoft/fabricv1.12.0 schema confirmed nofabric_spark_environmentresource exists in the provider at all β the real shape is one Environment item with one optional Spark-settings companion keyed 1:1 to it. See this module'sSCOPE.mdfor the full "MERGED CORRECTION" writeup, and Β§ Architecture Notes below for how that shapesmain.tf.
If these Terraform modules have been helpful to you or your organization, I'd appreciate your support in any of the following ways:
- β Star this repository to help others discover this Terraform module.
- π€ Connect with me on LinkedIn: linkedin.com/in/microsoftexpert
- β Buy me a coffee: buymeacoffee.com/microsoftexpert
Whether it's a star, a professional connection, or a coffee, every gesture helps keep these modules actively maintained and continually improving. Thank you for being part of the community!
graph TD
workspace["tf-mod-fabric-workspace"]
folder["tf-mod-fabric-folder"]
tag["tf-mod-fabric-tag"]
pool["tf-mod-fabric-spark-custom-pool"]
environment["tf-mod-fabric-environment"]
workspace -->|"workspace_id"| environment
folder -->|"folder_id (optional)"| environment
tag -->|"tags (optional GUIDs)"| environment
pool -->|"pool.name / pool.type (informational)"| environment
style environment fill:#0F6CBD,color:#ffffff
style workspace fill:#143551,color:#ffffff
style folder fill:#EDEDED,color:#111111
style tag fill:#EDEDED,color:#111111
style pool fill:#EDEDED,color:#111111
tf-mod-fabric-workspace is the keystone/target every Environment is scoped under. tf-mod-fabric-folder
and tf-mod-fabric-tag are consumed by GUID, never owned. tf-mod-fabric-spark-custom-pool is
referenced informationally by name/type inside spark_settings.pool β this module does not create
or manage custom pools.
graph TD
env["fabric_environment.this"]
settings["fabric_spark_environment_settings.this[0] 0..1 singleton (count, not for_each)"]
env -->|"environment_id"| settings
style env fill:#0F6CBD,color:#ffffff
style settings fill:#143551,color:#ffffff
| Resource | Cardinality | Notes |
|---|---|---|
fabric_environment.this |
1 (keystone) | Always created |
fabric_spark_environment_settings.this |
0..1 (count, singleton) |
Created only when var.spark_settings is non-null |
| Requirement | Value |
|---|---|
| Terraform | >= 1.12.0 |
microsoft/fabric provider |
~> 1.12.0 (confirm still current β the provider ships roughly every two weeks) |
| Provider block | None β the caller configures provider "fabric" {} and auth |
Schema notes that bite (confirmed against the live v1.12.0 schema via the terraform-registry MCP
and a terraform providers schema -json pull against the real provider binary):
tagsis a genuine, schema-confirmedSet of Stringonfabric_environmentβ a set offabric_tagGUIDs, not a key/value map. This directly contradicts this module suite's naming convention' blanket "notagsargument anywhere in this provider's schema" framing and this module suite's tags-naming rule's original "notagsvariable is ever added" instruction β both predate this discovery. Never pass a literal string; only GUIDs fromtf-mod-fabric-tag.workspace_id,environment_id, andpublication_statusare allForceNewonfabric_spark_environment_settingsβ changingpublication_statusbetween"Published"and"Staging"after creation destroys and recreates the settings record, it is never an in-place update.- Every nested structure on both resources β
timeouts,dynamic_executor_allocation,poolβ is a Terraform Plugin Framework Attribute, not a classic block. Confirmed empirically viaterraform providers schema -jsonagainst the real v1.12.0 binary: attemptingdynamic "timeouts" { content {... } }failsterraform validatewithUnsupported block type. All of these are assigned directly with=as object literals inmain.tf. - The
dynamic_executor_allocation.min_executors/.max_executorspairing is enforced independently ofenabled. This module's original scaffold assumed both fields were required whenenabled = trueand forbidden whenenabled = false. A liveterraform validateagainst the real v1.12.0 provider binary shows the provider's actual attribute validator only enforces "both set or both omitted" βenabled = truewith nomin_executors/max_executorsvalidates cleanly, as doesenabled = falsewith both set.variables.tfmodels only the confirmed constraint. driver_cores/executor_cores/driver_memory/executor_memoryare closed enums (4/8/16/32/64and"28g"/"56g"/"112g"/"224g"/"400g"respectively), but the provider does not statically cross-validate that a chosen core/memory pair is a legal combination for the underlying Spark pool β an invalid pairing can still surface only as an API-side error atapply.pool.nameandpool.typeare required together by the provider's own schema β this module modelspoolas an object with both fields required (not independently optional), so the type system enforces the pairing for free; there is no dedicatedvalidation{}needed for it.properties.publish_detailsis read-only and reflects the state of the last publish attempt (Cancelled/Cancelling/Failed/Running/Success/Waiting) β do not treat aWaiting/Runningstate surfaced inpublish_stateas a plan-time failure.- No delegated-auth exception applies to either resource as of v1.12.0 β both provider docs explicitly state "This resource supports Service Principal authentication." Reconfirm at authoring time per this module suite's authentication-model convention.
- Fabric Workspace Contributor (minimum) on the target workspace β required to create, update, and
delete an Environment item and its Spark settings sub-resource.
<VERIFY: confirm the exact minimum Fabric workspace role against Microsoft Learn's current Fabric permissions model β Fabric RBAC is workspace-role-derived rather than a separate item-level ACL system, and role names/grants have shifted across recent Fabric admin-portal releases.> - Entra: the calling Service Principal must be enabled under the tenant's "Service principals can use Fabric APIs" setting (see this module suite's authentication-model convention) β a blanket Fabric-API prerequisite, not specific to Environment items.
- No delegated-auth-only APIs are known to apply to
fabric_environmentorfabric_spark_environment_settingsas of v1.12.0.
- The target workspace's backing capacity must be
Active(not paused) for Environment creation and publish operations to succeed reliably. - Tenant-level SPN/MSI Fabric API access must already be enabled in the Fabric Admin Portal (see
Β§ Required Permissions above) β this module cannot enable it and cannot detect that it's missing from
a
plan. - Environment "publish" is an asynchronous, potentially long-running operation β a caller applying
spark_settingschanges should expect a publish cycle; generousspark_settings_timeoutsare recommended (see Β§ Architecture Notes).
tf-mod-fabric-environment/
βββ providers.tf # required_providers block (fabric ~> 1.12.0); no provider {} block
βββ variables.tf # display_name, workspace_id, description, folder_id, tags, spark_settings, timeouts, spark_settings_timeouts
βββ main.tf # fabric_environment.this (keystone) + fabric_spark_environment_settings.this (count-based singleton)
βββ outputs.tf # id, display_name, spark_environment_settings_id, publish_state
βββ README.md # this file
βββ SCOPE.md # cross-module contract (written before the .tf files were authored, per the composite-module rule)
βββ examples/
βββ basic/ # Environment only, no spark_settings
βββ complete/ # Environment + full spark_settings + folder + tags
# Caller's root module configures the fabric provider/auth β never this module.
provider "fabric" {
# tenant_id / client_id / client_secret, or use_oidc / use_msi, per environment
}
module "env_claims_default" {
source = "git::https://github.com/microsoftexpert/tf-mod-fabric-environment.git?ref=v1.0.0"
display_name = "env-claims-default-dev"
workspace_id = module.workspace_claims.id
}Consumes
| Input | Type | Source module |
|---|---|---|
workspace_id |
string |
tf-mod-fabric-workspace |
folder_id |
string (optional) |
tf-mod-fabric-folder |
tags (tag IDs) |
set(string) (optional) |
tf-mod-fabric-tag |
spark_settings.pool.name/.type |
string (optional, informational) |
tf-mod-fabric-spark-custom-pool |
Emits
| Output | Description | Consumed by |
|---|---|---|
id |
Environment item GUID | Any item module attaching a default Spark environment |
display_name |
Environment display name | Informational |
spark_environment_settings_id |
Settings child GUID, null if unset |
Informational |
publish_state |
Computed properties.publish_details.state |
Informational / pipeline gating |
1 Β· Minimal Environment β no Spark settings
The safe "empty call": no fabric_spark_environment_settings record is created.
module "env_claims_default" {
source = "git::https://github.com/microsoftexpert/tf-mod-fabric-environment.git?ref=v1.0.0"
display_name = "env-claims-default-dev"
workspace_id = module.workspace_claims.id
}βΉοΈ This Environment runs under default Fabric-managed Spark compute.
2 Β· Environment with a description and explicit folder placement
module "env_claims_raw" {
source = "git::https://github.com/microsoftexpert/tf-mod-fabric-environment.git?ref=v1.0.0"
display_name = "env-claims-raw-dev"
description = "Default-compute Environment for raw-zone Claims notebooks."
workspace_id = module.workspace_claims.id
folder_id = module.folder_raw.id
}π‘
folder_iddefaults tonull(workspace root) per this module suite's conventions Β§ Secure-by-default β supply it explicitly to nest under a governed folder hierarchy.
3 Β· Environment with tags from tf-mod-fabric-tag
module "env_claims_tagged" {
source = "git::https://github.com/microsoftexpert/tf-mod-fabric-environment.git?ref=v1.0.0"
display_name = "env-claims-tagged-prod"
workspace_id = module.workspace_claims.id
tags = [
module.tag_data_classification_internal.id
]
}π
tagsonfabric_environmentis a Set-of-String offabric_tagGUIDs β never a literal string.
4 Β· Minimal Spark settings (Published, no sizing)
module "env_claims_spark_minimal" {
source = "git::https://github.com/microsoftexpert/tf-mod-fabric-environment.git?ref=v1.0.0"
display_name = "env-claims-spark-minimal-dev"
workspace_id = module.workspace_claims.id
spark_settings = {
publication_status = "Published"
}
}5 Β· Small driver/executor sizing (4 cores / 28g)
module "env_claims_spark_small" {
source = "git::https://github.com/microsoftexpert/tf-mod-fabric-environment.git?ref=v1.0.0"
display_name = "env-claims-spark-small-dev"
workspace_id = module.workspace_claims.id
spark_settings = {
publication_status = "Published"
driver_cores = 4
driver_memory = "28g"
executor_cores = 4
executor_memory = "28g"
}
}6 Β· Large driver/executor sizing (64 cores / 400g)
module "env_claims_spark_large" {
source = "git::https://github.com/microsoftexpert/tf-mod-fabric-environment.git?ref=v1.0.0"
display_name = "env-claims-spark-large-prod"
workspace_id = module.workspace_claims.id
spark_settings = {
publication_status = "Published"
driver_cores = 64
driver_memory = "400g"
executor_cores = 64
executor_memory = "400g"
}
}
β οΈ The provider does not statically validate that a core/memory pair is a legal combination for the underlying capacity SKU β an invalid pairing surfaces only as an API-side error atapply.
7 Β· Dynamic executor allocation enabled
module "env_claims_spark_dynamic" {
source = "git::https://github.com/microsoftexpert/tf-mod-fabric-environment.git?ref=v1.0.0"
display_name = "env-claims-spark-dynamic-prod"
workspace_id = module.workspace_claims.id
spark_settings = {
publication_status = "Published"
dynamic_executor_allocation = {
enabled = true
min_executors = 2
max_executors = 8
}
}
}8 Β· Static allocation (dynamic_executor_allocation omitted)
module "env_claims_spark_static" {
source = "git::https://github.com/microsoftexpert/tf-mod-fabric-environment.git?ref=v1.0.0"
display_name = "env-claims-spark-static-prod"
workspace_id = module.workspace_claims.id
spark_settings = {
publication_status = "Published"
driver_cores = 8
executor_cores = 8
}
}βΉοΈ Omitting
dynamic_executor_allocationentirely leaves executor allocation under the provider's own computed default β this module never invents a default for a field the provider itself leaves computed.
9 Β· Workspace-scoped custom pool by name/type
module "env_claims_spark_pool_workspace" {
source = "git::https://github.com/microsoftexpert/tf-mod-fabric-environment.git?ref=v1.0.0"
display_name = "env-claims-spark-pool-ws-prod"
workspace_id = module.workspace_claims.id
spark_settings = {
publication_status = "Published"
pool = {
name = module.spark_custom_pool_claims.name
type = "Workspace"
}
}
}10 Β· Capacity-scoped starter pool
module "env_claims_spark_pool_capacity" {
source = "git::https://github.com/microsoftexpert/tf-mod-fabric-environment.git?ref=v1.0.0"
display_name = "env-claims-spark-pool-cap-prod"
workspace_id = module.workspace_claims.id
spark_settings = {
publication_status = "Published"
pool = {
name = "Starter Pool"
type = "Capacity"
}
}
}11 Β· Staging publication_status
module "env_claims_spark_staging" {
source = "git::https://github.com/microsoftexpert/tf-mod-fabric-environment.git?ref=v1.0.0"
display_name = "env-claims-spark-staging-dev"
workspace_id = module.workspace_claims.id
spark_settings = {
publication_status = "Staging"
}
}π
publication_statusisForceNewβ flipping this value on a later apply destroys and recreates the settings record rather than updating it in place.
12 Β· Published publication_status (contrast pair to #11)
module "env_claims_spark_published" {
source = "git::https://github.com/microsoftexpert/tf-mod-fabric-environment.git?ref=v1.0.0"
display_name = "env-claims-spark-published-prod"
workspace_id = module.workspace_claims.id
spark_settings = {
publication_status = "Published"
}
}13 Β· Spark properties
module "env_claims_spark_properties" {
source = "git::https://github.com/microsoftexpert/tf-mod-fabric-environment.git?ref=v1.0.0"
display_name = "env-claims-spark-props-prod"
workspace_id = module.workspace_claims.id
spark_settings = {
publication_status = "Published"
spark_properties = {
"spark.acls.enable" = "true"
"spark.sql.shuffle.partitions" = "200"
}
}
}π Every key must start with
"spark.", contain no whitespace, and not end with"."β enforced by avalidation{}block atterraform validatetime.
14 Β· Custom spark_settings_timeouts for a long-running publish
module "env_claims_spark_long_publish" {
source = "git::https://github.com/microsoftexpert/tf-mod-fabric-environment.git?ref=v1.0.0"
display_name = "env-claims-spark-longpub-prod"
workspace_id = module.workspace_claims.id
spark_settings = {
publication_status = "Published"
driver_cores = 16
executor_cores = 16
}
spark_settings_timeouts = {
create = "45m"
update = "45m"
}
}15 Β· ποΈ End-to-end composition
Wires tf-mod-fabric-workspace, tf-mod-fabric-folder, tf-mod-fabric-tag, and
tf-mod-fabric-spark-custom-pool outputs into this module's inputs.
module "workspace_claims" {
source = "git::https://github.com/microsoftexpert/tf-mod-fabric-workspace.git?ref=v1.0.0"
display_name = "ws-claims-bronze-prod"
capacity_id = azurerm_fabric_capacity.claims.id
}
module "folder_curated" {
source = "git::https://github.com/microsoftexpert/tf-mod-fabric-folder.git?ref=v1.0.0"
display_name = "curated"
workspace_id = module.workspace_claims.id
}
module "tag_data_classification_internal" {
source = "git::https://github.com/microsoftexpert/tf-mod-fabric-tag.git?ref=v1.0.0"
display_name = "data-classification:internal"
}
module "spark_custom_pool_claims" {
source = "git::https://github.com/microsoftexpert/tf-mod-fabric-spark-custom-pool.git?ref=v1.0.0"
display_name = "pool-claims-spark-prod"
workspace_id = module.workspace_claims.id
}
module "env_claims_spark_prod" {
source = "git::https://github.com/microsoftexpert/tf-mod-fabric-environment.git?ref=v1.0.0"
display_name = "env-claims-spark-prod"
description = "Spark-tuned Environment for Claims production data-engineering workloads."
workspace_id = module.workspace_claims.id
folder_id = module.folder_curated.id
tags = [
module.tag_data_classification_internal.id
]
spark_settings = {
publication_status = "Published"
driver_cores = 8
driver_memory = "56g"
executor_cores = 8
executor_memory = "56g"
dynamic_executor_allocation = {
enabled = true
min_executors = 2
max_executors = 6
}
pool = {
name = module.spark_custom_pool_claims.name
type = "Workspace"
}
spark_properties = {
"spark.acls.enable" = "true"
}
}
spark_settings_timeouts = {
create = "30m"
update = "30m"
}
}Summary: display_name (required), workspace_id (required), description/folder_id (optional,
default null), tags (optional, default []), spark_settings (optional, default null),
timeouts/spark_settings_timeouts (optional, default null).
Full object schemas
variable "display_name" {
type = string
}
variable "workspace_id" {
type = string
}
variable "description" {
type = string
default = null
}
variable "folder_id" {
type = string
default = null
}
variable "tags" {
type = set(string)
default = []
}
variable "spark_settings" {
type = object({
publication_status = string
driver_cores = optional(number)
driver_memory = optional(string)
executor_cores = optional(number)
executor_memory = optional(string)
runtime_version = optional(string)
dynamic_executor_allocation = optional(object({
enabled = optional(bool)
min_executors = optional(number)
max_executors = optional(number)
}))
pool = optional(object({
name = string
type = string
}))
spark_properties = optional(map(string), {})
})
default = null
}
variable "timeouts" {
type = object({
create = optional(string)
read = optional(string)
update = optional(string)
delete = optional(string)
})
default = null
}
variable "spark_settings_timeouts" {
type = object({
create = optional(string)
read = optional(string)
update = optional(string)
delete = optional(string)
})
default = null
}| Output | Description | Sensitive/Conditional |
|---|---|---|
id |
Environment item GUID (primary output) | β |
display_name |
Environment display name, echoed back | β |
spark_environment_settings_id |
Settings child GUID | Conditional β null when spark_settings was not supplied |
publish_state |
Computed publish state from the last publish attempt | β |
- The count-based conditional singleton.
fabric_spark_environment_settings.thisusescount = var.spark_settings != null ? 1: 0β a deliberate, documented exception to "alwaysfor_each, nevercount" (this module suite's code-standard convention). The rule againstcountexists to protect keyed collections from reordering when one entry is added/removed; a true 0-or-1 singleton with no plural relationship carries no such risk, socountis the correct, minimal tool. workspace_id/environment_idare always derived, never caller-supplied insidespark_settings.main.tfsetsworkspace_id = var.workspace_idandenvironment_id = fabric_environment.this.iddirectly β thespark_settingsobject intentionally has noworkspace_id/environment_idfields, so the settings record can never drift to reference a different workspace/environment than the one this module just created.- Nested Attributes, not blocks.
timeouts,dynamic_executor_allocation, andpoolare all Terraform Plugin Framework Attributes on the live v1.12.0 schema (confirmed viaterraform providers schema -json), somain.tfassigns each with a plain=object expression (timeouts = var.timeouts != null ? {... }: null), never adynamic "..." { content {... } }block β the latter failsterraform validatewithUnsupported block typeagainst this provider. trythroughoutspark_settingsreads. Because the settings resource only exists whencount = 1, every field read fromvar.spark_settingsinsidefabric_spark_environment_settings.thisis wrapped intry(..., null)so a nullvar.spark_settingsnever throws a "attempt to get attribute from null value" error during evaluation.- The min/max executor pairing is enforced independently of
enabled. See Β§ Provider / Versions above βvariables.tf'svalidation{}blocks model only the confirmed "both set or both omitted" constraint, not a dependency onenabled's value.
| Concern | Safe default | Opt-out |
|---|---|---|
| Spark compute settings | No fabric_spark_environment_settings record (spark_settings = null) |
Supply a spark_settings object |
| Folder placement | Workspace root (folder_id = null) |
Supply folder_id |
| Tags | Empty (tags = []) |
Supply fabric_tag GUIDs |
| Item definition tokens (n/a to this resource) | β | β |
| Spark property substitution | No properties (spark_properties = {}) |
Supply spark_properties |
cd C:\GitHubCode\newfabricmodules\tf-mod-fabric-environment
terraform init -backend=false
terraform validate
terraform fmt -checkPin ?ref=v1.0.0 in every consuming root module β never a branch. This module is plan-only; a human
applies from a reviewed CI pipeline.
terraform validate/fmt -checkcatch: enum typos (publication_status,driver_cores,pool.type), the conditionaldynamic_executor_allocation.min_executors/.max_executorspairing, malformedspark_propertieskeys, and missing required fields β all at zero cost, before any Fabric API call.- Only a live
plan/applyagainst a real tenant exercises: whether the chosen core/memory pairing is actually legal for the underlying capacity SKU, actual publish-cycle timing (properties.publish_details.state), and whetherruntime_versionis currently supported.
spark_environment_settings_id = null
environment_id = "9f2b1a3e-4c5d-4e6f-8a9b-0c1d2e3f4a5b"
publish_state = null
# after applying spark_settings:
spark_environment_settings_id = "1a2b3c4d-5e6f-7a8b-9c0d-1e2f3a4b5c6d"
publish_state = "Success"
| Symptom | Cause | Fix |
|---|---|---|
publish_state stuck at Waiting/Running |
Environment publish is asynchronous | Wait and re-plan/refresh; this is not a Terraform-level failure |
| Unexpected replace of the settings record | publication_status, workspace_id, or environment_id changed |
These are ForceNew on the provider β expect destroy/recreate, not an in-place update |
Core/memory combination rejected at apply only |
The provider does not statically cross-validate legal pairings | Confirm the pairing against current Fabric capacity SKU documentation before applying |
"Unsupported block type" on timeouts/pool/dynamic_executor_allocation if hand-editing main.tf |
These are Attributes, not blocks, on this provider | Assign directly with =, never dynamic "..." { content {... } } |
spark_properties key rejected |
Key doesn't start with "spark.", contains whitespace, or ends with "." |
Fix the key per the validation{} error message |
fabric_environmentandfabric_spark_environment_settingsprovider docs (microsoft/fabricv1.12.0)- Sibling modules:
tf-mod-fabric-workspace,tf-mod-fabric-folder,tf-mod-fabric-tag,tf-mod-fabric-spark-custom-pool - This module's
SCOPE.md