Skip to content

microsoftexpert/tf-mod-snowflake-stream

Folders and files

NameName
Last commit message
Last commit date

Latest commit

Β 

History

3 Commits
Β 
Β 
Β 
Β 
Β 
Β 
Β 
Β 
Β 
Β 
Β 
Β 
Β 
Β 
Β 
Β 

Repository files navigation

❄️ Snowflake Stream (on Table) Terraform Module

Manages a single Snowflake change-data-capture stream on a table (snowflake_stream_on_table) against the snowflakedb/snowflake provider, pinned ~> 2.17.

Terraform Snowflake Provider Module Version Module Type Resources Posture


🧩 Overview

  • ❄️ Manages a single snowflake_stream_on_table object β€” one keystone resource, no children, no cross-object grants.
  • πŸ”‘ Encodes both of Snowflake's AT/BEFORE Time Travel exclusivity rules (exactly one sub-field per clause; at most one of at/before) as terraform validate-time validation {} blocks β€” neither rule is enforced by the provider's own schema, so a malformed call would otherwise fail at apply time against a live Snowflake account instead.
  • ⚠️ Documents, loudly, the single most important gotcha in this entire module library: five fields on this resource (table, append_only, at, before, show_initial_rows) β€” plus the computed stale attribute β€” cannot be altered in place on the Snowflake side. Changing any of them causes the provider to silently drop and recreate the stream, and terraform plan renders this as an ordinary in-place ~ update, never a -/+ destroy and re-create.
  • 🧾 Emits fully_qualified_name as the safe cross-reference for feeding a downstream task's SQL text via plain string interpolation β€” not a typed input/output wire, since Snowflake tasks execute arbitrary SQL.

πŸ’‘ Why it matters: a stream is only useful if its accumulated, unconsumed change data survives until something reads it. This resource's provider-documented habit of recreating itself under an innocuous-looking ~ update is exactly the kind of drift that destroys change data silently in a regulated pipeline β€” modeling the gotcha as loudly as possible in variables.tf, this README, and SCOPE.md is the whole point of this module's design.


❀️ Support this project

If these Terraform modules have been helpful to you or your organization, I'd appreciate your support in any of the following ways:

Whether it's a star, a professional connection, or a coffee, every gesture helps keep these modules actively maintained and continually improving. Thank you for being part of the community!


πŸ—ΊοΈ Where this fits

flowchart LR
 database["tf-mod-snowflake-database"]
 schema["tf-mod-snowflake-schema"]
 table_ext["caller-supplied table fully_qualified_name β€” no sibling module yet"]
 thismod["tf-mod-snowflake-stream"]
 keystone[["snowflake_stream_on_table.this"]]
 task["tf-mod-snowflake-task"]

 database -->|"name feeds database"| thismod
 schema -->|"name feeds schema"| thismod
 table_ext -->|"fully_qualified_name feeds table"| thismod
 thismod -->|"creates"| keystone
 keystone -->|"fully_qualified_name string interpolation feeds sql_statement"| task

 style thismod fill:#29B5E8,color:#ffffff
 style keystone fill:#11567F,color:#ffffff
 style database fill:#ECEFF1,color:#000000
 style schema fill:#ECEFF1,color:#000000
 style table_ext fill:#ECEFF1,color:#000000
 style task fill:#ECEFF1,color:#000000
Loading

This module has a real sibling family, so the family DAG is not dropped. tf-mod-snowflake-database and tf-mod-snowflake-schema (both already authored in this catalog) feed this module's database/schema inputs with their bare name outputs β€” not fully_qualified_name, since the live provider schema documents both as plain bare identifiers. table is rendered as a light-neutral external, unmodeled node: it is a plain caller-supplied fully_qualified_name string today, since no tf-mod-snowflake-table or tf-mod-snowflake-stage module exists in this catalog yet. tf-mod-snowflake-task is shown as the natural downstream consumer of this module's fully_qualified_name output β€” tf-mod-snowflake-task has not yet been authored in this catalog (only its seed prompt files exist as of this writing), so this edge documents the intended future consumption pattern, not a live, exercised cross-module contract today. That edge is deliberately a plain string-interpolation relationship (e.g. SELECT * FROM $${module.orders_stream.fully_qualified_name} inside a task's sql_statement text), not a typed Terraform reference, because Snowflake tasks execute arbitrary SQL text rather than accepting structured inputs.


🧬 What this builds

flowchart TB
 subgraph inputs["Inputs"]
 database["database"]
 schema["schema"]
 name["name (required)"]
 table["table (required)"]
 append_only["append_only"]
 show_initial_rows["show_initial_rows"]
 at["at (Time Travel)"]
 before["before (Time Travel)"]
 copy_grants["copy_grants"]
 comment["comment"]
 end

 keystone[["snowflake_stream_on_table.this"]]

 subgraph outputs["Outputs"]
 fqn["fully_qualified_name"]
 outname["name"]
 id["id"]
 stale["stale"]
 stream_type["stream_type"]
 end

 inputs -->|"renders"| keystone
 keystone -->|"emits"| outputs

 style keystone fill:#11567F,color:#ffffff
 style inputs fill:#ECEFF1,color:#000000
 style outputs fill:#ECEFF1,color:#000000
Loading

Resource inventory:

Resource Count Notes
snowflake_stream_on_table.this 1 (keystone) No for_each children β€” standalone module. at/before are rendered as dynamic blocks on this single resource, not separate resources.

βœ… Provider / Versions

Requirement Value
Terraform >= 1.12.0
snowflakedb/snowflake provider ~> 2.17
Provider block Not present β€” the caller's root module configures the provider (account identifier, authentication) and passes any role alias in

Schema notes that bite:

  • ⚠️ The single biggest gotcha in this module library. table, append_only, at, before, show_initial_rows, and the computed stale attribute cannot be ALTERed on the Snowflake side. A change to any of them is applied by the provider as a silent drop/recreate of the stream β€” and terraform plan shows this as an in-place ~ update, not a -/+ destroy and re-create. There is no plan-time visual signal. Treat any diff touching these fields as destructive.
  • at/before mutual exclusivity is not provider-enforced. The live schema places no constraint on the nested at/before blocks beyond their four optional sub-fields each β€” this module's own validation {} blocks are the only thing standing between a malformed call and a live Snowflake SQL error at apply time.
  • append_only/show_initial_rows are STRING fields with a "default" sentinel, not real booleans on the provider's own schema β€” this module's bool variables are converted to "true"/"false" strings in main.tf; the provider's "default" sentinel is never surfaced.
  • copy_grants has no effect on first creation β€” it only matters at one of the implicit recreate events described above.
  • Import can leave persistent diffs on boolean fields. The provider's own docs recommend the IMPORT_BOOLEAN_DEFAULT experimental feature (a provider-configuration concern, set via preview_features_enabled) and reimporting if append_only/show_initial_rows show a persistent diff after terraform import.
  • database/schema/name are bare identifiers; table is not. This module validates that database, schema, and name contain none of |, ., ". table is deliberately exempt from that same check, since a correct fully_qualified_name value legitimately contains quotes and dots ("DB"."SCHEMA"."TABLE").
  • Case sensitivity. All identifier-shaped inputs are passed through as-is; Snowflake case-folds an unquoted identifier to upper case. This is a common source of "already exists" false negatives on import if the caller assumes the value is stored verbatim.

πŸ”‘ Required Snowflake Privileges

  • CREATE STREAM on the containing schema β€” required to create the keystone snowflake_stream_on_table.this object.
  • SELECT on the source table (var.table) β€” Snowflake requires the executing role to have read access on the table a stream monitors; without it, CREATE STREAM fails at apply time.
  • SELECT on the referenced stream β€” only required when at.stream or before.stream names another existing stream as the Time Travel reference point.
  • Subsequent updates (ALTER STREAM β€” limited to comment per the provider's own gotcha) and drops require ownership of the object or an equivalent privileged role β€” normally satisfied automatically by whichever role's session created it.

Snowflake Prerequisites

  • No specific Snowflake account edition requirement β€” streams are available on every edition.
  • No preview_features_enabled flag required β€” snowflake_stream_on_table is documented by the provider as Stable (GA), not preview-gated.
  • The executing role must already hold CREATE STREAM on the target schema and SELECT on the source table (see above) β€” provisioned by the caller's root module/provider alias; this module cannot grant itself the privileges it needs to run.
  • The containing database and schema (var.database/var.schema) must already exist β€” typically created by tf-mod-snowflake-database/tf-mod-snowflake-schema instances applied before, or in the same run as, this module.
  • The source table (var.table) must already exist before this module applies. No tf-mod-snowflake-table module exists in this catalog yet, so today the table is expected to be created out of band (or by a future sibling module) and referenced here as a plain string.
  • If at.stream/before.stream is populated, the referenced stream must already exist in the account before this module applies β€” this module does not create, look up, or validate that referenced stream's existence.

πŸ“ Module Structure

tf-mod-snowflake-stream/
β”œβ”€β”€ providers.tf # required_providers + required_version β€” no provider {} block
β”œβ”€β”€ variables.tf # database/schema/name/table, append_only, show_initial_rows, at, before, copy_grants, comment
β”œβ”€β”€ main.tf # the keystone snowflake_stream_on_table.this β€” no children
β”œβ”€β”€ outputs.tf # fully_qualified_name, name, id, stale, stream_type
β”œβ”€β”€ README.md # this file
β”œβ”€β”€ SCOPE.md # lightweight standalone spec β€” privileges, prerequisites, gotchas
└── examples/ # runnable example call sites

βš™οΈ Quick Start

module "orders_stream" {
  source = "git::https://github.com/microsoftexpert/tf-mod-snowflake-stream.git?ref=v1.0.0"

  database = "SALES_DB"
  schema   = "RAW"
  name     = "ORDERS_STREAM"
  table    = "\"SALES_DB\".\"RAW\".\"ORDERS\""
}

The caller's root module configures the snowflake provider (account identifier, authentication, and any role alias β€” see this suite's Authentication model convention). This module never declares account_name, organization_name, user, role, or any credential-shaped variable.


πŸ”Œ Cross-Module Contract

Consumes:

Input Type Source module
database string (bare identifier) tf-mod-snowflake-database's name output
schema string (bare identifier) tf-mod-snowflake-schema's name output
table string (fully_qualified_name-shaped) Caller-supplied literal today β€” no table-owning sibling module exists yet in this catalog
at.stream / before.stream (optional) string Another tf-mod-snowflake-stream instance's name/identifier, if referencing an existing stream as the Time Travel point

Emits:

Output Description Consumed by
fully_qualified_name The stream's fully-qualified identifier A future tf-mod-snowflake-task's sql_statement input, via plain string interpolation β€” NOT a structured wire (see Β§4)
name Bare name of the stream, as passed to var.name Documentation/display only
id Terraform resource ID (import identifier) Documentation/display only
stale Computed staleness flag Downstream pipeline-health monitoring
stream_type Computed stream type (informational) Downstream pipeline-health monitoring

πŸ“š Example Library

1 Β· Minimal call β€” secure/provider defaults

ℹ️ append_only = false, show_initial_rows = false, copy_grants = false are all the module's (and the provider's own) defaults β€” a full stream capturing inserts, updates, and deletes, with no historical-row replay and no grant preservation on first creation (grant preservation is irrelevant on first creation regardless).

module "orders_stream" {
  source = "git::https://github.com/microsoftexpert/tf-mod-snowflake-stream.git?ref=v1.0.0"

  database = "SALES_DB"
  schema   = "RAW"
  name     = "ORDERS_STREAM"
  table    = "\"SALES_DB\".\"RAW\".\"ORDERS\""
}
2 Β· Append-only stream for insert-only CDC

πŸ’‘ Use when downstream consumers only ever care about new rows (e.g. append-only event ingestion) and should never see synthetic update/delete records.

module "events_append_only_stream" {
  source = "git::https://github.com/microsoftexpert/tf-mod-snowflake-stream.git?ref=v1.0.0"

  database    = "EVENTS_DB"
  schema      = "RAW"
  name        = "EVENTS_APPEND_ONLY_STREAM"
  table       = "\"EVENTS_DB\".\"RAW\".\"CLICKSTREAM\""
  append_only = true
}
3 Β· `at.offset` β€” relative Time Travel point
module "orders_stream_at_offset" {
  source = "git::https://github.com/microsoftexpert/tf-mod-snowflake-stream.git?ref=v1.0.0"

  database = "SALES_DB"
  schema   = "RAW"
  name     = "ORDERS_STREAM_AT_OFFSET"
  table    = "\"SALES_DB\".\"RAW\".\"ORDERS\""

  at = {
    offset = "-3600"
  }
}
4 Β· `before.statement` β€” reference a query ID
module "orders_stream_before_statement" {
  source = "git::https://github.com/microsoftexpert/tf-mod-snowflake-stream.git?ref=v1.0.0"

  database = "SALES_DB"
  schema   = "RAW"
  name     = "ORDERS_STREAM_BEFORE_STATEMENT"
  table    = "\"SALES_DB\".\"RAW\".\"ORDERS\""

  before = {
    statement = "8e5d0ca9-005e-44e6-b858-a8f5b37c5726"
  }
}
5 Β· `at.timestamp` β€” exact Time Travel point
module "orders_stream_at_timestamp" {
  source = "git::https://github.com/microsoftexpert/tf-mod-snowflake-stream.git?ref=v1.0.0"

  database = "SALES_DB"
  schema   = "RAW"
  name     = "ORDERS_STREAM_AT_TIMESTAMP"
  table    = "\"SALES_DB\".\"RAW\".\"ORDERS\""

  at = {
    timestamp = "TO_TIMESTAMP_LTZ('2026-07-01 00:00:00')"
  }
}
6 Β· `before.offset` β€” relative Time Travel point on BEFORE
module "orders_stream_before_offset" {
  source = "git::https://github.com/microsoftexpert/tf-mod-snowflake-stream.git?ref=v1.0.0"

  database = "SALES_DB"
  schema   = "RAW"
  name     = "ORDERS_STREAM_BEFORE_OFFSET"
  table    = "\"SALES_DB\".\"RAW\".\"ORDERS\""

  before = {
    offset = "-1800"
  }
}
7 Β· `at.stream` β€” referencing another existing stream

πŸ”‘ Requires SELECT on the referenced stream, in addition to the base CREATE STREAM/SELECT privileges β€” see πŸ”‘ Required Snowflake Privileges above.

module "orders_stream_at_stream" {
  source = "git::https://github.com/microsoftexpert/tf-mod-snowflake-stream.git?ref=v1.0.0"

  database = "SALES_DB"
  schema   = "RAW"
  name     = "ORDERS_STREAM_AT_STREAM"
  table    = "\"SALES_DB\".\"RAW\".\"ORDERS\""

  at = {
    stream = module.orders_stream.name
  }
}
8 Β· ⚠️ `copy_grants = true` β€” preserving grants across an implicit recreate

⚠️ Set this explicitly whenever a production stream's grants must survive one of the implicit, plan-invisible recreates triggered by changing table/append_only/at/before/ show_initial_rows, or by Snowflake marking the stream stale. Leaving this at the module's default false means every one of those recreates silently drops all grants on the stream.

module "orders_stream_copy_grants" {
  source = "git::https://github.com/microsoftexpert/tf-mod-snowflake-stream.git?ref=v1.0.0"

  database    = "SALES_DB"
  schema      = "RAW"
  name        = "ORDERS_STREAM_GRANT_SAFE"
  table       = "\"SALES_DB\".\"RAW\".\"ORDERS\""
  copy_grants = true
}
9 Β· `show_initial_rows = true` β€” replay existing rows as inserts
module "orders_stream_initial_rows" {
  source = "git::https://github.com/microsoftexpert/tf-mod-snowflake-stream.git?ref=v1.0.0"

  database          = "SALES_DB"
  schema            = "RAW"
  name              = "ORDERS_STREAM_INITIAL_ROWS"
  table             = "\"SALES_DB\".\"RAW\".\"ORDERS\""
  show_initial_rows = true
}
10 Β· Stream with an explicit comment
module "orders_stream_documented" {
  source = "git::https://github.com/microsoftexpert/tf-mod-snowflake-stream.git?ref=v1.0.0"

  database = "SALES_DB"
  schema   = "RAW"
  name     = "ORDERS_STREAM_DOCUMENTED"
  table    = "\"SALES_DB\".\"RAW\".\"ORDERS\""
  comment  = "Feeds the nightly orders reconciliation task."
}
11 Β· Multiple streams at scale via a caller-level for_each

ℹ️ This module itself has no children (standalone), but a caller can still apply for_each at the module block to stamp out one stream per source table in a map β€” each instance is independently keyed, so adding or removing a table never disturbs its siblings.

variable "cdc_tables" {
  type = map(string)
  default = {
    orders    = "\"SALES_DB\".\"RAW\".\"ORDERS\""
    customers = "\"SALES_DB\".\"RAW\".\"CUSTOMERS\""
    invoices  = "\"SALES_DB\".\"RAW\".\"INVOICES\""
  }
}

module "cdc_streams" {
  source   = "git::https://github.com/microsoftexpert/tf-mod-snowflake-stream.git?ref=v1.0.0"
  for_each = var.cdc_tables

  database = "SALES_DB"
  schema   = "RAW"
  name     = "${upper(each.key)}_STREAM"
  table    = each.value
}
12 Β· ⚠️ Rejected shape β€” `at` and `before` set simultaneously

⚠️ This example is deliberately not runnable HCL. A validation failure cannot be shown as a working example, so this block is prose plus the invalid call, commented out.

A stream may reference at most one Time Travel point β€” at OR before, never both:

# module "invalid_at_and_before" {
# source = "git::https://github.com/microsoftexpert/tf-mod-snowflake-stream.git?ref=v1.0.0"
#
# database = "SALES_DB"
# schema = "RAW"
# name = "INVALID_AT_AND_BEFORE"
# table = "\"SALES_DB\".\"RAW\".\"ORDERS\""
#
# at = { offset = "-3600" }
# before = { offset = "-1800" } # at and before both set β€” rejected
# }

terraform validate fails this call site with:

Error: Invalid value for variable

 on variables.tf line 281, in variable "before":
 281: validation {

at and before are mutually exclusive β€” a stream may reference at most one Time Travel point (AT
or BEFORE), not both simultaneously.
13 Β· ⚠️ Rejected shape β€” more than one `at` sub-field set

⚠️ Also not runnable HCL β€” illustrates the internal mutual-exclusivity rule within a single at/before block.

# module "invalid_at_two_subfields" {
# source = "git::https://github.com/microsoftexpert/tf-mod-snowflake-stream.git?ref=v1.0.0"
#
# database = "SALES_DB"
# schema = "RAW"
# name = "INVALID_AT_TWO_SUBFIELDS"
# table = "\"SALES_DB\".\"RAW\".\"ORDERS\""
#
# at = {
# offset = "-3600"
# statement = "8e5d0ca9-005e-44e6-b858-a8f5b37c5726" # two sub-fields set β€” rejected
# }
# }

terraform validate fails with: at must set exactly one of offset, statement, stream, or timestamp when non-null (Snowflake's AT clause accepts exactly one Time Travel reference).

14 Β· πŸ—οΈ End-to-end composition β€” database, schema, stream, and a downstream task's SQL text

Wires tf-mod-snowflake-database's and tf-mod-snowflake-schema's name outputs into this module's database/schema inputs, and this module's fully_qualified_name output into a downstream task's sql_statement text via plain string interpolation.

module "sales_db" {
  source = "git::https://github.com/microsoftexpert/tf-mod-snowflake-database.git?ref=v1.0.0"

  name = "SALES_DB"
}

module "raw_schema" {
  source = "git::https://github.com/microsoftexpert/tf-mod-snowflake-schema.git?ref=v1.0.0"

  database = module.sales_db.name
  name     = "RAW"
}

module "orders_stream" {
  source = "git::https://github.com/microsoftexpert/tf-mod-snowflake-stream.git?ref=v1.0.0"

  database    = module.sales_db.name
  schema      = module.raw_schema.name
  name        = "ORDERS_STREAM"
  table       = "\"SALES_DB\".\"RAW\".\"ORDERS\""
  copy_grants = true
}

# ℹ️ tf-mod-snowflake-task has not yet been authored in this catalog (seed prompt files only as
# of this writing). Once it exists, its sql_statement input would consume this module's
# fully_qualified_name output as a plain string interpolated into arbitrary SQL text β€” NOT a
# structured Terraform reference, since Snowflake tasks execute SQL text, not typed inputs:
#
# module "orders_reconciliation_task" {
# source = "git::https://github.com/microsoftexpert/tf-mod-snowflake-task.git?ref=v1.0.0"
#
# database = module.sales_db.name
# schema = module.raw_schema.name
# name = "ORDERS_RECONCILIATION_TASK"
# sql_statement = "SELECT * FROM ${module.orders_stream.fully_qualified_name}"
# }

πŸ’‘ module.sales_db.name and module.raw_schema.name are the bare names, not fully_qualified_name β€” the live snowflake_stream_on_table schema documents database and schema as plain bare identifiers, unlike the table field, which does expect a fully_qualified_name-shaped value.


πŸ“₯ Inputs

Variable Type Default Required
database string β€” Yes
schema string β€” Yes
name string β€” Yes
table string β€” Yes
append_only bool false No
show_initial_rows bool false No
at object({ offset, statement, stream, timestamp }) null No
before object({ offset, statement, stream, timestamp }) null No
comment string null No
copy_grants bool false No
Full variable schemas
variable "database" {
  type = string
  # Required. Bare identifier (NOT fully_qualified_name) β€” source from
  # tf-mod-snowflake-database's `name` output. Avoid `|`, `.`, `"`.
}

variable "schema" {
  type = string
  # Required. Bare identifier β€” source from tf-mod-snowflake-schema's `name` output.
  # Avoid `|`, `.`, `"`.
}

variable "name" {
  type = string
  # Required. The stream's own identifier, unique within database + schema.
  # Avoid `|`, `.`, `"`.
}

variable "table" {
  type = string
  # Required. fully_qualified_name-shaped string for the source table (e.g.
  # snowflake_table.example.fully_qualified_name, or a plain literal since no
  # tf-mod-snowflake-table module exists yet). Non-empty check only β€” the `|`/`.`/`"`
  # exclusion does NOT apply here, since a real fully_qualified_name contains dots/quotes.
  # ⚠️ CRITICAL: cannot be changed in place β€” see README Β§15.
}

variable "append_only" {
  type    = bool
  default = false
  # Provider field is a STRING ("true"/"false"/"default"); this module converts the bool
  # in main.tf. ⚠️ CRITICAL: cannot be changed in place β€” see README Β§15.
}

variable "show_initial_rows" {
  type    = bool
  default = false
  # Same STRING-sentinel-bool quirk as append_only. ⚠️ CRITICAL: cannot be changed in
  # place β€” see README Β§15.
}

variable "at" {
  type = object({
    offset    = optional(string, null)
    statement = optional(string, null)
    stream    = optional(string, null)
    timestamp = optional(string, null)
  })
  default = null
  # Rendered as a `dynamic "at"` block. Exactly one sub-field must be set when non-null
  # (validation enforced). Mutually exclusive with `before` (validation enforced).
  # ⚠️ CRITICAL: cannot be changed in place β€” see README Β§15.
}

variable "before" {
  type = object({
    offset    = optional(string, null)
    statement = optional(string, null)
    stream    = optional(string, null)
    timestamp = optional(string, null)
  })
  default = null
  # Identical shape/rules to `at`. ⚠️ CRITICAL: cannot be changed in place β€” see README Β§15.
}

variable "comment" {
  type    = string
  default = null
  # The universal tail β€” free-text comment, no behavioral effect.
}

variable "copy_grants" {
  type    = bool
  default = false
  # Preserves grants across an implicit recreate (see README Β§15). No effect on first
  # creation. ⚠️ Set true explicitly for any production stream where grant continuity
  # matters.
}

🧾 Outputs

Output Description Sensitive / Conditional
fully_qualified_name The stream's fully-qualified identifier β€” the safe cross-reference; feeds a downstream task's SQL text via string interpolation Always present
name Bare name of the stream, as passed to var.name Always present
id Terraform resource ID (import identifier) Always present
stale Computed boolean β€” whether Snowflake considers the stream stale Always present; informational
stream_type Computed stream type (informational; exact legal value set not enumerated by the live schema) Always present; informational

🧠 Architecture Notes

  • ⚠️ Lead item β€” implicit, plan-invisible recreation. The live provider schema states, in its own words, that table, append_only, at, before, show_initial_rows, and the computed stale attribute "can not be ALTERed on Snowflake side... and a change on these fields means recreation of the resource... even though a change is marked as update, the resource is recreated." terraform plan never shows a -/+ destroy and re-create indicator for this class of change β€” only an ordinary ~ update. A caller must not rely on terraform plan alone to catch a stream-recreating change; review any diff touching these fields (or an unexpected stale = true in a prior terraform output) as if it were destructive, regardless of how the plan renders it.
  • at/before mutual exclusivity is enforced in variables.tf, not by the provider. Two rules are encoded as validation {} blocks: (1) within a single non-null at or before, exactly one of offset/statement/stream/timestamp may be set, mirroring Snowflake's own AT/BEFORE clause syntax; (2) at and before cannot both be non-null at once β€” a stream references at most one Time Travel point. Neither rule appears in the live schema itself, so without these blocks a violation would only surface as a Snowflake SQL error at apply time.
  • append_only/show_initial_rows bool-to-string conversion. The provider's underlying fields are STRINGs accepting "true"/"false"/"default". main.tf converts this module's real bool variables inline (var.append_only ? "true": "false"); the "default" sentinel is never surfaced as a caller option.
  • copy_grants interaction with the recreate gotcha. copy_grants only takes effect at one of the implicit recreate events above β€” it has no effect on first creation. A caller who wants a production stream's grants to survive a future table/append_only/at/before/ show_initial_rows change (or a stale-triggered recreate) must set copy_grants = true ahead of time; it cannot be applied retroactively after the recreate has already happened.
  • Catalog scope. snowflake_stream_on_view, snowflake_stream_on_external_table, and snowflake_stream_on_directory_table exist in the live provider (v2.18.0) but are out of scope for this module β€” each is a candidate for its own future standalone module, and this module is deliberately not genericized with a source_type discriminator to cover them.
  • dynamic blocks, not object-typed arguments. The live schema documents at/before as "Block List, Max: 1" β€” genuine nested configuration blocks, not flat object attributes β€” so main.tf renders each via a dynamic block gated on var.at/var.before being non-null, with try(..., null) wrapping every nested field access as cheap insurance.

🧱 Design Principles

Concern Secure default in this module Opt-out (caller must be explicit)
Object comment null β€” no default disclosure of intent in comment Caller sets comment explicitly
append_only false β€” a full stream capturing inserts, updates, and deletes (the provider's own effective default; also the more complete/safer choice for downstream consumers) Caller sets true explicitly for insert-only CDC
show_initial_rows false β€” historical rows are never surfaced as synthetic inserts Caller sets true explicitly
at / before Both null β€” no Time Travel reference point; the stream tracks changes from its own creation point forward Caller sets exactly one, with exactly one sub-field populated
copy_grants false, matching the provider's own default Caller sets true explicitly β€” recommended for any production stream where grant continuity across an implicit recreate matters
Grant scope N/A β€” this module creates no grant resources; cross-object grants are a tf-mod-snowflake-role-grant-bundle concern Not applicable

The empty call (database/schema/name/table only) produces a full stream that captures every insert, update, and delete on the source table, replays no historical rows, and has no Time Travel offset β€” consistent with this suite's "the empty call must produce the safe resource" rule. A caller has to type every additional argument to narrow the stream's behavior or preserve grants across a future recreate.


πŸš€ Runbook

cd C:\GitHubCode\newsnowflakemodules\tf-mod-snowflake-stream
terraform init -backend=false
terraform validate
terraform fmt -check

Pin the module source to ?ref=v1.0.0 β€” never a branch β€” in every caller. This is a plan-only authoring pipeline: no terraform plan or terraform apply is ever run as part of authoring this module. A human runs apply from a governed CI pipeline against a real Snowflake account.


πŸ§ͺ Testing

terraform validate and terraform fmt -check are the offline proof gate this module ships with:

  • What they cover: type correctness (every variable's object/scalar shape), the at/ before internal mutual-exclusivity rule, the at-vs-before mutual-exclusivity rule, and the table/database/schema/name identifier-character checks β€” all fire on terraform validate with no live Snowflake account required. HCL syntax and formatting are covered by fmt -check.
  • What only a real plan/apply exercises: whether the executing role actually holds CREATE STREAM on the schema and SELECT on the source table (privilege sufficiency), whether the source table (and any at.stream/before.stream reference) actually exists (object-existence dependencies this module deliberately does not check), and β€” most importantly β€” the implicit drop/recreate behavior on table/append_only/at/before/ show_initial_rows/stale, which is invisible to a single-configuration validate pass and only ever shows up (as an ordinary-looking ~ update) in a real plan against existing state.

πŸ’¬ Example Output

$ terraform output

fully_qualified_name = "\"SALES_DB\".\"RAW\".\"ORDERS_STREAM\""
name = "ORDERS_STREAM"
id = "\"SALES_DB\".\"RAW\".\"ORDERS_STREAM\""
stale = false
stream_type = "DEFAULT"

πŸ” Troubleshooting

Symptom Cause Fix
terraform validate fails with "at must set exactly one of offset, statement, stream, or timestamp" More than one (or the object was set but every sub-field left null) sub-field populated in at Set exactly one sub-field, or leave at = null
terraform validate fails with "before must set exactly one of..." Same rule, applied to before Set exactly one sub-field, or leave before = null
terraform validate fails with "at and before are mutually exclusive" Both at and before were set non-null Choose one Time Travel clause, not both
A plan touching table/append_only/at/before/show_initial_rows shows an ordinary ~ update, but the stream's change data is gone after apply Provider/Snowflake limitation β€” these fields cannot be ALTERed in place; the provider drops and recreates the stream without a -/+ plan indicator Expected behavior per the live schema; review any diff on these fields as destructive before applying, and set copy_grants = true ahead of time if grants must survive
stale = true appears in terraform output, and the next apply unexpectedly recreates the stream Snowflake's own staleness detection (offset fell outside the source table's data retention period) β€” this is not something this module or the caller's .tf changed Consume more frequently, or extend the source table's data retention; monitor the stale output rather than assuming a clean plan means the stream is healthy
Apply fails with a Snowflake authorization error on CREATE STREAM or reading the source table Executing role lacks CREATE STREAM on the schema or SELECT on the table Grant the missing privilege to the executing role, or apply through a provider alias configured with a role that already holds it
Persistent diff on append_only/show_initial_rows after terraform import Known provider import gap for STRING-sentinel-bool fields Enable the IMPORT_BOOLEAN_DEFAULT experimental feature (preview_features_enabled on the provider configuration) and reimport
at.stream/before.stream reference fails at apply with an authorization or "does not exist" error This module does not validate the referenced stream's existence or the executing role's SELECT privilege on it Confirm the referenced stream exists and the executing role has SELECT on it before applying

πŸ”— Related Docs

About

No description or website provided.

Topics

Resources

License

Stars

0 stars

Watchers

0 watching

Forks

Packages

 
 
 

Contributors

Languages