🟧 AWS AppFlow Terraform Module

Secure-by-default Amazon AppFlow data integration — connector profiles (Salesforce / Slack / ServiceNow / Zendesk / Redshift, deeply typed) and flows (S3 / Salesforce / Redshift / Zendesk source-and-destination, deeply typed), CMK encryption opt-in on both resources, and a sensitive-wrapped credentials surface so OAuth tokens and passwords never leak into plan output. Built for the AWS provider v6.x.

🧩 Overview

🔌 Provisions aws_appflow_connector_profile and aws_appflow_flow — the only two aws_appflow_* resources that exist in hashicorp/aws v6.54.0 (confirmed via the terraform-registry MCP).
🧭 Aggregation-flavored composite — two independent for_each collections, no single keystone, because a connector profile is a reusable peer of a flow, not its child.
🎯 Five connector types modeled deeply on both resources — Salesforce, Slack, ServiceNow, Zendesk, Redshift — chosen for our initial SaaS-integration use cases; extending to another of AppFlow's 22 connector types requires a documented schema + main.tf extension (see SCOPE.md), not an any escape hatch.
🗂️ Flow source/destination reuse AWS's own shared shapes — the "Generic Source Properties" { object } shape (Amplitude, Datadog, Dynatrace, Google Analytics, Infor Nexus, Marketo, ServiceNow, Singular, Slack, Trend Micro, Veeva, Zendesk) and "Generic Destination Properties" { object, error_handling_config } shape (EventBridge, Honeycode, Marketo) are wired in as real types — not any.
🔐 var.connector_profiles is sensitive = true — every modeled connector type carries OAuth tokens, client secrets, or passwords, so the whole variable is redacted from plan/state output.
🔑 CMK encryption is opt-in on both resources via kms_arn — null (default) uses the AppFlow-managed key.
🏷️ Tags apply to flows only — aws_appflow_connector_profile has no tags argument at all in the live schema (a documented exception to the "tags is universal" rule).
⏸️ No settable "review gate." flow_status is computed-only on the live schema — this module cannot force a new flow into a suspended/pending-review state; see Architecture Notes for the recommended process-level mitigation.

💡 Why it matters: AppFlow is a no-code/low-code bridge that can move regulated data (member/borrower PII) directly between a SaaS CRM/ITSM system and AWS storage with zero custom integration code — which means the Terraform module authoring the flow is the last configuration checkpoint before that data starts moving on a schedule.

❤️ Support this project

If these Terraform modules have been helpful to you or your organization, I'd appreciate your support in any of the following ways:

⭐ Star this repository to help others discover this Terraform module.
🤝 Connect with me on LinkedIn: linkedin.com/in/microsoftexpert
☕ Buy me a coffee: buymeacoffee.com/microsoftexpert

Whether it's a star, a professional connection, or a coffee, every gesture helps keep these modules actively maintained and continually improving. Thank you for being part of the community!

🗺️ Where this fits in the family

tf-mod-aws-appflow is a Phase 9 (Gap Closure Tier 2) module. It is a pure consumer — it creates no IAM role, bucket, key, or secret of its own; every one of those is wired in by ARN/name from a foundation module.

flowchart LR
 iam["tf-mod-aws-iam-role"]
 s3["tf-mod-aws-s3-bucket"]
 kms["tf-mod-aws-kms"]
 secrets["tf-mod-aws-secrets-manager"]
 appflow["tf-mod-aws-appflow"]
 connect["tf-mod-aws-connect"]
 detective["tf-mod-aws-detective"]
 location["tf-mod-aws-location"]
 quicksight["tf-mod-aws-quicksight"]

 iam -- "role_arn (glue_data_catalog, redshift properties)" --> appflow
 s3 -- "bucket_name (source/destination S3, Redshift staging)" --> appflow
 kms -- "kms_arn (optional CMK, profiles + flows)" --> appflow
 secrets -- "client_credentials_arn (Salesforce OAuth secret, optional)" --> appflow

 subgraph phase9["Phase 9 -- Gap Closure Tier 2"]
 appflow
 connect
 detective
 location
 quicksight
 end

 style appflow fill:#FF9900,color:#fff,stroke:#cc7a00,stroke-width:2px

ℹ️ tf-mod-aws-redshift (a Phase 1 database-family module) and tf-mod-aws-glue (Phase 3) are not required — Redshift intermediate_bucket_name/database_url and the Glue database_name are consumed as opaque caller-supplied strings per the live AppFlow provider schema.

🧬 What this module builds

flowchart TB
 subgraph AFMOD["tf-mod-aws-appflow"]
 cp["aws_appflow_connector_profile.this<br/>for_each var.connector_profiles"]
 flow["aws_appflow_flow.this<br/>for_each var.flows"]

 subgraph CRED["connector_profile_credentials -- 5 connector types modeled"]
 sf_c["salesforce"]
 sl_c["slack"]
 sn_c["service_now"]
 zd_c["zendesk"]
 rs_c["redshift"]
 end

 subgraph PROP["connector_profile_properties -- 5 connector types modeled"]
 sf_p["salesforce"]
 sl_p["slack"]
 sn_p["service_now"]
 zd_p["zendesk"]
 rs_p["redshift"]
 end

 subgraph SRC["source_flow_config -- one per flow"]
 src_props["source_connector_properties<br/>S3 / Salesforce fully typed<br/>+ AWS Generic Source shape"]
 end

 subgraph DEST["destination_flow_config -- list, min 1"]
 dest_props["destination_connector_properties<br/>S3 / Salesforce / Redshift / Zendesk<br/>+ AWS Generic Destination shape"]
 end

 subgraph TT["task (set) + trigger_config"]
 tasks["task[] -- field transforms"]
 trigger["trigger_config -- OnDemand / Scheduled / Event"]
 end

 cp --> CRED
 cp --> PROP
 flow --> SRC
 flow --> DEST
 flow --> TT
 end

 style cp fill:#FF9900,color:#fff,stroke:#cc7a00,stroke-width:2px
 style flow fill:#FF9900,color:#fff,stroke:#cc7a00,stroke-width:2px

Block	Role	Cardinality
`aws_appflow_connector_profile.this`	Named SaaS/AWS connector credentials + properties	0–N, `for_each var.connector_profiles`
`aws_appflow_flow.this`	Source → task(s) → destination(s) data-movement pipeline	0–N, `for_each var.flows`
`destination_flow_config`	Per-flow destination(s)	1–N per flow (live schema `min_items = 1`)
`task`	Per-flow field transformation	1–N per flow (live schema `min_items = 1`, rendered as a `set`)

✅ Provider / Versions

Requirement	Version
Terraform	`>= 1.12.0`
`hashicorp/aws`	`>= 6.0, < 7.0`

No provider {} block is declared inside the module — the caller's configured provider (and its region/credentials) is inherited. No region variable — AppFlow is a regional service with no us-east-1 global-resource coupling.

🔑 Required IAM Permissions

Least-privilege actions the Terraform identity needs.

Action	Required for	Notes
`appflow:CreateConnectorProfile`, `appflow:DescribeConnectorProfiles`, `appflow:UpdateConnectorProfile`, `appflow:DeleteConnectorProfile`	Connector profile lifecycle	Scope `Resource` to `arn:aws:appflow:<region>:<account>:connectorprofile/*`
`appflow:CreateFlow`, `appflow:DescribeFlow`, `appflow:DescribeFlowExecutionRecords`, `appflow:UpdateFlow`, `appflow:DeleteFlow`	Flow lifecycle	Scope `Resource` to `arn:aws:appflow:<region>:<account>:flow/*`
`appflow:UseConnectorProfile`	Attaching a connector profile to a flow	—
`appflow:TagResource`, `appflow:UntagResource`, `appflow:ListTagsForResource`	Applying `tags` to `aws_appflow_flow`	Connector profiles are NOT taggable — no such action applies to that resource type
`kms:DescribeKey`, `kms:GenerateDataKey`, `kms:Decrypt`	CMK-encrypted profiles/flows (`kms_arn`)	Only when a customer-managed key is supplied; the KMS key policy must also grant `appflow.amazonaws.com`
`s3:GetObject`, `s3:ListBucket`	S3 as a flow source	Granted via the bucket policy to `appflow.amazonaws.com`, not the Terraform identity — see AWS Prerequisites
`s3:PutObject`, `s3:AbortMultipartUpload`, `s3:ListMultipartUploadParts`, `s3:ListBucketMultipartUploads`, `s3:GetBucketAcl`, `s3:PutObjectAcl`	S3 as a flow destination	Same as above
`iam:PassRole` — *scoped to the exact role ARN(s), never `""`**	`metadata_catalog_config.glue_data_catalog.role_arn`; Redshift `role_arn`/`data_api_role_arn`	The Terraform identity's own policy must grant this on the exact ARN supplied
`secretsmanager:GetSecretValue`	Salesforce `client_credentials_arn`	Only if the profile references a Secrets Manager secret

ℹ️ No service-linked role is auto-created by either aws_appflow_connector_profile or aws_appflow_flow.

📋 AWS Prerequisites

SaaS-side OAuth app registration is OUTSIDE Terraform. Before a Salesforce, Slack, ServiceNow, or Zendesk connector profile can authenticate, an administrator must register an OAuth app / connected app / API client in that external system — AWS/Terraform cannot perform this registration. In practice, complete the OAuth handshake once (console or CLI) to obtain a long-lived refresh_token/access_token, then supply it to this module for ongoing Terraform-managed reuse.
S3 bucket policies for AppFlow are NOT managed by this module. Both source and destination S3 buckets need a bucket policy granting appflow.amazonaws.com the relevant s3:* actions — wire this from tf-mod-aws-s3-bucket's policy input.
PII / change-review posture. AppFlow can move regulated data between systems — treat every entry in var.flows as sensitive configuration deserving the same change-review rigor as a database schema change.
Region constraints. None — AppFlow has no us-east-1 global-resource requirement.
Quotas. 100 flows per account/Region and 100 connector profiles per account/Region (soft limits, default).

📁 Module Structure

tf-mod-aws-appflow/
├── providers.tf # terraform{} + required_providers (aws >= 6.0, < 7.0); no provider block
├── variables.tf # connector_profiles (sensitive), flows, tags -- no timeouts variable
├── main.tf # aws_appflow_connector_profile.this + aws_appflow_flow.this, both for_each
├── outputs.tf # connector_profiles, flows, flow_arns, connector_profile_arns, tags_all
├── README.md # this file
└── SCOPE.md # in/out-of-scope, IAM, prerequisites, emits, gotchas, design decisions

⚙️ Quick Start

module "appflow" {
  source = "git::https://github.com/microsoftexpert/tf-mod-aws-appflow?ref=v1.0.0"

  connector_profiles = {
    salesforce-crm = {
      connector_type = "Salesforce"
      credentials = {
        salesforce = {
          refresh_token = var.salesforce_refresh_token # from a completed OAuth handshake
        }
      }
      properties = {
        salesforce = {
          instance_url = "https://casey.my.salesforce.com"
        }
      }
    }
  }

  flows = {
    accounts-to-s3 = {
      source_flow_config = {
        connector_type         = "Salesforce"
        connector_profile_name = "salesforce-crm"
        source_connector_properties = {
          salesforce = { object = "Account" }
        }
      }
      destination_flow_config_list = [{
        connector_type = "S3"
        destination_connector_properties = {
          s3 = { bucket_name = module.landing_bucket.name } # tf-mod-aws-s3-bucket
        }
      }]
      task = [{
        source_fields      = ["Id", "Name"]
        task_type          = "Map"
        connector_operator = { salesforce = "NO_OP" }
      }]
      trigger_config = { trigger_type = "OnDemand" }
    }
  }
}

⚠️ Pin the source with ?ref=v1.0.0 — never a branch. var.connector_profiles is sensitive = true; never hardcode real tokens in version-controlled .tfvars — wire from Secrets Manager or an environment-injected variable.

🔌 Cross-Module Contract

Consumes

Input	Type	Source module
`connector_profiles[].kms_arn`, `flows[].kms_arn`	`string` (KMS key ARN)	`tf-mod-aws-kms` (`arn`)
`connector_profiles[].properties.redshift.bucket_name`, `flows[].*.destination_connector_properties.s3.bucket_name`	`string`	`tf-mod-aws-s3-bucket` (`name`)
`connector_profiles[].properties.redshift.role_arn`, `flows[].metadata_catalog_config.glue_data_catalog.role_arn`	`string` (IAM role ARN)	`tf-mod-aws-iam-role` (`arn`)
`connector_profiles[*].credentials.salesforce.client_credentials_arn`	`string` (Secrets Manager ARN)	`tf-mod-aws-secrets-manager` (`arn`)

Emits

Output	Description	Consumed by
`connector_profiles`	Map of profile key → `{ id, arn, credentials_arn, name, connector_type }`	Root config wiring `connector_profile_name` into `flows`
`connector_profile_arns`	Map of profile key → `arn`	IAM policy `Resource` blocks scoping `appflow:UseConnectorProfile`
`flows`	Map of flow key → `{ id, arn, name, flow_status, tags_all }`	Monitoring/governance tooling reading `flow_status`
`flow_arns`	Map of flow key → `arn`	IAM policy `Resource` blocks scoping `appflow:*`; EventBridge flow-status-change rules
`tags_all`	Map of flow key → merged tags (flows only)	governance/audit

📚 Example Library

1 · Minimal — Salesforce (source) → S3 (destination), on-demand trigger

module "appflow_sf_to_s3" {
  source = "git::https://github.com/microsoftexpert/tf-mod-aws-appflow?ref=v1.0.0"

  connector_profiles = {
    salesforce-crm = {
      connector_type = "Salesforce"
      credentials    = { salesforce = { refresh_token = var.salesforce_refresh_token } }
      properties     = { salesforce = { instance_url = "https://casey.my.salesforce.com" } }
    }
  }

  flows = {
    contacts-export = {
      source_flow_config = {
        connector_type         = "Salesforce"
        connector_profile_name = "salesforce-crm"
        source_connector_properties = {
          salesforce = { object = "Contact" }
        }
      }
      destination_flow_config_list = [{
        connector_type = "S3"
        destination_connector_properties = {
          s3 = { bucket_name = module.crm_export_bucket.name }
        }
      }]
      task = [{
        source_fields      = ["Id", "Email"]
        task_type          = "Map"
        connector_operator = { salesforce = "NO_OP" }
      }]
      trigger_config = { trigger_type = "OnDemand" }
    }
  }
}

2 · S3 (source) → Salesforce (destination), UPSERT write operation

module "appflow_s3_to_sf" {
  source = "git::https://github.com/microsoftexpert/tf-mod-aws-appflow?ref=v1.0.0"

  connector_profiles = {
    salesforce-crm = {
      connector_type = "Salesforce"
      credentials    = { salesforce = { refresh_token = var.salesforce_refresh_token } }
      properties     = { salesforce = { instance_url = "https://casey.my.salesforce.com" } }
    }
  }

  flows = {
    leads-import = {
      source_flow_config = {
        connector_type = "S3"
        source_connector_properties = {
          s3 = {
            bucket_name   = module.leads_landing_bucket.name
            bucket_prefix = "inbound/leads"
          }
        }
      }
      destination_flow_config_list = [{
        connector_type         = "Salesforce"
        connector_profile_name = "salesforce-crm"
        destination_connector_properties = {
          salesforce = {
            object               = "Lead"
            write_operation_type = "UPSERT"
            id_field_names       = ["Email"]
          }
        }
      }]
      task = [{
        source_fields      = ["Email", "Company"]
        task_type          = "Map"
        connector_operator = { s3 = "NO_OP" }
      }]
      trigger_config = { trigger_type = "OnDemand" }
    }
  }
}

3 · Scheduled trigger — hourly incremental pull

module "appflow_scheduled" {
  source = "git::https://github.com/microsoftexpert/tf-mod-aws-appflow?ref=v1.0.0"

  connector_profiles = {
    salesforce-crm = {
      connector_type = "Salesforce"
      credentials    = { salesforce = { refresh_token = var.salesforce_refresh_token } }
      properties     = { salesforce = { instance_url = "https://casey.my.salesforce.com" } }
    }
  }

  flows = {
    opportunities-hourly = {
      source_flow_config = {
        connector_type         = "Salesforce"
        connector_profile_name = "salesforce-crm"
        source_connector_properties = {
          salesforce = { object = "Opportunity" }
        }
        incremental_pull_config = { datetime_type_field_name = "LastModifiedDate" }
      }
      destination_flow_config_list = [{
        connector_type = "S3"
        destination_connector_properties = {
          s3 = { bucket_name = module.crm_export_bucket.name }
        }
      }]
      task = [{
        source_fields      = ["Id", "Amount", "StageName"]
        task_type          = "Map"
        connector_operator = { salesforce = "NO_OP" }
      }]
      trigger_config = {
        trigger_type = "Scheduled"
        trigger_properties = {
          scheduled = {
            schedule_expression = "rate(1hours)"
            data_pull_mode      = "Incremental"
          }
        }
      }
    }
  }
}

4 · On-demand trigger (secure-default review gate)

# flow_status cannot be forced to a suspended state (see Architecture Notes --
# it is computed-only on the live schema). The process mitigation is to
# create every new flow with trigger_type = "OnDemand" first, review it, then
# promote to "Scheduled" in a follow-up change once approved.
module "appflow_review_gate" {
  source = "git::https://github.com/microsoftexpert/tf-mod-aws-appflow?ref=v1.0.0"

  connector_profiles = {
    zendesk-support = {
      connector_type = "Zendesk"
      credentials    = { zendesk = { client_id = var.zendesk_client_id, client_secret = var.zendesk_client_secret } }
      properties     = { zendesk = { instance_url = "https://casey.zendesk.com" } }
    }
  }

  flows = {
    tickets-pending-review = {
      source_flow_config = {
        connector_type         = "Zendesk"
        connector_profile_name = "zendesk-support"
        source_connector_properties = {
          zendesk = { object = "Tickets" }
        }
      }
      destination_flow_config_list = [{
        connector_type = "S3"
        destination_connector_properties = {
          s3 = { bucket_name = module.support_export_bucket.name }
        }
      }]
      task = [{
        source_fields      = ["id", "subject"]
        task_type          = "Map"
        connector_operator = { zendesk = "NO_OP" }
      }]
      trigger_config = { trigger_type = "OnDemand" } # <-- review gate, not "Scheduled" yet
    }
  }
}

5 · Customer-managed KMS key (flows + connector profiles)

module "appflow_cmk" {
  source = "git::https://github.com/microsoftexpert/tf-mod-aws-appflow?ref=v1.0.0"

  connector_profiles = {
    redshift-warehouse = {
      connector_type = "Redshift"
      kms_arn        = module.appflow_kms.arn # tf-mod-aws-kms
      credentials    = { redshift = { username = "appflow_svc", password = var.redshift_password } }
      properties = {
        redshift = {
          bucket_name  = module.redshift_staging_bucket.name
          database_url = "jdbc:redshift://casey-cluster.abc123.us-east-1.redshift.amazonaws.com:5439/analytics"
          role_arn     = module.redshift_appflow_role.arn # tf-mod-aws-iam-role
        }
      }
    }
  }

  flows = {
    orders-to-redshift = {
      kms_arn = module.appflow_kms.arn # PII-bearing flow -- CMK required per posture

      source_flow_config = {
        connector_type = "S3"
        source_connector_properties = {
          s3 = { bucket_name = module.orders_landing_bucket.name, bucket_prefix = "orders" }
        }
      }
      destination_flow_config_list = [{
        connector_type         = "Redshift"
        connector_profile_name = "redshift-warehouse"
        destination_connector_properties = {
          redshift = {
            intermediate_bucket_name = module.redshift_staging_bucket.name
            object                   = "orders"
          }
        }
      }]
      task = [{
        source_fields      = ["order_id", "customer_id"]
        task_type          = "Map"
        connector_operator = { s3 = "NO_OP" }
      }]
      trigger_config = { trigger_type = "OnDemand" }
    }
  }
}

6 · Credential-sensitivity example — plan output redaction

# var.connector_profiles is sensitive = true. A `terraform plan` diff for
# any change under this module shows only:
# ~ connector_profiles = (sensitive value)
# never the individual client_secret / refresh_token / password fields.
module "appflow_secrets_demo" {
  source = "git::https://github.com/microsoftexpert/tf-mod-aws-appflow?ref=v1.0.0"

  connector_profiles = {
    slack-notifications = {
      connector_type = "Slack"
      credentials = {
        slack = {
          client_id     = var.slack_client_id
          client_secret = var.slack_client_secret # never appears in plan output
        }
      }
      properties = { slack = { instance_url = "https://casey.slack.com" } }
    }
  }

  flows = {
    slack-channel-export = {
      source_flow_config = {
        connector_type         = "Slack"
        connector_profile_name = "slack-notifications"
        source_connector_properties = {
          slack = { object = "#compliance-alerts" }
        }
      }
      destination_flow_config_list = [{
        connector_type = "S3"
        destination_connector_properties = {
          s3 = { bucket_name = module.slack_export_bucket.name }
        }
      }]
      task = [{
        source_fields      = ["text", "ts"]
        task_type          = "Map"
        connector_operator = { slack = "NO_OP" }
      }]
      trigger_config = { trigger_type = "OnDemand" }
    }
  }
}

7 · ServiceNow (basic auth) source flow

module "appflow_servicenow" {
  source = "git::https://github.com/microsoftexpert/tf-mod-aws-appflow?ref=v1.0.0"

  connector_profiles = {
    servicenow-itsm = {
      connector_type = "Servicenow"
      credentials    = { service_now = { username = "appflow_svc", password = var.servicenow_password } }
      properties     = { service_now = { instance_url = "https://casey.service-now.com" } }
    }
  }

  flows = {
    incidents-export = {
      source_flow_config = {
        connector_type         = "Servicenow"
        connector_profile_name = "servicenow-itsm"
        source_connector_properties = {
          service_now = { object = "incident" }
        }
      }
      destination_flow_config_list = [{
        connector_type = "S3"
        destination_connector_properties = {
          s3 = { bucket_name = module.itsm_export_bucket.name }
        }
      }]
      task = [{
        source_fields      = ["number", "short_description"]
        task_type          = "Map"
        connector_operator = { service_now = "NO_OP" }
      }]
      trigger_config = { trigger_type = "OnDemand" }
    }
  }
}

8 · Multiple destinations on a single flow

module "appflow_multi_destination" {
  source = "git::https://github.com/microsoftexpert/tf-mod-aws-appflow?ref=v1.0.0"

  connector_profiles = {
    salesforce-crm = {
      connector_type = "Salesforce"
      credentials    = { salesforce = { refresh_token = var.salesforce_refresh_token } }
      properties     = { salesforce = { instance_url = "https://casey.my.salesforce.com" } }
    }
  }

  flows = {
    accounts-fanout = {
      source_flow_config = {
        connector_type         = "Salesforce"
        connector_profile_name = "salesforce-crm"
        source_connector_properties = {
          salesforce = { object = "Account" }
        }
      }
      destination_flow_config_list = [
        {
          connector_type = "S3"
          destination_connector_properties = {
            s3 = { bucket_name = module.raw_landing_bucket.name }
          }
        },
        {
          connector_type = "S3"
          destination_connector_properties = {
            s3 = { bucket_name = module.analytics_landing_bucket.name, bucket_prefix = "accounts" }
          }
        },
      ]
      task = [{
        source_fields      = ["Id", "Name"]
        task_type          = "Map"
        connector_operator = { salesforce = "NO_OP" }
      }]
      trigger_config = { trigger_type = "OnDemand" }
    }
  }
}

9 · Field masking task (Mask task_type)

module "appflow_masked_export" {
  source = "git::https://github.com/microsoftexpert/tf-mod-aws-appflow?ref=v1.0.0"

  connector_profiles = {
    salesforce-crm = {
      connector_type = "Salesforce"
      credentials    = { salesforce = { refresh_token = var.salesforce_refresh_token } }
      properties     = { salesforce = { instance_url = "https://casey.my.salesforce.com" } }
    }
  }

  flows = {
    contacts-masked = {
      source_flow_config = {
        connector_type         = "Salesforce"
        connector_profile_name = "salesforce-crm"
        source_connector_properties = {
          salesforce = { object = "Contact" }
        }
      }
      destination_flow_config_list = [{
        connector_type = "S3"
        destination_connector_properties = {
          s3 = { bucket_name = module.masked_export_bucket.name }
        }
      }]
      task = [
        {
          source_fields      = ["Id", "Name"]
          task_type          = "Map"
          connector_operator = { salesforce = "NO_OP" }
        },
        {
          source_fields     = ["Phone"]
          task_type         = "Mask"
          destination_field = "Phone"
          task_properties   = { MASK_LENGTH = "4" }
        },
      ]
      trigger_config = { trigger_type = "OnDemand" }
    }
  }
}

10 · Glue Data Catalog metadata cataloging

module "appflow_cataloged" {
  source = "git::https://github.com/microsoftexpert/tf-mod-aws-appflow?ref=v1.0.0"

  connector_profiles = {
    salesforce-crm = {
      connector_type = "Salesforce"
      credentials    = { salesforce = { refresh_token = var.salesforce_refresh_token } }
      properties     = { salesforce = { instance_url = "https://casey.my.salesforce.com" } }
    }
  }

  flows = {
    cataloged-accounts = {
      source_flow_config = {
        connector_type         = "Salesforce"
        connector_profile_name = "salesforce-crm"
        source_connector_properties = {
          salesforce = { object = "Account" }
        }
      }
      destination_flow_config_list = [{
        connector_type = "S3"
        destination_connector_properties = {
          s3 = { bucket_name = module.catalog_bucket.name }
        }
      }]
      task = [{
        source_fields      = ["Id", "Name"]
        task_type          = "Map"
        connector_operator = { salesforce = "NO_OP" }
      }]
      trigger_config = { trigger_type = "OnDemand" }

      metadata_catalog_config = {
        glue_data_catalog = {
          database_name = "casey_crm_catalog"
          role_arn      = module.glue_catalog_role.arn # tf-mod-aws-iam-role
          table_prefix  = "appflow_"
        }
      }
    }
  }
}

11 · for_each pattern — one flow per business domain

locals {
  crm_objects = {
    accounts      = "Account"
    contacts      = "Contact"
    opportunities = "Opportunity"
  }
}

module "appflow_domains" {
  source = "git::https://github.com/microsoftexpert/tf-mod-aws-appflow?ref=v1.0.0"

  connector_profiles = {
    salesforce-crm = {
      connector_type = "Salesforce"
      credentials    = { salesforce = { refresh_token = var.salesforce_refresh_token } }
      properties     = { salesforce = { instance_url = "https://casey.my.salesforce.com" } }
    }
  }

  flows = {
    for key, object_name in local.crm_objects : "sf-${key}-export" => {
      source_flow_config = {
        connector_type         = "Salesforce"
        connector_profile_name = "salesforce-crm"
        source_connector_properties = {
          salesforce = { object = object_name }
        }
      }
      destination_flow_config_list = [{
        connector_type = "S3"
        destination_connector_properties = {
          s3 = { bucket_name = module.crm_export_bucket.name, bucket_prefix = key }
        }
      }]
      task = [{
        source_fields      = ["Id"]
        task_type          = "Map"
        connector_operator = { salesforce = "NO_OP" }
      }]
      trigger_config = { trigger_type = "OnDemand" }
    }
  }
}

12 · import block — adopt an existing flow and connector profile

import {
  to = module.appflow.aws_appflow_connector_profile.this["salesforce-crm"]
  id = "salesforce-crm"
}

import {
  to = module.appflow.aws_appflow_flow.this["contacts-export"]
  id = "contacts-export"
}

module "appflow" {
  source = "git::https://github.com/microsoftexpert/tf-mod-aws-appflow?ref=v1.0.0"

  connector_profiles = {
    salesforce-crm = {
      connector_type = "Salesforce"
      credentials    = { salesforce = { refresh_token = var.salesforce_refresh_token } }
      properties     = { salesforce = { instance_url = "https://casey.my.salesforce.com" } }
    }
  }

  flows = {
    contacts-export = {
      source_flow_config = {
        connector_type              = "Salesforce"
        connector_profile_name      = "salesforce-crm"
        source_connector_properties = { salesforce = { object = "Contact" } }
      }
      destination_flow_config_list = [{
        connector_type                   = "S3"
        destination_connector_properties = { s3 = { bucket_name = module.crm_export_bucket.name } }
      }]
      task = [{
        source_fields      = ["Id", "Email"]
        task_type          = "Map"
        connector_operator = { salesforce = "NO_OP" }
      }]
      trigger_config = { trigger_type = "OnDemand" }
    }
  }
}

13 · Secure-by-default opt-out — AppFlow-managed key instead of a CMK

# Opting out of the CMK-preferred posture. Only acceptable for non-PII,
# ephemeral sandbox flows -- never for member/borrower data.
module "appflow_sandbox" {
  source = "git::https://github.com/microsoftexpert/tf-mod-aws-appflow?ref=v1.0.0"

  connector_profiles = {
    sandbox-slack = {
      connector_type = "Slack"
      # kms_arn omitted -- DOCUMENTED EXCEPTION: non-PII sandbox profile only
      credentials = { slack = { client_id = var.slack_client_id, client_secret = var.slack_client_secret } }
      properties  = { slack = { instance_url = "https://casey-sandbox.slack.com" } }
    }
  }

  flows = {
    sandbox-export = {
      # kms_arn omitted -- same documented exception
      source_flow_config = {
        connector_type              = "Slack"
        connector_profile_name      = "sandbox-slack"
        source_connector_properties = { slack = { object = "#sandbox-test" } }
      }
      destination_flow_config_list = [{
        connector_type                   = "S3"
        destination_connector_properties = { s3 = { bucket_name = module.sandbox_bucket.name } }
      }]
      task = [{
        source_fields      = ["text"]
        task_type          = "Map"
        connector_operator = { slack = "NO_OP" }
      }]
      trigger_config = { trigger_type = "OnDemand" }
    }
  }
}

14 · Tags — merge with provider default_tags (flows only)

# Caller's provider block owns default_tags; resource tags win on key conflict.
# Connector profiles are NOT taggable -- only aws_appflow_flow receives tags.
provider "aws" {
  region = "us-east-1"
  default_tags {
    tags = { Owner = "platform", ManagedBy = "terraform" }
  }
}

module "appflow_tagged" {
  source = "git::https://github.com/microsoftexpert/tf-mod-aws-appflow?ref=v1.0.0"

  connector_profiles = {
    salesforce-crm = {
      connector_type = "Salesforce"
      credentials    = { salesforce = { refresh_token = var.salesforce_refresh_token } }
      properties     = { salesforce = { instance_url = "https://casey.my.salesforce.com" } }
    }
  }

  flows = {
    contacts-export = {
      source_flow_config = {
        connector_type              = "Salesforce"
        connector_profile_name      = "salesforce-crm"
        source_connector_properties = { salesforce = { object = "Contact" } }
      }
      destination_flow_config_list = [{
        connector_type                   = "S3"
        destination_connector_properties = { s3 = { bucket_name = module.crm_export_bucket.name } }
      }]
      task = [{
        source_fields      = ["Id", "Email"]
        task_type          = "Map"
        connector_operator = { salesforce = "NO_OP" }
      }]
      trigger_config = { trigger_type = "OnDemand" }
    }
  }

  tags = {
    Environment = "prod" # resource tag -- wins over default_tags on key conflict
    DataClass   = "npi"
  }
}
# module.appflow_tagged.tags_all["contacts-export"] == { Owner, ManagedBy, Environment, DataClass }

15 · End-to-end composition — IAM role + S3 + KMS + Secrets Manager + Salesforce-to-Redshift (finale)

# Customer-managed CMK for both the connector profile and the flow
module "appflow_kms" {
  source = "git::https://github.com/microsoftexpert/tf-mod-aws-kms?ref=v1.0.0"
  name   = "casey-appflow"
}

# Secrets Manager secret holding the Salesforce connected-app client credentials
module "salesforce_oauth_secret" {
  source      = "git::https://github.com/microsoftexpert/tf-mod-aws-secrets-manager?ref=v1.0.0"
  name        = "casey/appflow/salesforce-oauth"
  kms_key_arn = module.appflow_kms.arn
}

# Redshift staging bucket
module "redshift_staging_bucket" {
  source      = "git::https://github.com/microsoftexpert/tf-mod-aws-s3-bucket?ref=v1.0.0"
  name        = "casey-appflow-redshift-staging"
  kms_key_arn = module.appflow_kms.arn
}

# IAM role AppFlow assumes to write into Redshift via the Data API
module "redshift_appflow_role" {
  source = "git::https://github.com/microsoftexpert/tf-mod-aws-iam-role?ref=v1.0.0"
  name   = "casey-appflow-redshift"

  assume_role_policy = jsonencode({
    Version = "2012-10-17"
    Statement = [{
      Effect    = "Allow"
      Principal = { Service = "appflow.amazonaws.com" }
      Action    = "sts:AssumeRole"
    }]
  })

  inline_policies = {
    redshift-data-api = {
      policy = jsonencode({
        Version = "2012-10-17"
        Statement = [
          {
            Effect   = "Allow"
            Action   = ["redshift-data:ExecuteStatement", "redshift-data:DescribeStatement"]
            Resource = "*"
          },
          {
            Effect   = "Allow"
            Action   = ["s3:PutObject", "s3:GetObject", "s3:ListBucket"]
            Resource = [module.redshift_staging_bucket.arn, "${module.redshift_staging_bucket.arn}/*"]
          }
        ]
      })
    }
  }
}

# The Terraform identity applying this module needs iam:PassRole scoped to
# module.redshift_appflow_role.arn -- never "*" (see Required IAM Permissions).

module "appflow" {
  source = "git::https://github.com/microsoftexpert/tf-mod-aws-appflow?ref=v1.0.0"

  connector_profiles = {
    salesforce-crm = {
      connector_type = "Salesforce"
      kms_arn        = module.appflow_kms.arn
      credentials = {
        salesforce = {
          client_credentials_arn = module.salesforce_oauth_secret.arn
          refresh_token          = var.salesforce_refresh_token
        }
      }
      properties = {
        salesforce = { instance_url = "https://casey.my.salesforce.com" }
      }
    }

    redshift-warehouse = {
      connector_type = "Redshift"
      kms_arn        = module.appflow_kms.arn
      credentials    = { redshift = { username = "appflow_svc", password = var.redshift_password } }
      properties = {
        redshift = {
          bucket_name  = module.redshift_staging_bucket.name
          database_url = "jdbc:redshift://casey-cluster.abc123.us-east-1.redshift.amazonaws.com:5439/analytics"
          role_arn     = module.redshift_appflow_role.arn
        }
      }
    }
  }

  flows = {
    opportunities-to-warehouse = {
      kms_arn     = module.appflow_kms.arn
      description = "Salesforce Opportunity records into the analytics warehouse"

      source_flow_config = {
        connector_type          = "Salesforce"
        connector_profile_name  = "salesforce-crm"
        incremental_pull_config = { datetime_type_field_name = "LastModifiedDate" }
        source_connector_properties = {
          salesforce = { object = "Opportunity" }
        }
      }

      destination_flow_config_list = [{
        connector_type         = "Redshift"
        connector_profile_name = "redshift-warehouse"
        destination_connector_properties = {
          redshift = {
            intermediate_bucket_name = module.redshift_staging_bucket.name
            object                   = "opportunities"
            error_handling_config = {
              bucket_name   = module.redshift_staging_bucket.name
              bucket_prefix = "errors"
            }
          }
        }
      }]

      task = [{
        source_fields      = ["Id", "Amount", "StageName", "CloseDate"]
        task_type          = "Map"
        connector_operator = { salesforce = "NO_OP" }
      }]

      trigger_config = {
        trigger_type = "Scheduled"
        trigger_properties = {
          scheduled = {
            schedule_expression = "rate(6hours)"
            data_pull_mode      = "Incremental"
          }
        }
      }

      metadata_catalog_config = {
        glue_data_catalog = {
          database_name = "casey_crm_catalog"
          role_arn      = module.redshift_appflow_role.arn
          table_prefix  = "appflow_"
        }
      }
    }
  }

  tags = { Environment = "prod", App = "crm-warehouse-sync", DataClass = "npi" }
}

📥 Inputs

ℹ️ High-level grouping:

Connector profiles: connector_profiles — map(object({...})), sensitive = true, default {}. Per-entry: connector_type (Salesforce | Slack | Servicenow | Zendesk | Redshift), connector_label, connection_mode (Public default | Private), kms_arn, credentials (matching-type object), properties (matching-type object).
Flows: flows — map(object({...})), required, no default. Per-entry: description, kms_arn, source_flow_config (single, required), destination_flow_config_list (list, min 1, required), task (list, min 1, required), trigger_config (single, required), metadata_catalog_config (optional).
Universal: tags (flows only — connector profiles have no tags argument).

No timeouts variable — neither aws_appflow_connector_profile nor aws_appflow_flow defines a timeouts {} block in the live provider schema.

🧾 Outputs

Connector profiles: connector_profiles (map: id, arn, credentials_arn, name, connector_type), connector_profile_arns (map of arn)
Flows: flows (map: id, arn, name, flow_status, tags_all), flow_arns (map of arn)
Tags: tags_all (map, flows only)

ℹ️ No output is marked sensitive — the module emits no secrets (only ARNs, names, and status). Note that the connector_profiles input variable is marked sensitive = true (it carries OAuth tokens/secrets/passwords); see Architecture Notes.

🧠 Architecture Notes

ARN formats: connector profile — arn:aws:appflow:<region>:<account-id>:connectorprofile/<name>; flow — arn:aws:appflow:<region>:<account-id>:flow/<name>.
id is Optional+Computed on both resources and equals name in practice — confirmed via terraform providers schema -json; AppFlow has no separate internal ID distinct from the caller-supplied name.
name is FORCE-NEW (immutable) on both resources — AppFlow exposes no rename operation for either a connector profile or a flow; changing a map key in var.connector_profiles/var.flows destroys and recreates that resource.
flow_status is COMPUTED ONLY. It cannot be set as an input (confirmed via the live schema — computed = true with no optional/required marker). A flow activates according to its trigger_config.trigger_type immediately on creation: OnDemand flows sit idle until manually started, Scheduled/Event flows begin running per their configuration right away. This module cannot implement a Terraform-level "review gate" — the recommended process mitigation is to create new flows with trigger_type = "OnDemand" first, review the configuration and run it manually once, then promote to Scheduled/Event in a follow-up change.
tags ↔ tags_all ↔ default_tags applies to flows only. aws_appflow_connector_profile has no tags/tags_all attribute at all in the live schema — a documented exception to the "tags is universal" rule, parallel to (but distinct from) the ASG dynamic "tag" exception: here there is no tag mechanism whatsoever to hook into.
var.connector_profiles is sensitive = true and cannot be used directly as a for_each source. Terraform rejects sensitive values as for_each arguments ("could be exposed as a resource instance key"). This module iterates nonsensitive(toset(keys(var.connector_profiles))) (profile NAMES only) and re-reads the real object via var.connector_profiles[each.key] inside the resource body — every credential value stays redacted in plan/state output; only the map keys (profile names, never secret) become resource instance keys.
connector_type casing differs from the nested block name. connector_type = "Servicenow" (AWS's own mixed-case enum) selects the service_now (snake_case) credentials/properties block — getting this wrong renders no credentials block at all rather than erroring at connector_type validation time; a dedicated validation block cross-checks that the matching sub-object is populated.
bucket_prefix is REQUIRED on the flow SOURCE s3 block, contrary to registry doc prose which lists it "(Optional)" — the live v6.54.0 JSON schema marks it required = true. The DESTINATION s3 block's bucket_prefix is genuinely optional.
destination_flow_config is a repeatable block (min_items = 1, no max) on the live schema, so flows[*].destination_flow_config_list is a list(object(...)), even though most flows populate exactly one destination.
task is a set block on the live resource, not a list — this module models it as list(object(...)) for caller ergonomics since ordering has no operational meaning to AppFlow.
No timeouts {} block on either resource — confirmed via the live JSON schema; this module has no timeouts variable, unlike most composites in this library.
No us-east-1 constraint — AppFlow is a regional service; the module declares no region variable and inherits the caller's provider region.
Destroy ordering. Deleting a connector profile that a flow still references fails at the AppFlow API level (a flow must be deleted or repointed first) — Terraform's own dependency graph handles this automatically only when the flow references the profile via this module's own resources; out-of-band profile references are the caller's responsibility to sequence.

🧱 Design Principles

Secure-by-default posture and the explicit opt-out for each:

Hardened default	Behavior	Opt-out / control
Encryption (connector profiles + flows)	`kms_arn = null` (AppFlow-managed key) by default on both resources	Supply a customer-managed CMK ARN (wire from `tf-mod-aws-kms`) — recommends a CMK for any profile/flow touching PII
Connection mode	`connection_mode = "Public"` (AWS's own default)	Set `"Private"` to route over AWS PrivateLink where the connector supports it
Credential exposure	`var.connector_profiles` marked `sensitive = true` in full	Not overridable — a hard floor
Destination write-failure handling	`error_handling_config` exposed on Salesforce/Redshift/Zendesk/generic destinations	Caller must supply `error_handling_config.bucket_name` to capture failed records — this module never silently drops failures, but also never forces a specific error bucket
Flow activation state	AWS default (no settable review gate — `flow_status` is computed-only)	Use `trigger_type = "OnDemand"` as a process-level review gate before promoting to `"Scheduled"`/`"Event"`
Write operation type (Salesforce/Zendesk destinations)	`write_operation_type = "INSERT"` (AWS's own append-only default)	Set `"UPSERT"`/`"UPDATE"`/`"DELETE"` explicitly — `"UPSERT"` additionally requires `id_field_names`

Why an explicit connector-type subset instead of any: AppFlow's credential/properties/source/destination schemas are genuinely polymorphic across 22 connector types with no single shared shape. This module deeply types Salesforce, Slack, ServiceNow, Zendesk, and Redshift (plus AWS's own shared "Generic Source/Destination Properties" shapes for several more), and explicitly declines to add an any escape hatch — Terraform dynamic blocks require statically-known block names, so an any-typed "generic connector" field could never actually render onto the resource without a matching dynamic block anyway. See SCOPE.md "Design decisions" for the full connector-type coverage table.

🚀 Runbook

terraform init -backend=false
terraform validate
terraform fmt -check
terraform plan # requires valid AWS credentials (profile / SSO / OIDC) + a region
terraform apply
terraform output

⚠️ plan / apply require valid AWS credentials and a configured region (provider block / AWS_PROFILE / SSO / OIDC). Always pin the module source with ?ref=v1.0.0, never a branch.

🧪 Testing

terraform init -backend=false && terraform validate — schema + reference integrity, including all cross-variable validation blocks (connector-type enum, credentials/properties-matching-connector_type, destination/task minimum-count, trigger/task-type enums).
terraform fmt -check — formatting.
terraform plan against a sandbox account — confirm connector_profiles renders as (sensitive value) in the plan diff, and that exactly one credentials/properties block populates per connector profile.
Confirm iam:PassRole is scoped correctly on the Terraform identity's own policy before the first apply in a shared account.
After apply, manually trigger an OnDemand flow once via the console/CLI and confirm records land in the destination before promoting the flow to Scheduled.

💬 Example Output

Apply complete! Resources: 2 added, 0 changed, 0 destroyed.

Outputs:

connector_profile_arns = {
 "salesforce-crm" = "arn:aws:appflow:us-east-1:123456789012:connectorprofile/salesforce-crm"
}
connector_profiles = {
 "salesforce-crm" = {
 "arn" = "arn:aws:appflow:us-east-1:123456789012:connectorprofile/salesforce-crm"
 "connector_type" = "Salesforce"
 "credentials_arn" = "arn:aws:appflow:us-east-1:123456789012:connectorprofile/salesforce-crm/credential"
 "id" = "salesforce-crm"
 "name" = "salesforce-crm"
 }
}
flow_arns = {
 "contacts-export" = "arn:aws:appflow:us-east-1:123456789012:flow/contacts-export"
}
flows = {
 "contacts-export" = {
 "arn" = "arn:aws:appflow:us-east-1:123456789012:flow/contacts-export"
 "flow_status" = "Draft"
 "id" = "contacts-export"
 "name" = "contacts-export"
 "tags_all" = {}
 }
}
tags_all = {
 "contacts-export" = {}
}

🔍 Troubleshooting

Error: Invalid for_each argument mentioning "sensitive value": you edited main.tf's for_each on aws_appflow_connector_profile.this — it must stay nonsensitive(toset(keys(var.connector_profiles))), never var.connector_profiles directly, because that variable is sensitive = true.
connector_profiles[*]: the credentials AND properties object matching connector_type must both be set: the connector_type string (e.g. "Slack") doesn't match a populated credentials/properties sub-object (e.g. you set credentials.salesforce instead of credentials.slack). Remember connector_type uses AWS's mixed-case enum while the sub-object key is snake_case (Servicenow → service_now).
A connector profile renders with no credentials at all (empty block): same root cause as above but caught only at apply if a validation block was bypassed — double-check the connector_type casing against the nested object key.
AccessDenied on firehose-style iam:PassRole errors for Redshift/Glue: the Terraform identity's policy is missing (or incorrectly scoped) iam:PassRole on the exact role_arn/data_api_role_arn supplied. Never widen this to "*".
Flow created but no data appears in the destination: confirm the destination S3 bucket policy grants appflow.amazonaws.com the required actions (this module does not manage that policy — see AWS Prerequisites); confirm the connector profile's OAuth token/refresh_token hasn't expired.
Need to suspend a flow pending review, but flow_status won't set: expected — flow_status is computed-only on the live provider schema. Use trigger_type = "OnDemand" as the process-level review gate instead (see Architecture Notes).
Tag drift / unexpected tags on a flow: caused by default_tags overlap. tags_all merges resource tags over provider default_tags with resource tags winning — if a value differs from what you set, a default_tags entry is colliding. Remember connector profiles never show up in tags_all — they have no tags at all.
Credential-chain failures (NoCredentialProviders / ExpiredToken): no valid AWS credentials resolved. Set AWS_PROFILE, refresh SSO, or confirm OIDC role assumption in CI. The module never takes AWS credentials as variables (SaaS credentials in connector_profiles are a separate, intentional exception).
region / us-east-1 errors: not applicable — AppFlow has no global-resource coupling; confirm the caller's provider region is correct for where the flow/profile should live.

🔗 Related Docs

Terraform Registry — hashicorp/aws provider: aws_appflow_connector_profile, aws_appflow_flow
AWS — Amazon AppFlow User Guide (supported connectors, flow triggers, data transformation tasks)
AWS — Amazon AppFlow API Reference (CreateConnectorProfile, CreateFlow, StartFlow)
AWS — Amazon AppFlow quotas
— tf-mod-aws-iam-role, tf-mod-aws-s3-bucket, tf-mod-aws-kms, tf-mod-aws-secrets-manager (sibling modules)

🧡 "Infrastructure as Code should be standardized, consistent, and secure."

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Repository files navigation

🟧 AWS AppFlow Terraform Module

🧩 Overview

❤️ Support this project

🗺️ Where this fits in the family

🧬 What this module builds

✅ Provider / Versions

🔑 Required IAM Permissions

📋 AWS Prerequisites

📁 Module Structure

⚙️ Quick Start

🔌 Cross-Module Contract

Consumes

Emits

📚 Example Library

📥 Inputs

🧾 Outputs

🧠 Architecture Notes

🧱 Design Principles

🚀 Runbook

🧪 Testing

💬 Example Output

🔍 Troubleshooting

🔗 Related Docs

About

Uh oh!

Releases

Packages

Uh oh!

Contributors

Uh oh!

Languages

Name		Name	Last commit message	Last commit date
Latest commit History 3 Commits
.gitignore		.gitignore
LICENSE		LICENSE
README.md		README.md
SCOPE.md		SCOPE.md
main.tf		main.tf
outputs.tf		outputs.tf
providers.tf		providers.tf
variables.tf		variables.tf

Folders and files

Latest commit

History

Repository files navigation

🟧 AWS AppFlow Terraform Module

🧩 Overview

❤️ Support this project

🗺️ Where this fits in the family

🧬 What this module builds

✅ Provider / Versions

🔑 Required IAM Permissions

📋 AWS Prerequisites

📁 Module Structure

⚙️ Quick Start

🔌 Cross-Module Contract

Consumes

Emits

📚 Example Library

📥 Inputs

🧾 Outputs

🧠 Architecture Notes

🧱 Design Principles

🚀 Runbook

🧪 Testing

💬 Example Output

🔍 Troubleshooting

🔗 Related Docs

About

Topics

Resources

License

Uh oh!

Stars

Watchers

Forks

Releases

Packages 0

Uh oh!

Contributors

Uh oh!

Languages

Packages