Instantiating Multiple Providers with a loop

[JakeNeyer]

Current Terraform Version

Terraform v0.11.11

Use-cases

In my current situation, I am using the AWS provider so I will scope this feature request to that specific provider, although this may extend to other providers as well.

I am attempting to create resources in multiple AWS accounts. The number of the accounts ranges from 0....x and will be dynamic. I would like to be able to instantiate multiple providers which can assume a role in each account, and in turn, create resources with the associated provider without hard-coding providers for each subsequent account.

For example, something like this:

variable "accounts" {
  type    = "list"

  default = ["123456789012", "210987654321"]
}
variable "names" {
  type = "map"

  default = {
    "123456789012" = "foo"
    "210987654321" = "bar"
  }
}
provider "aws" {
  count   = "${length(var.accounts)}"
  alias   = "${lookup(var.names, element(var.accounts, count.index))}"
  
  assume_role {
    role_arn = "arn:aws:iam::${element(var.accounts, count.index)}:role/ASSUMEDROLE"
  }

resource "aws_instance" "web" {
  count         = "${length(var.accounts)}"
  provider      = "aws.${lookup(var.names, element(var.accounts, count.index))}"
  ami           = "${data.aws_ami.ubuntu.id}"
  instance_type = "t2.micro"

  tags = {
    Name = "HelloWorld"
  }
}

[debovema]

With a more generic formulation, it would be great to have a count attribute in providers, not only to instantiate multiple times the same provider but also to choose between several providers: some having a zero count, some having a non-zero count.

[hhh0505]

Has anyone ever found any kind of workaround for this kind of scenario? We have multiple AWS accounts and would like to be able to deploy modules across all accounts without having to have repeatable code for each account.

[sc250024]

@jaken551 Have you tried this with Terraform 0.12 ? Did you make any progress? I ask because I'm looking for the same functionality.

[kuwas]

We are using terraform 0.12 and have the same issue when using the terraform-provider-aws and a few other providers. Basically this is a problem in any provider that wasnt designed to work across multiple organizations/accounts/workspaces/etc. By comparison, terraform-provider-google and terraform-provider-tfe dont really require this feature.

To summarize, the ask here is to allow provider blocks to instantiated dynamically and be dependent on computed values from other resources, such as creating a new AWS account and then using the account ID to assume role to perform actions in the new child account.

provider blocks will also need to support for_each and count to allow loops.

---

@hhh0505 Our current workaround is for our main terraform pipeline to create:

• 1x AWS master account user with only AssumeRole access to the child account
• 1x GitHub repository with some boilerplate code managed by our custom terraform-provider-git
• 1x Terraform Cloud workspace with the AWS user secrets and the Github repo as the source

At this point, the individual workspaces act as the pipelines to their child accounts. As for the boilerplate code, they can be updated in the main pipeline's repo and will propagate to the downstream repos whenever the main terraform runs.

[autodeck]

@jaken551 @kuwas - I'd also like to see provider blocks support for_each. My use case being creating aws_ses_domain_identity resources in multiple regions. These regions being defined by a list variable i.e.

resource "aws_ses_domain_identity" "example" {
  for_each = var.ses_regions

  provider = each.value
  domain = var.ses_domain
}

[shankarsundaram]

One workaround is, you need to maintain separate directory for each account and call the same terraform module by passing correct values to the modules / keep some default values. to run plan / apply in all directory use wrapper such as terragrunt.
https://blog.gruntwork.io/terragrunt-how-to-keep-your-terraform-code-dry-and-maintainable-f61ae06959d8
https://davidbegin.github.io/terragrunt/use_cases/execute-terraform-commands-on-multiple-modules-at-once.html

[rgevaert]

I'm an enterprise user and doing a for_each over the vault provider would reduce a lot of code if you could just do:

variable "namespaces" {
  type        = set(string)
  default     = []
  description = "Names to be created"
}

resource "vault_namespace" "namespace" {
  for_each = var.namespaces
  path     = each.key
}

provider "vault" {
  for_each = var.namespaces
  alias = each.key
  namespace = each.key
}

instead of copy pasting the same piece for each namespace.

[Roxyrob]

Terraform deployment environment protection using separated Cloud Providers accounts is a great method to reduce configuration/deployment error Blast Radius. E.g. different AWS accounts can be used to make deployment environments of any kind (devel, stage, prod, sandbox), with potentially a lot of account with same "common configuration".

Without a solution like for_each in provider block lot of identical code must be replicated, raising a lot the risk of typing errors on addition of each new account/environment.
Example:

• using aws profile in provider alias to assume role in accounts
• a master account manage e.g. identity account and any other single deployment/environment account (devel, prod, sandboxN, ecc.)
• creating user and permissions in identity account
• creating base common config for each account (assumable roles with policies, terraform state with S3/DynamoDB, etc.)

Based on account info like:

locals {
  accounts = {
    "devel" = { ... }
    "prod" = { ... }    
    "sandbox1" = { ... }    
    "sandbox2" = { ... }    
    ...
  }
}

for_each in provider allow to achieve a common configuration using DRY code blocks, looping on accounts as base key for many decoupled resources:

 module "example1" {
    source = "..."
    for_each local.account
    provider {
      aws = "aws.${each.key}"
    }
    ... create assumable roles or terraform states (S3/dunamoDB) or other common resources in the same way making DRY code ...
 }

[bryankaraffa]

+1 on this. We would like to use Terraform to deploy/manage IAM Roles in a list of AWS Accounts -- assuming a known role in each account for access. Below is an example implementation that would work for us with Terraform >v0.13 and the existing module.for_each -- just also need an equivalent provider.for_each:

## Just some data... a list(map())
locals {
  aws_accounts = [
    { "aws_account_id": "123456789012", "foo_value": "foo",    "bar_value": "bar"    },
    { "aws_account_id": "987654321098", "foo_value": "foofoo", "bar_value": "barbar" },
    ]
}
## Here's the proposed magic... `provider.for_each`
provider "aws" {
  for_each = local.aws_accounts
  alias = each.value.aws_account_id
  assume_role {
    role_arn    = "arn:aws:iam::${each.value.aws_account_id}:role/TerraformAccessRole"
  }
}
## Modules reference the provider dynamically using `each.value.aws_account_id`
module "foo" {
  source = "./foo"
  for_each = local.aws_accounts
  providers = {
    aws = "aws.${each.value.aws_account_id}"
  }
  foo = each.value.foo_value
}
module "bar" {
  source = "./bar"
  for_each = local.aws_accounts
  providers = {
    aws = "aws.${each.value.aws_account_id}"
  }
  bar = each.value.bar_value
}

As a workaround for not having provider.for_each or another solution for this, we are considering using a script to compile the Terraform from the dataset which is not ideal.

[DonBower]

Allowing alias as a variable in the provider config would also make for cleaner code:

variable awsEnvironments {
  type = list
  default = [
    "paas-luse1",
    "paas-lusw2",
    "gi-luse1",
    "gi-lusw2"
  ]
}

variable awsRegion {
  type = map
  default = {
    paas-luse1 = "us-east-1"
    paas-lusw2 = "us-west-2"
    gi-luse1   = "us-east-1"
    gi-lusw2   = "us-west-2"
  }
}

variable project {
  type = map
  default = {
    paas-lusw2 = {
      localEnvironment = "paas-luse1",
      localVPCSuffix = "paas",
      peerEnvironment = "paas-lusw2",
      peerVPCSuffix= "paas"}
  }
    gi-luse1 = {
      localEnvironment = "paas-luse1",
      localVPCSuffix = "paas",
      peerEnvironment = "gi-lusw2",
      peerVPCSuffix= "paas"}
  }
    gi-lusw2 = {
      localEnvironment = "paas-luse1",
      localVPCSuffix = "paas",
      peerEnvironment = "gi-lusw2",
      peerVPCSuffix= "paas"}
  }
}

provider "aws" {
  count                               = length(var.awsEnvironments)
  profile                             = var.awsEnvironments[count.index]
  alias                               = var.awsEnvironments[count.index]
  region                              = var.awsRegion[var.awsEnvironments[count.index]]

  ignore_tags {
    key_prefixes                      = ["Core:"]
  }
}

module "peering" {
  source                              = "../tf-source/vpc/peering"
  for_each = var.project

  providers = {
    aws.local = each.value.localEnvironment
    aws.peer = each.value.peerEnvironment
  }

  localEnvironment                    = each.value.localEnvironment
  peerEnvironment                     = each.value.peerEnvironment

  localSuffix                         = each.value.localVPCSuffix
  peerSuffix                          = each.value.peerVPCSuffix

}

In this case, the module "../tf-source/vpc/peering" would be expecting two AWS Providers, local and peer.

[ajbouh]

@DonBower AFAIK, alias is allowed in the providers config for modules

[DonBower]

@DonBower AFAIK, alias is allowed in the providers config for modules

It is. but not as a variable....

Error: Variables not allowed

  on provider.tf line 27, in provider "aws":
  27:   alias                               = var.awsEnvironments[count.index]

Variables may not be used here.

original comment updated....

[kuwas]

@ajbouh @DonBower
Yes, you can pass aliased providers into modules, but the providers and their aliases are still statically defined. Furthermore, when using for_each with modules, you must pass in the same provider instance to every module instance within the loop.

I recall reading a comment from @apparentlymart on the reasoning around this current limitation.

Example 1: this does not work

provider "azurerm" {
  for_each = toset(["one", "two"])
  alias    = each.value
}
module "test" {
  for_each  = toset(["one", "two"])
  providers = { azurerm = azurerm[each.value] }
}

Example 2: this also does not work

provider "azurerm" { alias = "one" }
provider "azurerm" { alias = "two" }
module "test" {
  for_each  = toset(["one", "two"])
  providers = { azurerm = azurerm[each.value] }
}

Example 3: this does work

provider "azurerm" { alias = "one" }
provider "azurerm" { alias = "two" }
module "test-one" {
  for_each  = toset(["one"])
  providers = { azurerm = azurerm.one }
}
module "test-two" {
  for_each  = toset(["two"])
  providers = { azurerm = azurerm.two }
}

[oliverlucas85]

I've literally tried to this exact same thing and have been unable.

module "peering" {
  for_each = var.vnet_peering_config
  source   = "./config/network_peering"
  providers = {
    azurerm.custom = "azurerm.${each.value.provider_alias}"
  }

  resource_group_name = module.resource_group.rg.name
  ne_virtual_network  = azurerm_virtual_network.default
  we_virtual_network  = azurerm_virtual_network.default

  remote_virtual_network_name = each.virtual_network_name
  remote_resource_group_name  = each.resource_group_name
}

I'm actually pretty upset that I cant do it, as with peering each "peer" would need to go into another subscription!

Sigh.

[efernandes-dev-ops]

Any updates if this is on the roadmap?

[TulyOpt]

Hi,

Passing a provider as variable would be very useful, is it doable ?

[augustgerro]

Any updates? Also depends from this ...
For AWS provider the only way to use multi-regions for me is dublicate code many times

provider "aws" {
	  alias = "euwestprovider"
	  region = "eu-west-1"
	  ...
}
	
provider "aws" {
	  alias = "eucentralprovider"
	  region = "eu-central-1"
	  ...
}
  
	resource "aws_instance" "ec2-eu-west"{
	  provider = "aws.euwestprovider"
	  ami = "ami-030dbca661d402413"
	  instance_type = "t2.nano"
	}
	
	resource "aws_instance" "ec2-eu-central" {
	  provider = "aws.eucentralprovider"
	  ami = "ami-0ebe657bc328d4e82"
	  instance_type = "t2.nano"
	}

[Lroca88]

Any updates from the team?
This thread has been opened since 2019, please give it some attention.

[ced3eals]

+1 on the topic.
Can you give us some feedback? We are using multiple accounts in my company, and it will be great to be able to loop through organization accounts to create the same resources. for_each in providers block is really needed.
Thanks.

[dhdersch]

In my humble but perhaps irrelevant opinion, Terraform 1.0 shouldn't be released until this feature is implemented.

[egeexyz]

Hey all 👋 - I found a solution workaround that works well enough to create multiple sets of S3 buckets (or any resource) across multiple regions using multiple providers and modules. It's not as clean as provider interpolation would be, but it does work.

An example using two S3 buckets in different regions:

provider "aws" {
  alias   = "use1"
  region  = "us-east-1"
}

provider "aws" {
  alias   = "use2"
  region  = "us-east-2"
}

module "s3_use1" {
  source    = "./s3s"
  providers = {
    aws.region = aws.use1
  }
  bucket_name = "my-bucket-in-use1"
}

module "s3_use2" {
  source    = "./s3s"
  providers = {
    aws.region = aws.use2
  }
  bucket_name = "my-bucket-in-use2"
}

in ./s3s:

provider "aws" {
  alias = "region"
}

variable "bucket_name" {
}

resource "aws_s3_bucket" "my_s3_bucket" {
  bucket   = var.bucket_name
  provider = aws.region
}

This solution was inspired by this Stack Overflow answer.

[gswallow]

Big +1 to get this fixed please. Just dealing with the available regions is hard; I can't imagine having to deal with this for multiple AWS accounts.

[vierkean]

Hello,
I finally took the time to update from terraform version 0.11 to the latest version.
Now I have to realize that this is not so easy because in version 0.11 we pass provider alias as variable.

With which version will this be possible again?

[freakinhippie]

Hello, I finally took the time to update from terraform version 0.11 to the latest version. Now I have to realize that this is not so easy because in version 0.11 we pass provider alias as variable.

With which version will this be possible again?

@vierkean this should probably be opened as question on the hashicorp developer forums. However, providers within modules can be defined using the configuration_aliases directive in the required_providers block. See the docs here.

There is a notable exception to the legacy passing providers by alias which is that now all providers defined in the configuration_aliases must be explicitly passed into the module calls rather than inherited implicitly.

[deepbrook]

For anyone looking for a workaround, there's really only one that made sense to me:

Multiple invocations of the same plan with different inputs for each provider.

I run my configuration in a CI/CD job matrix, each with different provider credentials/configs for the terraform command's environment.

IMO, completely feasible for most use-cases. And I think in cases where it isn't, there's probably a code smell somewhere which could be taken care of with a little refactoring.

[kahawai-sre]

@deepbrook I'm likely missing something here ... but does that mean you are maintaining a single deployment (plan/apply/state file) per distinct provider instance in your environment?
If that is the case, and if there are approvals etc for each plan, does that not become a burden as the number of pipeline runs (alias configurations) scales?
Also, does that approach not make it harder to orchestrate resource dependencies across multiple target deployment scopes i.e. through a single deployment?

[Mike-Nahmias]

@kahawai-sre I don't use a job matrix like @deepbrook mentioned but I basically took the same approach. Yes, for me I have a separate deployment/state per provider instance (different regions). We haven't had any issues so far as we scale. For handling resource dependencies I use Terragrunt. I use it for other stuff too but it makes it really easy to handle the inter-deployment dependency piece.

[deepbrook]

@deepbrook I'm likely missing something here ... but does that mean you are maintaining a single deployment (plan/apply/state file) per distinct provider instance in your environment? If that is the case, and if there are approvals etc for each plan, does that not become a burden as the number of pipeline runs (alias configurations) scales? Also, does that approach not make it harder to orchestrate resource dependencies across multiple target deployment scopes i.e. through a single deployment?

@kahawai-sre, that's exactly what that means, yes :)

My deployments do not run at a big scale, to be fair. As for approvals, I do not have those - I deploy infrastructure to preprod and run tests to ensure all features required are reachable/configured correctly, and then deploy them.

I do have some dependencies between the deployments; since my infrastructure isn't managed in a single repository, but several ones (basic platform infra like networking and IAM roles in one, k8s cluster and database in another one each, etc) It's only matter of triggering down-stream pipelines (I like pipelines :P ).

@Mike-Nahmias approach using terragrunt sounds like a good alternative if you're a mono repo kinda person. :)

[dbhaigh]

Same issue - New day... I have a load of terraform deploying resources to several environments, each requiring their own provider alias

I want to dedupe this lovely collection of n00dles to a single resource block with a for_each loop that iterates over a local or var that provides the required provider alias for each resource

I don't want to have to use modules for this as I feel it's an ugly way of doing this and will make things far harder and more complicated than the how I'm doing it now (separating each resource block, and any associated resources, out into it's own file), which makes it very easy to deploy new resources of the same type (ctrl-C, Ctrl-V - change the names of the guilty, and et viola! there you are)

What I want is to be able to have something like this

resource "resource_type" "resource_name" {
for_each = local/var.resource_values
provider = each.provider
...
}

Without being told that "each" is an invalid provider reference

This is a major hole, and an old hole - one that should've been backfilled in years ago

Can we please get some action on this? and save me from having to go down the ugly bunny hole of modules?

TIA

[Poltergeisen]

700 upvotes on this makes me think that it's something we should at least get some sort've update / explanation on why this isn't possible.

A ton of the Terraform configurations just "work" with this type of syntax, why not providers as well?

[nitoxys]

700 upvotes on this makes me think that it's something we should at least get some sort've update / explanation on why this isn't possible.

A ton of the Terraform configurations just "work" with this type of syntax, why not providers as well?

There's another request like this, but the developer stated that providers are loaded on init and it would have to be rearchitected. Technically provider_override could be an option within a provider.

[osterman]

Fwiw, we've solved this problem using atmos with Terraform and regularly deploy Terraform root modules (what we call components) to dozens or more of accounts in every single AWS region (e.g. for compliance baselines), without HCL code generation. IMO, even if this requested feature existed in terraform, we would not use it because it tightly couples the terraform state to multiple regions, which not only breaks the design principle for DR (that regions share nothing), but makes the blast radius of any change massive and encourages terralythic root module design. In our design pattern, we instantiate a root module once per region, ensuring that each instantiation is decoupled from the other. The one exception to this is when we set up things like transit gateways (with hubs and spokes), then we declare two providers, so we can configure source and target destinations. This ensures no two gateways are tightly coupled to each other.

TL;DR: I acknowledge why at first glance not supporting multiple providers in a loop seems like an awful limitation and why some things would be simplified if it were supported; that said, we deploy enormous infrastructures at Cloud Posse in strict enterprise environments, and don't feel any impact of this inherent limitation.

[yves-vogl]

I could not agree more to @osterman

I'm following this thread since a long time and I also came here because I thought that I needed this feature. Meanwhile I think that relying on a feature like this could possibly be a sign of a bad design. As @osterman said - just think of the enourmous blast radius.

One could argue that there are certain cases where you e.g. want to dynamically create provider instances e.g. in an AWS organization to assume the OrganizationAccountAccessRole for bootstrapping things. But once you think about managing the whole lifecycle of hundreds of those accounts you'll realize that this does not scale. Instead I now use AFT with AWS Control Tower to customize those accounts.

And maybe a few words on all the hate against Hashicorp. I also think that their pricing model
is insane and from the times I had contact with the sales team (and even afterwards with their legal team) I can tell you that they somehow
overestimate what they bring to the table as there are valuable alternatives nowadays. But please also keep in mind that there are lot of companies which make thousands of dollars each day without giving anything back to the community.
Everyone who demands features should ask himself how much he's giving back in form
of contributions or donations before starting rude demands and spreading hate.

Yes, Hashicorp is bad at communication. And in my opinion they somehow now try to squeeze a lot of money out of their customers. But please do not forget what their impact on our community had been. I remember the old days back in around 2010 when @mitchellh created Vagrant. It was the first time we could manage those VM in a way which did not suck.

Having said all the those things… I understand the demand and I feel that the lack of communication from Hashicorp contributed to this situation.

They better could have pointed out the reasons this feature is not available and demonstrate how to solve this problem by proper design.

Love matters,
Yves

[afrazkhan]

There's another use case which I think hasn't been mentioned: Optional resource creation within a module.

I have a module which mostly provisions AWS resources, but there's a single Github resource that is created conditionally (if a particular key is present in one of the module's variable objects). It's likely that many of the people using the module aren't using Github, or don't need this particular feature / resource, so I don't want to add it to the required providers block.