Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

v0.13.2 fails to import resources when interpolation is used in provider in module #26211

Closed
demonemia opened this issue Sep 10, 2020 · 23 comments
Labels
bug confirmed a Terraform Core team member has reproduced this issue import Importing resources v0.13 Issues (primarily bugs) reported against v0.13 releases

Comments

@demonemia
Copy link

demonemia commented Sep 10, 2020

Terraform Version

Terraform v0.13.2
+ provider registry.terraform.io/hashicorp/azurerm v2.26.0
+ provider registry.terraform.io/hashicorp/kubernetes v1.13.1

Terraform Configuration Files

Sample Repo

Debug Output

Log file

Expected Behavior

Terraform should import the existing resource into the state file

Actual Behavior

Statefile is not updated, error message is displayed.

> terraform import module.resource_group_1.azurerm_resource_group.resource_group /subscriptions/00000000-0000-0000-0000-000000000000/resourcegroups/resource-group-1

module.resource_group_1.azurerm_resource_group.resource_group: Importing from ID "/subscriptions/00000000-0000-0000-0000-000000000000/resourcegroups/resource-group-1"...
module.resource_group_1.azurerm_resource_group.resource_group: Import prepared!
  Prepared azurerm_resource_group for import
module.resource_group_1.azurerm_resource_group.resource_group: Refreshing state... [id=/subscriptions/00000000-0000-0000-0000-000000000000/resourcegroups/resource-group-1]

Error: Invalid provider configuration

  on resource_group_1/resource_group_1.tf line 17:
  17: provider kubernetes {

The configuration for
module.resource_group_1.provider["registry.terraform.io/hashicorp/kubernetes"].kubernetes_cluster
depends on values that cannot be determined until apply.

Steps to Reproduce

terraform init
terraform import module.resource_group_1.azurerm_resource_group.resource_group /subscriptions/00000000-0000-0000-0000-000000000000/resourcegroups/resource-group-1
or
terraform import module.resource_group_1.null_resource.test_resource test

Additional Context

terraform plan works

This was also an issue in terraform v0.13.0.

References

#25816

@demonemia demonemia added bug new new issue not yet triaged labels Sep 10, 2020
@demonemia demonemia changed the title v0.13.0 fails to import resources when interpolation is used in provider in module v0.13.2 fails to import resources when interpolation is used in provider in module Sep 10, 2020
@kimor79
Copy link

kimor79 commented Sep 14, 2020

i think this is similar to what i'm experiencing in #25574

@alisdair
Copy link
Member

alisdair commented Sep 18, 2020

Thanks for reporting this, @demonemia, and thanks for the reproduction repository. I'm able to reproduce the issue with Terraform 0.13.3.

I'm not sure what the cause is, but there are several other similar import bugs due to the import process not fully evaluating the configuration. Possibly related: #26258.

@alisdair alisdair added confirmed a Terraform Core team member has reproduced this issue import Importing resources and removed new new issue not yet triaged labels Sep 18, 2020
@danieldreier danieldreier added the v0.13 Issues (primarily bugs) reported against v0.13 releases label Sep 29, 2020
@jdtommy
Copy link

jdtommy commented Oct 5, 2020

Still going on in 13.4 with AWS and trying to import iam roles and route 53 zones

@lkwg82
Copy link

lkwg82 commented Oct 6, 2020

@jdtommy Did it work?

Is there some workaround?

@fernandoiury
Copy link

fernandoiury commented Oct 7, 2020

I had a similar issue, but with kubernetes provider

terraform import module.XXXXXXX.kubernetes_namespace.istio_system istio-system

Error: Invalid provider configuration

  on ../../modules/XXXXXXX/main.tf line 22:
  22: provider "kubernetes" {

The configuration for
module.XXXXXXXXXX.provider["registry.terraform.io/hashicorp/kubernetes"].YYYYYYY
depends on values that cannot be determined until apply.

And my workaround was changing the provider from

provider "kubernetes" {
  load_config_file = false
  version          = " ~> 1.11.0"

  host                   = "https://${module.XXXXXXXXXX.endpoint}"
  cluster_ca_certificate = base64decode(module.XXXXXXXX.cluster_ca_certificate)
  token                  = module.XXXXXXXXXXX.access_token
}

to

provider "kubernetes" {
  load_config_file = true
  version          = " ~> 1.11.0"
}

the import worked and I switched the code back after.

@cloud-rocket
Copy link

cloud-rocket commented Oct 9, 2020

@fernandoiury, same workaround did not work for me

@swetli
Copy link

swetli commented Oct 14, 2020

Just had the same issue with kubernetes and helm providers. My workaround was to temporarily comment the entire "offending" provider blocks, do the import then uncomment them back.
Normal terraform operations were possible after that.

@lkwg82
Copy link

lkwg82 commented Oct 14, 2020

Just had the same issue with kubernetes and helm providers. My workaround was to temporarily comment the entire "offending" provider blocks, do the import then uncomment them back.
Normal terraform operations were possible after that.

This seems to me very suspicious. Is it another bug?

@swetli
Copy link

swetli commented Oct 14, 2020

This seems to me very suspicious. Is it another bug?

No idea but I did get the errors:

The configuration for provider["registry.terraform.io/hashicorp/kubernetes"]
depends on values that cannot be determined until apply.

...

The configuration for provider["registry.terraform.io/hashicorp/helm"]
depends on values that cannot be determined until apply.

Both providers use interpolation with the base64decode function for the certificates

@brikis98
Copy link
Contributor

brikis98 commented Oct 15, 2020

Same issue here. We have a module foo with a nested provider block, and that provider block depends on a resource (note: code snippet below is simplified):

provider "aws" {
  alias  = "logs"
  assume_role {
    role_arn = "arn:aws:iam::${aws_organizations_account.child_accounts["logs"].id}:role/foo"
  }
}

We use module foo in some code, along with several other modules such as bar:

module "foo" {
  source = "../foo"
  # ... params ...
}

module "bar" {
  source = "../bar"
  # ... params ...
}

plan and apply work fine. However, when I try to run import on a resource in bar (i.e., something totally unrelated to foo) with Terraform 0.13.4, I get this error:

$ terraform import 'module.bar.aws_iam_user.xxx' 'xxx'
module.bar.aws_iam_user.xxx xxx
module.bar.aws_iam_user.xxx: Importing from ID "xxx"...
module.bar.aws_iam_user.xxx: Import prepared!
  Prepared aws_iam_user for import
module.bar.aws_iam_user.xxx: Refreshing state... [id=xxx]

Error: Invalid provider configuration

  on ../../modules/foo/main.tf line 59:
  59: provider "aws" {

The configuration for
module.foo.provider["registry.terraform.io/hashicorp/aws"].logs
depends on values that cannot be determined until apply.

Note that import worked fine with Terraform 0.12.29. Also, commenting out the assume_role makes import work.

@brikis98
Copy link
Contributor

brikis98 commented Oct 15, 2020

BTW, this looks fairly similar to #13018, although it seems 0.13 has made that issue even worse.

brikis98 added a commit to gruntwork-io/terragrunt that referenced this issue Oct 15, 2020
I have come across yet another `import` bug with Terraform. The [original one](hashicorp/terraform#13018) prompted us to add the `aws-provider-patch` command. The [new one](hashicorp/terraform#26211), which is the motivation for this PR, requires that the `aws-provider-patch` command can override not only top-level attributes in the `provider` block, but also attributes nested within blocks within the `provider` block, such as a `role_arn` within an `assume_role` block. This PR allows you to specify these nested attributes using a "dot" notation: e.g., `--terragrunt-override-attr assume_role.role_arn=""`.
@brikis98
Copy link
Contributor

brikis98 commented Oct 15, 2020

Note that, as an ugly/hacky workaround for #13018, we added an aws-provider-patch command to Terragrunt, as described here. I'm now updating that command to support nested blocks in this PR as a workaround for the new issue described here.

@brikis98
Copy link
Contributor

brikis98 commented Oct 15, 2020

OK, the updated terragrunt awss-provider-patch workaround is now available in v0.25.4.

@dalen
Copy link

dalen commented Oct 22, 2020

terraform 0.13.5 seems to have fixed this for plan & apply, but not for import

@katlimruiz
Copy link

katlimruiz commented Nov 19, 2020

I had a similar issue, but with kubernetes provider

terraform import module.XXXXXXX.kubernetes_namespace.istio_system istio-system

Error: Invalid provider configuration

  on ../../modules/XXXXXXX/main.tf line 22:
  22: provider "kubernetes" {

The configuration for
module.XXXXXXXXXX.provider["registry.terraform.io/hashicorp/kubernetes"].YYYYYYY
depends on values that cannot be determined until apply.

And my workaround was changing the provider from

provider "kubernetes" {
  load_config_file = false
  version          = " ~> 1.11.0"

  host                   = "https://${module.XXXXXXXXXX.endpoint}"
  cluster_ca_certificate = base64decode(module.XXXXXXXX.cluster_ca_certificate)
  token                  = module.XXXXXXXXXXX.access_token
}

to

provider "kubernetes" {
  load_config_file = true
  version          = " ~> 1.11.0"
}

the import worked and I switched the code back after.

This workaround worked for me, I was able to import my kubernetes clusters.

Terraform v0.13.5

  • provider registry.terraform.io/cloudflare/cloudflare v2.13.2
  • provider registry.terraform.io/hashicorp/azuread v1.0.0
  • provider registry.terraform.io/hashicorp/azurerm v2.36.0
  • provider registry.terraform.io/hashicorp/helm v1.3.2
  • provider registry.terraform.io/hashicorp/kubernetes v1.13.3
  • provider registry.terraform.io/hashicorp/time v0.6.0

@lkwg82
Copy link

lkwg82 commented Nov 23, 2020

Has anyone a workaround for aws?

@katlimruiz
Copy link

katlimruiz commented Nov 24, 2020

In the end, the solution I found better was to separate the terraform states across several steps (indeed harder but it does solves many problems).

In summary, it would be to

  1. Generate your first terraform state which contains your kubernetes cluster creation.
  2. After the cluster definition, include a local_file generation with the kubernetes config.
  3. In your second terraform state, load the file and configure the provider with it.

I will try to create a medium article for it, but in summary these are the steps that I've come up with for a long term solution.

hulquest pushed a commit to NetApp/terragrunt that referenced this issue Dec 2, 2020
I have come across yet another `import` bug with Terraform. The [original one](hashicorp/terraform#13018) prompted us to add the `aws-provider-patch` command. The [new one](hashicorp/terraform#26211), which is the motivation for this PR, requires that the `aws-provider-patch` command can override not only top-level attributes in the `provider` block, but also attributes nested within blocks within the `provider` block, such as a `role_arn` within an `assume_role` block. This PR allows you to specify these nested attributes using a "dot" notation: e.g., `--terragrunt-override-attr assume_role.role_arn=""`.
@jason-huling
Copy link

jason-huling commented Dec 3, 2020

I was able to get a workaround for my scenario in aws, hopefully it will help someone else until this is resolved.

I have a module aws-account that provisions new accounts with aws_organizations_account and then uses an aws provider with string interpolation for the assume role, taking the account id from aws_organizations_account. High level looks something like this:

module/aws-account main.tf + providers.tf contents:

resource "aws_organizations_account" "this" {
  # resource content...
}

provider "aws" {
  alias  = "this"
  region = "us-west-2"
  assume_role {
    role_arn = "arn:aws:iam::${aws_organizations_account.this.id}:role/OrganizationAccountAccessRole"
  }
}

# bootstrap some account resources with `aws.this` provider.

I can't really comment out this provider or the assume role without effecting other consumers of the module, so I did the following.

  1. Initial state
# Need to import into 'module.aws_account.aws_organizations_account.this'
module "aws_account" {
  source = "module/aws-account"
  # ...
}
  1. Comment out the module and define a temporary top level resource for aws_organizations_account
resource "aws_organizations_account" "temp" {}

# module "aws_account" {
#   source = "module/aws-account"
#   # ...
# }
  1. Import into the top level resource:
terraform import aws_organizations_account.temp xxxxxxxxxxxx
  1. Move the state from the top level resource into the module:
terraform state mv aws_organizations_account.temp module.aws_account.aws_organizations_account.this
  1. Done, revert code back to step 1

Hope it helps.

@donspablo
Copy link

donspablo commented Jan 31, 2021

Same issue here - I am using Terraform and Helm to deploy to GoogleCloud, I have got a couple ideas from this thread, thanks for sharing.

provider "google" {
  project = var.project
  region  = var.region
}

data "google_client_config" "provider" {}

provider "kubernetes" {
  load_config_file = "false"
  host                   = module.kubernetes.cluster.endpoint
  token                  = data.google_client_config.provider.access_token
  cluster_ca_certificate = base64decode(module.kubernetes.cluster.master_auth[0].cluster_ca_certificate)
}

provider "helm" {
  kubernetes {
    host                   = module.kubernetes.cluster.endpoint
    token                  = data.google_client_config.provider.access_token
    cluster_ca_certificate = base64decode(module.kubernetes.cluster.master_auth[0].cluster_ca_certificate)
  }
}

Exception:

`Terraform 0.11 and earlier required all non-constant expressions to be
provided via interpolation syntax, but this pattern is now deprecated. To
silence this warning, remove the "${ sequence from the start and the }"
sequence from the end of this expression, leaving just the inner expression.

Template interpolation syntax is still used to construct strings from
expressions when the template includes multiple interpolation sequences or a
mixture of literal strings and interpolations. This deprecation applies only
to templates that consist entirely of a single interpolation sequence.


Error: Unsupported argument

  on providers.tf line 16, in provider "kubernetes":
  16:   load_config_file = "false"

An argument named "load_config_file" is not expected here.

!!
!! terraform failed:
!!      "exit status 1"
!!`

@donspablo
Copy link

donspablo commented Jan 31, 2021

This is working for me!

provider "kubernetes" {
  version                = "~> 1.11.1"
  load_config_file       = false
  ...
}

donspablo added a commit to donspablo/gitpod that referenced this issue Jan 31, 2021
temp fix: terraform-provider-kubernetes
# See hashicorp/terraform#26211 (comment)
version = "~> 1.10.0"
donspablo added a commit to donspablo/gitpod that referenced this issue Jan 31, 2021
temp fix: terraform-provider-kubernetes
# See hashicorp/terraform#26211 (comment)
version = "~> 1.10.0"
@dilloncwheeler
Copy link

dilloncwheeler commented Feb 26, 2021

This issue is supposed to be fixed in the latest release for what it's worth.
#13018 (comment)

@jbardin
Copy link
Member

jbardin commented Mar 23, 2021

Closing as confirmed this case is working in the latest release.

Thanks!

@jbardin jbardin closed this as completed Mar 23, 2021
@ghost
Copy link

ghost commented Apr 23, 2021

I'm going to lock this issue because it has been closed for 30 days . This helps our maintainers find and focus on the active issues.

If you have found a problem that seems similar to this, please open a new issue and complete the issue template so we can capture all the details necessary to investigate further.

@hashicorp hashicorp locked as resolved and limited conversation to collaborators Apr 23, 2021
Sign up for free to subscribe to this conversation on GitHub. Already have an account? Sign in.
Labels
bug confirmed a Terraform Core team member has reproduced this issue import Importing resources v0.13 Issues (primarily bugs) reported against v0.13 releases
Projects
None yet
Development

No branches or pull requests