Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Terraform crash issue while using ignore lifecycle change for "aws_wafv2_web_acl" #32410

Closed
AIUPFK1 opened this issue Dec 19, 2022 · 4 comments
Closed
Labels
bug duplicate issue closed because another issue already tracks this problem waiting-response An issue/pull request is waiting for a response from the community

Comments

@AIUPFK1
Copy link

AIUPFK1 commented Dec 19, 2022

Terraform Version

Terraform v1.3.6
on linux_amd64

FYI- I tried using older terraform versions as well like - v1.3.0, v1.1.7 but the same result, also tried latest as well as older version of aws provider i.e 4.47, 4.29, 4.40 etc

Terraform Configuration Files

provider.tf -

terraform {
  required_providers {
    aws = {
      source  = "hashicorp/aws"
      version = "4.47.0"
    }
  }

  backend "s3" {
    profile = "default"
    encrypt = true
  }
}
..
...
...

webacl.tf -

resource "aws_wafv2_web_acl" "example" {
  name        = format("%s%sWebAcl%s", local.resource_name_prefix, title(var.web_acl_name), var.global_suffix)
  description = var.web_acl_description
  scope       = var.web_acl_scope

  default_action {
    block {}
  }

  visibility_config {
    cloudwatch_metrics_enabled = var.web_acl_metrics_enabled
    metric_name                = lower(var.web_acl_name)
    sampled_requests_enabled   = var.web_acl_sampled_requests_enabled
  }

  lifecycle {
    ignore_changes = [
      rule,
      default_action,
      description,
      name
    ]
  }

  tags = merge({
    Name = format("%s%sWebAcl%s", local.resource_name_prefix, title(var.web_acl_name), var.global_suffix)
  }, var.my_tags)
}

module.tf -

module "module-web-acls" {
  source = "../modules/waf-web-acl"

  providers = {
    aws = aws.us-east-1
  }

  for_each                      = var.waf_web_acls
  web_acl_name           = format("%s", title(each.value.web_acl_name))
  web_acl_description = each.value.web_acl_description
}

We have around 10-15 waf_web_acls under waf_web_acls as shown below as a ....(dot dot)
webacl.tfvars -

waf_web_acls = {
  "cloudfront-admin" = {
    web_acl_name        = "Admin"
    web_acl_description = "description-admin"
  }
.
..
...
....
.....

}

Debug Output

Terraform is crashing while running terraform plan, here is the TRACE log for the same, keeping this short to avoid unwanted verboseness.

Tried

..
...
....
2022-12-19T11:55:20.000Z [WARN]  Provider "registry.terraform.io/hashicorp/aws" produced an unexpected new value for module.module-web-acls["cloudfront-admin"].aws_wafv2_web_acl.example during refresh.
      - .description: was cty.StringVal("<removing sensitive info1>"), but now cty.StringVal("<removing sensitive info2>")
      - .lock_token: was cty.StringVal("<removing sensitive info1>"), but now cty.StringVal("<removing sensitive info2>")
      - .default_action[0].allow: block count changed from 0 to 1
      - .default_action[0].block: block count changed from 1 to 0
      - .visibility_config[0].metric_name: was cty.StringVal("<removing sensitive info>"), but now cty.StringVal("<removing sensitive info>")
2022-12-19T11:55:20.034Z [WARN]  Provider "registry.terraform.io/hashicorp/aws" produced an unexpected new value for module.module-web-acls["cloudfront-media"].aws_wafv2_web_acl.example during refresh.
      - .lock_token: was cty.StringVal("<removing sensitive info1>"), but now cty.StringVal("<removing sensitive info1>")
      - .visibility_config[0].metric_name: was cty.StringVal("<removing sensitive info1>"), but now cty.StringVal("<removing sensitive info1>")

!!!!!!!!!!!!!!!!!!!!!!!!!!! TERRAFORM CRASH !!!!!!!!!!!!!!!!!!!!!!!!!!!!

Terraform crashed! This is always indicative of a bug within Terraform.
Please report the crash with Terraform[1] so that we can fix this.

When reporting bugs, please include your terraform version, the stack trace
shown below, and any additional information which may help replicate the issue.

[1]: https://github.com/hashicorp/terraform/issues

!!!!!!!!!!!!!!!!!!!!!!!!!!! TERRAFORM CRASH !!!!!!!!!!!!!!!!!!!!!!!!!!!!

value is null
goroutine 414 [running]:
runtime/debug.Stack()
        /usr/local/go/src/runtime/debug/stack.go:24 +0x65
runtime/debug.PrintStack()
        /usr/local/go/src/runtime/debug/stack.go:16 +0x19
github.com/hashicorp/terraform/internal/logging.PanicHandler()
..
...
....

Expected Behavior

Background info - We are creating aws_wafv2_web_acl resource using terraform, where we are adding a new web ACL into tfvars file for the same.

After successful creation of aws_wafv2_web_acl in AWS acc, if we tried to run terraform plan, this will show us only changes related to desired resources except lifecycle ignore added inwebacl.tf`.

we are having CICD automation where we are rule to already created web_Acl,
hence to avoid drifts we have added lifecycle ignore_changes block inwebacl.tf.

Expected Behavior-

terraform plan will run with some either drifts or no drifts.

Actual Behavior

There are 2 actual behavior observed -

  1. Terraform is undergoing the crash state as shown in the output section above.
  2. VM, where terraform is running going to 100% CPU, uses and kills terraform proc.

when we tried to remove lifecycle, ignore changes block(configuration) from webacl.tfvars, terraform plan will work without crash, although we faced unwanted drifts too.

Steps to Reproduce

  1. terraform init
  2. terraform apply
  3. terraform plan

Additional Context

I have seen and tried a couple of issues in the GitHub issues section regarding the same, stating that updation of terraform AWS provider version / terraform version, but no luck.

My observation -

As we tried to apply rule outside terraform, aws provider version of terraform is crashing to load the plan for the same.

References

No response

@AIUPFK1 AIUPFK1 added bug new new issue not yet triaged labels Dec 19, 2022
@AIUPFK1 AIUPFK1 changed the title Terraform issue while using ignore lifecycle change for "aws_wafv2_web_acl" Terraform crash issue while using ignore lifecycle change for "aws_wafv2_web_acl" Dec 19, 2022
@jbardin
Copy link
Member

jbardin commented Dec 19, 2022

Hi @AIUPFK1,

Thanks for filing the issue. We need the contents of the stack trace which was omitted here to see where the crash may have originated.

Thanks!

@jbardin jbardin added the waiting-response An issue/pull request is waiting for a response from the community label Dec 19, 2022
@AIUPFK1
Copy link
Author

AIUPFK1 commented Jan 2, 2023

tf.log

Hi @jbardin Added TRACE log, please find attachments,

Removed project specific sensitive info such as token, role, Accounts details etc.

please do let me know, if anything needed from my end

@jbardin
Copy link
Member

jbardin commented Jan 3, 2023

Thanks @AIUPFK1, this looks like a duplicate of #31536.

@jbardin jbardin closed this as completed Jan 3, 2023
@crw crw added duplicate issue closed because another issue already tracks this problem and removed new new issue not yet triaged labels Jan 5, 2023
@github-actions
Copy link

github-actions bot commented Feb 5, 2023

I'm going to lock this issue because it has been closed for 30 days ⏳. This helps our maintainers find and focus on the active issues.
If you have found a problem that seems similar to this, please open a new issue and complete the issue template so we can capture all the details necessary to investigate further.

@github-actions github-actions bot locked as resolved and limited conversation to collaborators Feb 5, 2023
Sign up for free to subscribe to this conversation on GitHub. Already have an account? Sign in.
Labels
bug duplicate issue closed because another issue already tracks this problem waiting-response An issue/pull request is waiting for a response from the community
Projects
None yet
Development

No branches or pull requests

3 participants