terraform resource doesn't exist when using modules and data resource

[bernadinm]

Community Note

• Please vote on this issue by adding a 👍 reaction to the original issue to help the community and maintainers prioritize this request
• Please do not leave "+1" or "me too" comments, they generate extra noise for issue followers and do not help prioritize the request
• If you are interested in working on this issue or have submitted a pull request, please leave a comment

Terraform Version

$ terraform -v
Terraform v0.11.11
+ provider.aws v1.54.0

Affected Resource(s)

On a brand new deployment, there is an update in place where there should just only be an apply

An execution plan has been generated and is shown below.
Resource actions are indicated with the following symbols:
  + create
  ~ update in-place

Terraform will perform the following actions:

  ~ data.aws_security_group.peer
  ~ data.aws_security_group.this
  + aws_security_group.peer
  + aws_security_group.this

The script then errors with the lines below:

* data.aws_security_group.peer: data.aws_security_group.peer: no matching SecurityGroup found
* data.aws_security_group.this: data.aws_security_group.this: no matching SecurityGroup found

• aws_security_group
• data.aws_security_group

Terraform Configuration Files

locals {
  region_hub = "us-east-1"
  region_spoke_1 = "us-west-2"
}

provider "aws" {
  region = "${local.region_hub}"
}

provider "aws" {
  region = "${local.region_spoke_1}"
  alias  = "remote"
}

resource "aws_vpc" "region1" {
  provider = "aws"
  cidr_block = "10.1.0.0/16"
}

resource "aws_vpc" "region2" {
  provider   = "aws.remote"
  cidr_block = "10.2.0.0/16"
}

resource "aws_security_group" "this" {
  provider    = "aws"
  name        = "test-internal-firewall"
  description = "Allow all inbound traffic"
  vpc_id      = "${aws_vpc.region1.id}"

  ingress {
    from_port   = 0
    to_port     = 0
    protocol    = "-1"
    cidr_blocks = ["0.0.0.0/0"]
  }
}

resource "aws_security_group" "peer" {
  provider    = "aws.remote"
  name        = "test-internal-firewall"
  description = "Allow all inbound traffic"
  vpc_id      = "${aws_vpc.region2.id}"

  ingress {
    from_port   = 0
    to_port     = 0
    protocol    = "-1"
    cidr_blocks = ["0.0.0.0/0"]
  }
}

data "aws_security_group" "this" {
  provider = "aws"
  filter {
    name   = "group-name"
    values = ["*internal-firewall*"]
  }

  filter {
    name   = "vpc-id"
    values = ["${aws_vpc.region1.id}"]
  }
}

data "aws_security_group" "peer" {
  provider = "aws.remote"
  filter {
    name   = "group-name"
    values = ["*internal-firewall*"]
  }

  filter {
    name   = "vpc-id"
    values = ["${aws_vpc.region2.id}"]
  }
}

output "sg_this" {
  description = "security_group_this_region"
  value       = ["${data.aws_security_group.this.id}"]
}

output "sg_peer" {
  description = "security_group_peer_region"
  value       = ["${data.aws_security_group.peer.id}"]
}

Debug Output

terraform plan: https://gist.github.com/bernadinm/52a62dff96fcefb630a7736a15847f3f
terraform apply: https://gist.github.com/bernadinm/c3bbb54e420ead2d13cbc24270b3ec09

The plan shows update in place for the data resource

and the error shows that it doesn't know that the security group was created

Panic Output

Expected Behavior

I expected it to print out the output of the security group

Actual Behavior

It said no security group existed and when you rerun terraform apply, it recognizes it.

Steps to Reproduce

1. terraform apply
2. rerun terraform apply

[bernadinm]

I understand this is a critical racing issue as terraform almost doesn't know when the resource should exist. In this case, I'd recommend seldom use of the data resource and requiring submodules to require specific information from other modules for it to interoperate properly.

[bernadinm]

related to: hashicorp/terraform#17034

[tracypholmes]

Hi, @bernadinm! Thank you for using Terraform and for opening up this question. Issues on GitHub are intended to be related to bugs or feature requests with the provider codebase. Please use https://discuss.hashicorp.com/c/terraform-providers for community discussions, and questions around Terraform.

Closing this in favor of hashicorp/terraform#17034.

If you believe that your issue was miscategorized as a question or closed in error, please create a new issue using one of the following provided templates: bug report or feature request. Please make sure to provide us with the appropriate information so we can best determine how to assist with the given issue.

[ghost]

I'm going to lock this issue because it has been closed for 30 days ⏳. This helps our maintainers find and focus on the active issues.

If you feel this issue should be reopened, we encourage creating a new issue linking back to this one for added context. Thanks!