Verify terraform configuration files matches state file contents

[osintalex]

Terraform Version

v1.3.9

Use Cases

When your state file doesn't match what's in your configuration files.

In practice I think this would come up when managing drift on projects that haven't been maintained in a while. I came across the issue while working on some legacy stuff that hadn't been touched in a few years. I had this problems:

Lots of the stuff in terraform configuration files didn't match what was in the state file. I think this is probably because the state file was old or in a remote backend somewhere that got lost. Perhaps this is a rare issue but it feels to me like the sort of thing which might come up often when inheriting legacy projects.

To solve this I was doing a lot of importing and I noticed that this could create some issues, f/e:

Suppose I had a block like this:

resource "aws_security_group" "allow_tls" {
  name        = "allow_tls"
  description = "Allow TLS inbound traffic"
  vpc_id      = aws_vpc.main.id

  ingress {
    description      = "TLS from VPC"
    from_port        = 443
    to_port          = 443
    protocol         = "tcp"
    cidr_blocks      = [aws_vpc.main.cidr_block]
    ipv6_cidr_blocks = [aws_vpc.main.ipv6_cidr_block]
  }

  egress {
    from_port        = 0
    to_port          = 0
    protocol         = "-1"
    cidr_blocks      = ["0.0.0.0/0"]
    ipv6_cidr_blocks = ["::/0"]
  }

  tags = {
    Name = "allow_tls"
  }
}

But in my state file, the egress cidr_blocks value was actually cidr_blocks = ["10.0.0.1/0"], which is also what is actually in this security group in the remote infrastructure. If I run terraform plan -refresh-only this shows the infrastructure as being up to date, even though what's in my configuration block is wrong.

Now imagine this on dozens of files - I think it would be useful to have a way of validating that my configuration code matches what's in state. Perhaps something like terraform compare or terraform validate -check-state.

Apologies if I've missed an easy way of doing that which exists already - I couldn't find one!

Attempted Solutions

To get round this I was doing the following:

1. Initially running terraform plan -refresh-only but AFAIK this checks based on what's in my state file. So if my state file is correct and my configuration is wrong, then this passes.
2. To try and spot the manual configuration errors, I would runterraform state show <resource identifier> and then manually comparing that with what's in the configuration file.

Proposal

I think (hopefully) an implementation for this could be simple:

For every configuration block...
Get the corresponding block in the state file (i.e. output of terraform state show <resource identifier>...
Check if the details in the configuration block match what's in the relevant state file section. For my above example that would mean check that cidr_blocks = ["0.0.0.0/0"] is equal to what the egress cidr blocks value is in the state file along with all the other attributes, i.e description, name, etc.

If they don't, raise a helpful error.

Happy to try a PR for this if you think it would be helpful.

Last thought - I wonder if this irrelevant given existing discussions around auto-generating config files from terraform import since that would also solve this issue, although that would require deleting your existing config in a situation such as the one I've run into.

References

Issue 15608

[jbardin]

Hi @osintalex,

What you're describing is essentially what terraform plan does, except instead of an error, it shows the difference between the current state and the proposed new state. Seeing how the goal of terraform is to modify the infrastructure to match what is declared in the configuration, this is the most natural workflow for handling changes present in the configuration. Can you expand on why the normal plan output is not sufficient for your use case?

Thanks!

[osintalex]

Hey,

Ah yeah sorry, I should have made the difference clearer.

The end goal for what I'm trying to do is different. I'm in a situation where I know many of the configuration files are incorrect, so I'm looking for a way to (as much as possible) programmatically rewrite them.

I realised I missed out a few steps. This is what my workflow looks like at present:

1. Identify configuration file that is incorrect through manual inspection, i.e. I know it has the wrong IPs in an AWS security group like the example above.
2. Run terraform import to import the remote resource to local state, since I know the remote resource is correct as it's been running in production for a while.
3. Run terraform state show to get the imported resource in a terraform code block.
4. Manually refactor that code block in a configuration file so it's a sensible kind of thing to put in a code block, i.e. terraform state show often gives back id like values that aren't allowed.

This is quite time consuming so I was hoping for a better way to do this.

I don't think I can use terraform plan for this. As I understand it, terraform plan assumes that my configuration files describe my desired state. However, here it's actually the other way round - the local state file actually describes my desired state because that's where the correct information about the resource is. It's correct because of step 2 above.

I guess what I would really like is an 'inverse' version of terraform plan, where it gives me a plan for rewriting the configuration files to match the local state.

[jbardin]

Thanks @osintalex,

Improving the import workflow is something we're working on already, so this seems inline with the current development. Showing the possible configuration values derived from the current state, in a format other than the plan changes, is something we want to have the ability to do. It is not possible for Terraform to know exactly what the config should look like, but a starting value can definitely be generated, and subsequent calls to plan can be used to refine that value as it is incorporated into the configuration as a whole.

[osintalex]

Thanks! A starting value without disallowed characteristics (such as owner_id for an aws_security_group) would be super helpful.