Whitespace changes in heredoc force new resources

[yissachar]

I have the following ECS task:

resource "aws_ecs_task_definition" "web" {
  family = "web"
  container_definitions = <<EOF
[{
  "name": "web",
  "image": "imagelocation",
  "memory": 500,
  "essential": true,
  "portMappings": [
    {
      "containerPort": 80,
      "hostPort": 80
    }
  ]
}]
EOF
}

If I then add an extra space after one of the commas, this will force a new version of the task as Terraform thinks that it has changed.

For my particular situation this causes problems since we have developers on Mac and Windows. Since we have Git set to convert line-endings, this means the local file on Windows uses \r\n for line-endings, and Mac uses \n. Terraform sees that the container_definitions has some changes and forces an update of all the resources involved.

I understand that it might be out of scope for Terraform to handle something like this, but it would be nice if there were some way of dealing with the issue. For now we are forced to restrict Terraform plan and apply usage to one OS.

[jen20]

Hi @yissachar - thanks for opening this issue - I can certainly see the concern here.

One option here is to normalise the line endings to the correct \n format prior to calculating the hash which determines whether there is a change, regardless of operating system. The risk there is that if a HEREDOC is used for a template_file for example, and then used in a provisioner, we might get the line endings incorrect for the target operating system, though this is likely also the case now.

@phinze or @catsby do you have a preferred approach here?

[phinze]

It seems reasonable to me that for the purposes of hashing and diff calculation that we'd normalize line endings, but for content we'd respect the line endings we see.

It would be sort of a big change though, since updating the logic would have the potential to trigger unwanted diffs across all resources for existing users. We'd probably need to write state migration code to prevent this from happening.

I wonder if this is something we could address more directly at the resource level by doing state normalization on this field. @yissachar is aws_ecs_task_definition the only place you're seeing this problem?

[yissachar]

I'm also seeing it on aws_iam_role_policy on the policy field.

[oillio]

I am seeing a similar issue in aws_sqs_queue on the policy field.

I don't know if it is the same issue or not. I have a HEREDOC setup and every time I apply, it wants to update this field. It looks like the state is being stored with all whitespace removed, but the diff is being done against the original version with whitespace.

[Morriz]

+1
Hope this be resolved

[rsalmond]

@oillio in case you haven't seen it yet the sqs policy json is now normalized (#4780), works for me as of 0.6.15.

[mitchellh]

Moving this to an AWS issue, we now have the technology to fix this:

• We need to change the StateFunc to normalize JSON (normalizeJsonString)
• We need a state migration function so that we can do this w/o breaking old states.

[ghost]

I'm going to lock this issue because it has been closed for 30 days ⏳. This helps our maintainers find and focus on the active issues.

If you have found a problem that seems similar to this, please open a new issue and complete the issue template so we can capture all the details necessary to investigate further.