kubernetes_manifest always shows changes in creationTimestamp

[torbendury]

Terraform Version, Provider Version and Kubernetes Version

Terraform version: v1.1.7
Kubernetes provider version: 2.8.0
Kubernetes version: v1.22.1 (okd)

Affected Resource(s)

• kubernetes_manifest

Terraform Configuration Files

locals {
  crd_version = "v1.15.1"
  crds = [
    "https://raw.githubusercontent.com/fission/fission/${local.crd_version}/crds/v1/fission.io_canaryconfigs.yaml",
    "https://raw.githubusercontent.com/fission/fission/${local.crd_version}/crds/v1/fission.io_environments.yaml",
    "https://raw.githubusercontent.com/fission/fission/${local.crd_version}/crds/v1/fission.io_functions.yaml",
    "https://raw.githubusercontent.com/fission/fission/${local.crd_version}/crds/v1/fission.io_httptriggers.yaml",
    "https://raw.githubusercontent.com/fission/fission/${local.crd_version}/crds/v1/fission.io_kuberneteswatchtriggers.yaml",
    "https://raw.githubusercontent.com/fission/fission/${local.crd_version}/crds/v1/fission.io_messagequeuetriggers.yaml",
    "https://raw.githubusercontent.com/fission/fission/${local.crd_version}/crds/v1/fission.io_packages.yaml",
    "https://raw.githubusercontent.com/fission/fission/${local.crd_version}/crds/v1/fission.io_timetriggers.yaml",
  ]
}

data "http" "fission-crd" {
  for_each = toset(local.crds)
  url      = each.key
}


resource "kubernetes_manifest" "fission-crd" {
  for_each = { for http in data.http.fission-crd : http.url => http }
  manifest = { for k, v in yamldecode(each.value.body) : k => v if !(length(regexall("status", k)) > 0) }
}

Debug Output

none so far, unsure if needed

Panic Output

none so far

Steps to Reproduce

1. insert above block and create the resources
2. run a terraform plan afterwards

Expected Behavior

Since nothing has changed in the YAML manifests, I expect the terraform plan to exit without showing changes

Actual Behavior

Terraform always wants to modify the resource:

  # kubernetes_manifest.fission-crd["https://raw.githubusercontent.com/fission/fission/v1.15.1/crds/v1/fission.io_messagequeuetriggers.yaml"] will be updated in-place
  ~ resource "kubernetes_manifest" "fission-crd" {
      ~ object   = {
          ~ metadata   = {
              ~ creationTimestamp          = null -> (known after apply)
                # (15 unchanged elements hidden)
            }
            # (3 unchanged elements hidden)
        }
        # (1 unchanged attribute hidden)
    }

  # kubernetes_manifest.fission-crd["https://raw.githubusercontent.com/fission/fission/v1.15.1/crds/v1/fission.io_packages.yaml"] will be updated in-place
  ~ resource "kubernetes_manifest" "fission-crd" {
      ~ object   = {
          ~ metadata   = {
              ~ creationTimestamp          = null -> (known after apply)
                # (15 unchanged elements hidden)
            }
            # (3 unchanged elements hidden)
        }
        # (1 unchanged attribute hidden)
    }

  # kubernetes_manifest.fission-crd["https://raw.githubusercontent.com/fission/fission/v1.15.1/crds/v1/fission.io_timetriggers.yaml"] will be updated in-place
  ~ resource "kubernetes_manifest" "fission-crd" {
      ~ object   = {
          ~ metadata   = {
              ~ creationTimestamp          = null -> (known after apply)
                # (15 unchanged elements hidden)
            }
            # (3 unchanged elements hidden)
        }
        # (1 unchanged attribute hidden)
    }

Important Factoids

I'm running OKD 4.9, I did not have time yet to try and reproduce it e.g. on a GKE, AKS or something.

References

• none that I'm aware of

Community Note

• Please vote on this issue by adding a 👍 reaction to the original issue to help the community and maintainers prioritize this request
• If you are interested in working on this issue or have submitted a pull request, please leave a comment

[alexsomesan]

Hi! The problem here is that the provider strips creationTimestamp from all manifests because it's an attribute that produces unexpected diffs and it's not user actionable.
You can try to resolve this issue by removing the creationTimestamp attribute from the object that you get out of yamldecode.

However, we recommend all our users to convert the yaml manifests to HCL offline, using the tfk8s tool. This is the preferred workflow for kubernetes_manifest as it allows full use of Terraform's HCL language to customize the resources before they are applied.

[vladimirtiukhtin]

@alexsomesan this does not apply to CRDs, doesn't it? What is an advice for CRD users?

[jeffbeal-scale]

I had a CRD written with an HCL map and was having this issue. The previous author had written creationTimestamp = null in the map; removing that fixed the issue:

resource "kubernetes_manifest" "originissuers_crd" {
  manifest = {
    apiVersion = "apiextensions.k8s.io/v1"
    kind       = "CustomResourceDefinition"
    metadata = {
      creationStamp = null #Removing this line and applying resolved the diff
      name = "originissuers.cert-manager.k8s.cloudflare.com"
    }

[kierdavis]

Can confirm that tfk8s emitted creationTimestamp = null lines when fed with some CRDs in YAML form, and that removing these lines and running a terraform apply made the spurious changes go away.

[vlad-ivanov-name]

For those using kustomization provider, you can hack your way around without manually editing the generated data structures with the following expression

  manifest = merge(
    { for k, v in each.value : k => v if k != "status" && k != "metadata" },
    contains(keys(each.value), "metadata") ?
      { metadata : { for m_k, m_v in each.value["metadata"] : m_k => m_v if m_k != "creationTimestamp" } } :
      {}
  )

[TOTom]

Hi, I encounter this question too. all my "CRD" need modification every time because of field "object.metadata.creationTimestamp". i also add " metadata.creationTimestamp" in the computed_field , but no help. hope this bug can be fixed soon.

~ resource "kubernetes_manifest" "prometheus_operator_crds" {

  ~ object          = {
      ~ metadata   = {
          + creationTimestamp          = (known after apply)
            name                       = "alertmanagerconfigs.monitoring.coreos.com"
            # (14 unchanged attributes hidden)
        }
        # (3 unchanged attributes hidden)
    }
    # (1 unchanged attribute hidden)
}

[vladimirtiukhtin]

As suggested by @alexsomesan, I converted yamls offline removing all occurrences of creationTimestamp and it worked. I guess having resources offline is the way forward and is in fact more robust