Error: updating EKS Cluster compute config: No changes needed for EKS Auto Mode configuration provided

[lgenasi-gocity]

Description

After upgrading to v21.3.0 we were unable to apply the latest plan because it's trying to disable Auto Mode when it's already disabled. On applying we got the following error:

│ Error: updating EKS Cluster (common-dev-euw2) compute config: operation error EKS: UpdateClusterConfig, https response error StatusCode: 400, RequestID: ef080ac4-2702-412b-8371-a39333273621, InvalidRequestException: No changes needed for EKS Auto Mode configuration provided

And here was the plan with no changes to our config, the only change being the upgrade to v21.3.0:

  # module.eks_cluster.aws_eks_cluster.this[0] will be updated in-place
  ~ resource "aws_eks_cluster" "this" {
        id                            = "common-dev-euw2"
        name                          = "common-dev-euw2"
        tags                          = {
            "BrandName"             = "common"
            "ClusterName"           = "common-dev-euw2"
            "Environment"           = "dev-euw2"
            "Name"                  = "eks-nodes.common-dev-euw2"
            "terraform-aws-modules" = "eks"
        }
        # (14 unchanged attributes hidden)

      + compute_config {
          + enabled = false
        }

      ~ kubernetes_network_config {
            # (3 unchanged attributes hidden)

          + elastic_load_balancing {
              + enabled = false
            }
        }

      + storage_config {
          + block_storage {
              + enabled = false
            }
        }

        # (4 unchanged blocks hidden)
    }

• ✋ I have searched the open/closed issues and my issue is not listed.

⚠️ Note

Before you submit an issue, please perform the following first:

1. Remove the local .terraform directory (! ONLY if state is stored remotely, which hopefully you are following that best practice!): rm -rf .terraform/
2. Re-initialize the project root to pull down modules: terraform init
3. Re-attempt your terraform plan or apply and check if the issue still persists

Versions

• Module version [Required]: v21.3.0

• Terraform version: 1.13.2

• Provider version(s):
  • hashicorp/external v2.3.5
  • hashicorp/local v2.5.3
  • hashicorp/null v3.2.4
  • hashicorp/random v3.7.2
  • hashicorp/aws v6.13.0

Reproduction Code [Required]

Steps to reproduce the behaviour:

1. Have an EKS cluster provisioned with Auto Mode disabled
2. Upgrade to v21.3.0 of this module
3. Run terraform apply

Expected behavior

Apply to work as nothing has changed. Or some guidance on how to prevent auto mode being disabled when it's already disabled, otherwise we can't upgrade the module version.

Actual behavior

Apply fails because request to disable Auto Mode doesn't work if it's already disabled.

Terminal Output Screenshot(s)

Additional context

[bryantbiggs]

can you share a reproduction

[beiermi]

Same error on our side with 21.3.0.
The same code applied without any issues with 21.1.5 last week.

[bryantbiggs]

replying "same..." won't help - if you have a reproduction you can share, we can dig in further

[caussourd]

can you share a reproduction

@bryantbiggs I'm having exactly the same issue with an existing cluster (code similar to the "EKS Managed Node Group" example in the README). When changing the module version to 21.3, I'm getting the error mentioned in this issue.

[cnocula-peg]

bryantbiggs Reproducible for me.

Existing EKS cluster, but we do not have an EKS Auto Mode enabled. Upgrading to 21.3 is coming up with this change in the plan:

Where the "compute_config" block appears for the first time.

The apply phase fails then:

We reverted back to 21.2.

Note: There is an already open issue on in the provider project. Similar symptoms? hashicorp/terraform-provider-aws#40582

Though please note: They talk about disabling Auto Mode it after it was enabled. On our side, we never hat that enabled at all.

So basically the reproduction is the same as the OP posted:

Steps to reproduce the behaviour:

• Have an EKS cluster provisioned with Auto Mode disabled
• Upgrade to v21.3.0 of this module
• Run terraform apply

[lgenasi-gocity]

@bryantbiggs What do you want me to provide to dig further? As you can see from the initial report the plan trys to set compute_config enabled to false which fails if the EKS cluster has never had Auto Mode enabled. I understand this is an issue with the AWS API, I'm just wondering if there's any parameters I need to pass to the module to get it to just ignore this completely.

[lgenasi-gocity]

Could be wrong but It looks to me like this change in the latest release has made it so that if you don't define a compute_config (which I'm not) it will create a compute_config with the defaults (ie. enabled = false) which is fine but AWS doesn't allow updating, as per the error message in the original report.

It's now creating a compute_config because the array in the for_each has an empty object in it ([{}]) whereas before it was an empty array when compute_config was null.

[bryantbiggs]

Collecting details to share with the EKS service team. For now, pin your module version to v21.2.0

[maksym-butusov]

• 1 getting the same issue

[3H-cphillips]

Collecting details to share with the EKS service team. For now, pin your module version to v21.2.0

This does not work for clusters that already have corrupted state.

Our Experience

• Brand new cluster created on 1.31 (pre-Auto Mode)
• Standard upgrade to 1.32 two weeks ago with node group rotation
• Today: Simply adding a new node group triggers: InvalidParameterException: When Compute Config nodeRoleArn is null or empty, nodePool list cannot be populated

The State Corruption Issue

Once compute_config enters your state with default values (node_pools = ["general-purpose", "system"]), the module keeps trying to manage it even after:

• Downgrading to v21.2.0
• Setting compute_config = null
• Completely omitting compute_config

Required State Surgery

If your state is already corrupted, you must manually clean it:

Pull state, set node_pools to empty array

terragrunt state pull > state.json
jq '(.resources[] | select(.type == "aws_eks_cluster") | .instances[0].attributes.compute_config[0].node_pools) = []' state.json > state-fixed.json

Increment serial and push back

Warning: Even after cleaning, terraform refresh will restore the bad values from AWS, requiring multiple state cleanups or avoiding refresh operations.

TL;DR: Once compute_config poisoning occurs, pinning to v21.2.0 alone won't fix it - manual state intervention is required.

[bryantbiggs]

PR pending on provider - awaiting initial feedback on the approach hashicorp/terraform-provider-aws#44334

[Bo-Taegu]

Same issue
No changes needed for EKS Auto Mode configuration provided

[vatsal-kavida]

Same issue, Commenting so I get notified when the fix is merged