Elasticache: InvalidParameterCombination: Cannot set snapshotting cluster for cluster mode enabled replication group #6412

ghost · 2018-11-09T17:20:03Z

This issue was originally opened by @ngkuznetsov as hashicorp/terraform#19336. It was migrated here as a result of the provider split. The original body of the issue is below.

Terraform Version

Terraform v0.11.10
+ provider.aws v1.42.0

Terraform Configuration Files

resource "aws_elasticache_replication_group" "ptr_persistence" {
  automatic_failover_enabled    = true
  replication_group_id          = "${var.replication_group_id}"
  replication_group_description = "Redis cluster for PTR"
  node_type                     = "${var.node_type}"
  parameter_group_name          = "${aws_elasticache_parameter_group.ptr_persistence.name}"
  port                          = 6379
  subnet_group_name             = "${aws_elasticache_subnet_group.ptr_persistence.name}"
  security_group_ids            = ["${aws_security_group.ptr_persistence.id}"]
  snapshot_retention_limit      = "35"
  snapshot_window               = "14:19-16:00"
  maintenance_window            = "05:00-06:00"
  cluster_mode {
    num_node_groups             = 1
    replicas_per_node_group     = 2
  }
}

resource "aws_elasticache_parameter_group" "ptr_persistence" {
  name   = "${var.name}"
  family = "redis4.0"
  count = 1
  parameter {
    name  = "cluster-enabled"
    value = "yes"
  }
}

Expected Behavior

Terrafom changes snapshot configuration of Elasticache Redis cluster, which is restored from an automatic backup.

Actual Behavior

Terraform apply against restored from backup Elasticache Redis cluster fails with the error:

error updating Elasticache Replication Group (prd273-mik): InvalidParameterCombination: Cannot set snapshotting cluster for cluster mode enabled replication group.

Steps to Reproduce

Provision Redis cluster with backuping (snapshot) configuration:

terraform apply -var-file=terraform.tfvars

Create a backup manually or wait while a backup is created automatically.
Remove Redis cluster manually in AWS console (simulation of cluster failure).
Restore Redis cluster from the backup manually in AWS console.

Snapshot configuration:

$ aws elasticache describe-snapshots
{
    "Engine": "redis", 
    "CacheParameterGroupName": "prd273-mik", 
    "VpcId": "vpc-0165f1db07d228505", 
    "NodeSnapshots": [
        {
            "SnapshotCreateTime": "2018-11-09T14:19:43Z", 
            "CacheNodeId": "0001", 
            "CacheClusterId": "prd273-mik-0001-003", 
            "NodeGroupId": "0001", 
            "CacheNodeCreateTime": "2018-11-09T12:51:09.161Z", 
            "CacheSize": "5 MB"
        }
    ], 
    "NumNodeGroups": 1, 
    "SnapshotName": "automatic.prd273-mik-2018-11-09-14-19", 
    "ReplicationGroupId": "prd273-mik", 
    "AutoMinorVersionUpgrade": true, 
    "SnapshotRetentionLimit": 35, 
    "AutomaticFailover": "enabled", 
    "SnapshotStatus": "available", 
    "SnapshotSource": "automated", 
    "SnapshotWindow": "14:19-16:00", 
    "EngineVersion": "4.0.10", 
    "CacheSubnetGroupName": "prd273-mik-habitat-ptrpersistence", 
    "ReplicationGroupDescription": "Redis cluster for PTR", 
    "Port": 6379, 
    "PreferredMaintenanceWindow": "sun:05:00-sun:06:00", 
    "CacheNodeType": "cache.m1.small"
}

Always Redis cluster is restored from a backup to a NEW Cluster.
The new cluster is restored with default snapshot configuratations:

    "SnapshotRetentionLimit": 0,   <----   `0` - means disabling snapshotting
    "SnapshotWindow": "23:00-00:00", <----   any available window

Restored redis configuration:

$ aws elasticache describe-cache-clusters
        {
            "Engine": "redis", 
            "AuthTokenEnabled": false, 
            "CacheParameterGroup": {
                "CacheNodeIdsToReboot": [], 
                "CacheParameterGroupName": "prd273-mik", 
                "ParameterApplyStatus": "in-sync"
            }, 
            "SnapshotRetentionLimit": 0, 
            "CacheClusterId": "prd273-mik-0001-001", 
            "CacheSecurityGroups": [], 
            "NumCacheNodes": 1, 
            "AtRestEncryptionEnabled": false, 
            "SnapshotWindow": "23:00-00:00", 
            "CacheClusterCreateTime": "2018-11-09T14:49:23.167Z", 
            "ReplicationGroupId": "prd273-mik", 
            "AutoMinorVersionUpgrade": true, 
            "CacheClusterStatus": "available", 
            "PreferredAvailabilityZone": "eu-west-1a", 
            "ClientDownloadLandingPage": "https://console.aws.amazon.com/elasticache/home#client-download:", 
            "TransitEncryptionEnabled": false, 
            "CacheSubnetGroupName": "prd273-mik-habitat-ptrpersistence", 
            "EngineVersion": "4.0.10", 
            "PendingModifiedValues": {}, 
            "PreferredMaintenanceWindow": "sun:05:00-sun:06:00", 
            "CacheNodeType": "cache.m1.small"
        },

Run terraform apply for changing Elasticache Redis cluster configuration and get the error:

An execution plan has been generated and is shown below.
Resource actions are indicated with the following symbols:
  ~ update in-place

Terraform will perform the following actions:

  ~ module.redis.aws_elasticache_replication_group.ptr_persistence
      replication_group_description: " " => "Redis cluster for PTR"
      security_group_ids.#:          "0" => "1"
      security_group_ids.146061859:  "" => "sg-02a25ce45ef1c23be"
      snapshot_retention_limit:      "0" => "35"
      snapshot_window:               "22:30-23:30" => "14:19-16:00"


Plan: 0 to add, 1 to change, 0 to destroy.

Do you want to perform these actions?
  Terraform will perform the actions described above.
  Only 'yes' will be accepted to approve.

  Enter a value: yes

module.redis.aws_elasticache_replication_group.ptr_persistence: Modifying... (ID: prd273-mik)
  replication_group_description: " " => "Redis cluster for PTR"
  security_group_ids.#:          "0" => "1"
  security_group_ids.146061859:  "" => "sg-02a25ce45ef1c23be"
  snapshot_retention_limit:      "0" => "35"
  snapshot_window:               "22:30-23:30" => "14:19-16:00"

Error: Error applying plan:

1 error(s) occurred:

* module.redis.aws_elasticache_replication_group.ptr_persistence: 1 error(s) occurred:

* aws_elasticache_replication_group.ptr_persistence: error updating Elasticache Replication Group (prd273-mik): InvalidParameterCombination: Cannot set snapshotting cluster for cluster mode enabled replication group.
	status code: 400, request id: e60ed1be-e41f-11e8-9d89-0beb86abdd38

Terraform does not automatically rollback in the face of errors.
Instead, your Terraform state file has been partially updated with
any resources that successfully completed. Please address the error
above and apply again to incrementally change your infrastructure.

Workaround

Modify the cluster manually in AWS console:
set snapshot_retention_limit >0 and <35
set snapshot_window to any value like 20:00-23:00

$ aws elasticache describe-cache-clusters
        {
            "Engine": "redis", 
            "AuthTokenEnabled": false, 
            "CacheParameterGroup": {
                "CacheNodeIdsToReboot": [], 
                "CacheParameterGroupName": "prd273-mik", 
                "ParameterApplyStatus": "in-sync"
            }, 
            "SnapshotRetentionLimit": 0, 
            "CacheClusterId": "prd273-mik-0001-001", 
            "CacheSecurityGroups": [], 
            "NumCacheNodes": 1, 
            "AtRestEncryptionEnabled": false, 
            "SnapshotWindow": "20:00-23:00", 
            "CacheClusterCreateTime": "2018-11-09T14:49:23.167Z", 
            "ReplicationGroupId": "prd273-mik", 
            "AutoMinorVersionUpgrade": true, 
            "CacheClusterStatus": "available", 
            "PreferredAvailabilityZone": "eu-west-1a", 
            "ClientDownloadLandingPage": "https://console.aws.amazon.com/elasticache/home#client-download:", 
            "TransitEncryptionEnabled": false, 
            "CacheSubnetGroupName": "prd273-mik-habitat-ptrpersistence", 
            "EngineVersion": "4.0.10", 
            "PendingModifiedValues": {}, 
            "PreferredMaintenanceWindow": "sun:05:00-sun:06:00", 
            "CacheNodeType": "cache.m1.small"
        },

Run terraform apply again:

An execution plan has been generated and is shown below.
Resource actions are indicated with the following symbols:
  ~ update in-place

Terraform will perform the following actions:

  ~ module.redis.aws_elasticache_replication_group.ptr_persistence
      replication_group_description: " " => "Redis cluster for PTR"
      security_group_ids.#:          "0" => "1"
      security_group_ids.146061859:  "" => "sg-02a25ce45ef1c23be"
      snapshot_retention_limit:      "1" => "35"
      snapshot_window:               "20:00-23:00" => "02:00-05:00"


Plan: 0 to add, 1 to change, 0 to destroy.

Do you want to perform these actions?
  Terraform will perform the actions described above.
  Only 'yes' will be accepted to approve.

  Enter a value: yes 

module.redis.aws_elasticache_replication_group.ptr_persistence: Modifying... (ID: prd273-mik)
  replication_group_description: " " => "Redis cluster for PTR"
  security_group_ids.#:          "0" => "1"
  security_group_ids.146061859:  "" => "sg-02a25ce45ef1c23be"
  snapshot_retention_limit:      "1" => "35"
  snapshot_window:               "20:00-23:00" => "02:00-05:00"
module.redis.aws_elasticache_replication_group.ptr_persistence: Still modifying... (ID: prd273-mik, 10s elapsed)
module.redis.aws_elasticache_replication_group.ptr_persistence: Still modifying... (ID: prd273-mik, 20s elapsed)
module.redis.aws_elasticache_replication_group.ptr_persistence: Still modifying... (ID: prd273-mik, 30s elapsed)
module.redis.aws_elasticache_replication_group.ptr_persistence: Modifications complete after 34s (ID: prd273-mik)

Apply complete! Resources: 0 added, 1 changed, 0 destroyed.

The text was updated successfully, but these errors were encountered:

jsanant · 2019-10-28T14:03:24Z

@aeschright / @bflad - Is this issue resolved in the provider?

ervinb · 2020-04-14T08:09:56Z

@jsanant For what it's worth, I just ran into this with the latest provider and the proposed workaround is still working.

Zogoo · 2020-12-02T13:20:08Z

Any update or even any response from team? This is happening again Terraform 0.12.26 with aws provider 3.7.

rustlingwind · 2021-06-15T01:11:30Z

Also met this issue, with Terraform 0.14.3 and aws provider 3.37.0

amontalban · 2021-08-05T21:15:35Z

Just faced this with Terraform 1.0.4 and AWS provider 3.52.0.

davidhiebert · 2022-02-21T20:49:10Z

Running into same scenario w/ Terraform 1.1.6, AWS provider 4.2.0

euthuppan · 2022-04-19T21:58:35Z

This issue still exists for us..

spirosekulovski · 2022-06-28T15:07:02Z

+1 Just ran into this on Terraform 1.1.7 and AWS provider 4.20.1
It has been almost 4 years since this issue was first reported.

deepuashokan85 · 2023-03-30T22:28:22Z

+1 I too ran into this issue on Terraform v1.3.7.

maazmalik1 · 2023-06-05T11:37:10Z

This issue still exists on Terraform 1.3.7 and AWS provider v4.50.0

rooty0 · 2023-08-26T01:06:38Z

unbelievable, this is from 2019 and still not fixed

mmoreno43 · 2024-01-30T18:50:45Z

Still facing this issue while running terraform version 1.5.2

mijatovicdev · 2024-03-19T15:02:39Z

This is still an issue 6 years later with Terraform 1.6.6.

ghost mentioned this issue Nov 9, 2018

Elasticache: InvalidParameterCombination: Cannot set snapshotting cluster for cluster mode enabled replication group hashicorp/terraform#19336

Closed

bflad added the service/elasticache Issues and PRs that pertain to the elasticache service. label Nov 12, 2018

aeschright added the needs-triage Waiting for first response or review from a maintainer. label Jun 24, 2019

aeschright added bug Addresses a defect in current functionality. and removed needs-triage Waiting for first response or review from a maintainer. labels Nov 20, 2019

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Elasticache: InvalidParameterCombination: Cannot set snapshotting cluster for cluster mode enabled replication group #6412

Elasticache: InvalidParameterCombination: Cannot set snapshotting cluster for cluster mode enabled replication group #6412

ghost commented Nov 9, 2018

jsanant commented Oct 28, 2019

ervinb commented Apr 14, 2020

Zogoo commented Dec 2, 2020

rustlingwind commented Jun 15, 2021

amontalban commented Aug 5, 2021

davidhiebert commented Feb 21, 2022

euthuppan commented Apr 19, 2022

spirosekulovski commented Jun 28, 2022

deepuashokan85 commented Mar 30, 2023

maazmalik1 commented Jun 5, 2023

rooty0 commented Aug 26, 2023

mmoreno43 commented Jan 30, 2024

mijatovicdev commented Mar 19, 2024

Elasticache: InvalidParameterCombination: Cannot set snapshotting cluster for cluster mode enabled replication group #6412

Elasticache: InvalidParameterCombination: Cannot set snapshotting cluster for cluster mode enabled replication group #6412

Comments

ghost commented Nov 9, 2018

Terraform Version

Terraform Configuration Files

Expected Behavior

Actual Behavior

Steps to Reproduce

Workaround

jsanant commented Oct 28, 2019

ervinb commented Apr 14, 2020

Zogoo commented Dec 2, 2020

rustlingwind commented Jun 15, 2021

amontalban commented Aug 5, 2021

davidhiebert commented Feb 21, 2022

euthuppan commented Apr 19, 2022

spirosekulovski commented Jun 28, 2022

deepuashokan85 commented Mar 30, 2023

maazmalik1 commented Jun 5, 2023

rooty0 commented Aug 26, 2023

mmoreno43 commented Jan 30, 2024

mijatovicdev commented Mar 19, 2024