AWS::RDS::DBCluster ignores DBSubnetGroupName property when creating a clone

[rlister]

1. AWS::RDS::DBCluster

2. Cloudformation now supports RDS clones, courtesy of SourceDBClusterIdentifier and RestoreType properties (see AWS::RDS (Aurora Cloning Support) #18). The cluster creates correctly, with the exception of property DBSubnetGroupName, which appears to be ignored. Rather than using the configured value, the resource silently defaults to using default as a value.

3. Expected behavior: Cloudformation should use property DBSubnetGroupName.

4. I am testing with the following resource template:

"DbCluster": {
  "Type": "AWS::RDS::DBCluster",
  "DependsOn": [
    "SubnetGroup",
    "DbClusterParamGroup"
  ],
  "DeletionPolicy": "Delete",
  "Properties": {
    "SourceDBClusterIdentifier": {
      "Ref": "SourceCluster"
    },
    "RestoreType": "copy-on-write",
    "DBClusterParameterGroupName": {
      "Ref": "DbClusterParamGroup"
    },
    "DBSubnetGroupName": {
      "Ref": "SubnetGroup"
    },
    "UseLatestRestorableTime": true,
    "VpcSecurityGroupIds": [
      {
        "Fn::ImportValue": {
          "Fn::Sub": [
            "${vpc}-SgDb",
            {
            }
          ]
        }
      }
    ]
  }
}

5. Links to existing API doc: https://docs.aws.amazon.com/AWSCloudFormation/latest/UserGuide/aws-resource-rds-dbcluster.html

6. Category tag (optional) -> Database

7. Any additional context: I can create a working cluster using both AWS CLI and ruby-sdk with the same set of properties, and the subnet group is set correctly.

[kderholt]

Will this be fixed?

[d2kx]

Ran into the same issue today. Great addition to the RDS CloudFormaton set otherwise ❤

[piotrtpay]

Ran into the same issue, but I'm not actually getting Internal Failure, which I'm able to reproduce using the API ( when DBSubnetGroupName is specified, there's no issue, but it's not being passed from CloudFormation, and if I leave it empty, I get the same error )

[kderholt]

Still nothin from AWS on this ? This should just be fixed, its an annoying bug which would be much appreciated if fixed!

[piotrtpay]

@kderholt

A workaround I was able to use was to create a custom resource lambda, to which I pass the DB name via CDK, and which uses boto3 to create the DB clone. By using crhelper you can make it also delete the DB on stack deletion. The lambda is created during the CDK run. That's actually what AWS themselves do in some cases ( ECR stack uses a custom resource lambda for ECR creation for example )

[dillonbrowne]

Is there another workaround besides a lambda?

[kderholt]

I'm amazed that this is not yet fixed. They should remove it from the cloudformation api/function as it does not seem like a priority for AWS. It's been like this for almost a year now.
To answer your question, we used sdk and just "wait" for it to clone finish (stateful wait, instead of the stateless (from cf or lambda like piotrtpay suggest)).

[sureshc]

Would love to see a resolution to this issue as well. Currently we're programmatically creating / deleting the Resources needed to clone a cluster within the same non-default VPC. We' prefer to use CloudFormation.

[mikey-]

I came across this issue(and #18) today; I wonder if this issue occurs due to the fact that dbSubnetGroupName(model.getDBSubnetGroupName()) is not included in one of the RDS resource provider's translator function:

These functions seem to be responsible for creating API requests from ResourceModels, including the relevant properties and excluding the irrelevant properties. Unfortunately, it seems to me that this is the DBSubnetGroupName property has been mistakenly "ignored", and why CloudFormation always creates cloned clusters in the default VPC and subnet group.

Anyway, I hope that this helps :)

[mikey-]

Update: I made a similar comment to the one I made above, but on the internal issue which tracks this one - and I've received a reply saying

this is right. we should include the field. @osdrv is the primary engineer and will be able to address

So we can look forward to that :)

[chucksellick]

Unbelievable, I wasted most of today trying to get a cloned db instance, doesn't help that the error I saw was completely unfathomable and it additionally takes about 30 minutes each run to find out that my template has failed again. Well thank you for putting the information here, at least I eventually found out this was an AWS bug.

For reference in case anyone else is searching for the same error, the specific error this triggered for me was:

Subnet group <DBSubnetGroupName> is different from subnet group of cluster <DBClusterIdentifier>

Presumably this is because the cluster subnet group name is silently changing to default, therefore it now conflicts with the instance I was also creating which does honour the subnet group name.

Not the first time I've wasted significant chunks of my day chasing around an issue where Cloudformation just isn't doing what it should be doing. Would be nice if there was a little more response from AWS on some of these issues, I've seen some other tickets around this that are just closed without comment.

Well I guess for now I just clone the DB by hand and hope this gets fixed some time this decade.

[fschroder-slyp]

From my testing I think the security group property is being ignored too. Referenced code above seems to confirm this.

As part of debugging this I deleted default resources and got error messages like:

• No default subnet detected in VPC
• The VPC vpc-xxxxxx associated with the default subnet group does not exist. Please delete the default subnet group or specify another subnet group.

[mikey-]

@fschroder-slyp nice pickup Fed! Yeah, looks like the translator function needs to include vpcSecurityGroupIds(model.getVpcSecurityGroupIds()).

[scottiemc7]

I believe EngineMode is similarly being ignored when cloning. I was trying to create a provisioned clone of a serverless database and ran across this issue. Works fine from the CLI, but CF is 🤷‍♂️

[spg]

I experience the same issue while deploying a stack with the CDK (Version 2.20.0). Here is the message in the Cloudformation events:

Subnet group mysubnetgroup is different from subnet group of cluster my-stack-cluster-1o9ycaXXXXX (Service: AmazonRDS; Status Code: 400; Error Code: InvalidParameterCombination; Request ID: 39ffaf1b-0a2c-4334-80ce-XXXXXXXX; Proxy: null)

Here is the CDK stack:

import * as cdk from 'aws-cdk-lib'
import ec2 from 'aws-cdk-lib/aws-ec2'
import rds from 'aws-cdk-lib/aws-rds'
import { Construct } from 'constructs'

interface StackProps extends cdk.StackProps {
  sourceDbClusterIdentifier: string
}

export class MyStack extends cdk.Stack {
  constructor(scope: Construct, id: string, props: StackProps) {
    super(scope, id, props)

    const sourceDbClusterIdentifier = 'cloud-database-postgresdbdbclusterb3fd976e-XXXXXXXX'
    const engine = 'aurora-postgresql'
    const engineVersion = '13.4'

    const subnetGroup = new rds.CfnDBSubnetGroup(this, 'subnetgroup', {
      dbSubnetGroupDescription: 'for testing',
      subnetIds: ['subnet-0dfb6be3236XXXX', 'subnet-00a09a796fe8XXXXX', 'subnet-0b83fcabffXXXXXX'],
      dbSubnetGroupName: 'mysubnetgroup',
    })

    const securityGroup = new ec2.CfnSecurityGroup(this, 'id', {
      vpcId: 'vpc-0c6497d314XXXXXX'
      groupDescription: 'testing',
    })

    const cluster = new rds.CfnDBCluster(this, 'Cluster', {
      engine,
      useLatestRestorableTime: true,
      sourceDbClusterIdentifier,
      engineVersion,
      dbSubnetGroupName: subnetGroup.ref,
      vpcSecurityGroupIds: [securityGroup.ref],
    })

    new rds.CfnDBInstance(this, 'instance', {
      dbInstanceClass: 'db.r5.large',
      dbClusterIdentifier: cluster.ref,
      engine,
      engineVersion,
      publiclyAccessible: true,
      dbSubnetGroupName: subnetGroup.ref,
    })

    new cdk.CfnOutput(this, 'ClusterEndpointAddress', { value: cluster.attrEndpointAddress })
    new cdk.CfnOutput(this, 'ClusterEndpointPort', { value: cluster.attrEndpointPort })
  }
}

[moataz-mhmd]

Addressed in aws-cloudformation/aws-cloudformation-resource-providers-rds#329. Change deployed globally.

[rlister]

This works now. Thanks.