Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Cannot create Aurora Serverless cluster using RDS Construct #929

Open
patoncrispy opened this issue Oct 14, 2018 · 15 comments
Open

Cannot create Aurora Serverless cluster using RDS Construct #929

patoncrispy opened this issue Oct 14, 2018 · 15 comments
Assignees

Comments

@patoncrispy
Copy link

@patoncrispy patoncrispy commented Oct 14, 2018

There doesn't seem to be a way to create an Aurora Serverless database cluster using the RDS Construct. EngineMode is available in the underlying cfn but not in the Construct library.

Target Framework: netcoreapp2.1
Amazon.CDK: 0.12.0
Amazon.CDK.AWS.RDS: 0.12.0

@patoncrispy patoncrispy changed the title Cannot create Aurora Serverless cluster using RDS Construct [DotNetCore] - Cannot create Aurora Serverless cluster using RDS Construct Oct 14, 2018
@eladb

This comment has been minimized.

Copy link
Contributor

@eladb eladb commented Oct 14, 2018

Thanks for reporting. As a workaround, you should be able to use property overrides

@eladb eladb added the feature label Oct 14, 2018
@patoncrispy

This comment has been minimized.

Copy link
Author

@patoncrispy patoncrispy commented Oct 14, 2018

Ah sweet, thanks!

@rix0rrr rix0rrr changed the title [DotNetCore] - Cannot create Aurora Serverless cluster using RDS Construct Cannot create Aurora Serverless cluster using RDS Construct Oct 16, 2018
@rix0rrr rix0rrr added the gap label Jan 4, 2019
@ApocDev

This comment has been minimized.

Copy link

@ApocDev ApocDev commented Jul 30, 2019

I set this up for our usage, it's ugly, but it may be of use to someone who needs a "quick and dirty" version of Serverless Aurora:

Please ignore the lack of comments, and possibly broken code, I removed some proprietary information. 🤕

import {Connections, ISecurityGroup, IVpc, Port, SecurityGroup, SubnetSelection} from "@aws-cdk/aws-ec2";
import {
    CfnDBCluster,
    CfnDBSubnetGroup,
    DatabaseSecret,
    Endpoint,
    SecretRotation,
    SecretRotationApplication,
    SecretRotationOptions
} from "@aws-cdk/aws-rds";
import {AttachmentTargetType, ISecretAttachmentTarget, SecretAttachmentTargetProps, SecretTargetAttachment} from "@aws-cdk/aws-secretsmanager";
import {Construct, RemovalPolicy, Token} from "@aws-cdk/core";

export interface ServerlessAuroraProps {
    readonly vpc: IVpc;
    readonly subnets: SubnetSelection;

    readonly clusterName: string;

    readonly masterUsername?: string;
    readonly securityGroup?: ISecurityGroup;
    readonly secretRotationApplication?: SecretRotationApplication;

    readonly maxCapacity: number;
}

export class ServerlessAurora extends Construct implements ISecretAttachmentTarget {
    public securityGroupId: string;
    public clusterIdentifier: string;
    public clusterEndpoint: Endpoint;
    public secret: SecretTargetAttachment;
    public connections: Connections;
    public vpc: IVpc;
    public vpcSubnets: SubnetSelection;
    public secretRotationApplication: SecretRotationApplication;
    public securityGroup: ISecurityGroup;

    constructor(scope: Construct, id: string, private props: ServerlessAuroraProps) {
        super(scope, id);

        this.vpc = props.vpc;
        this.vpcSubnets = props.subnets;
        this.secretRotationApplication = props.secretRotationApplication || SecretRotationApplication.MYSQL_ROTATION_SINGLE_USER;

        const secret = new DatabaseSecret(this, "MasterUserSecret", {
            username: props.masterUsername || "root",
        });

        const securityGroup = props.securityGroup || new SecurityGroup(this, "DatabaseSecurityGroup", {
            allowAllOutbound: true,
            description: `DB Cluster (${props.clusterName}) security group`,
            vpc: props.vpc
        });
        this.securityGroup = securityGroup;
        this.securityGroupId = securityGroup.securityGroupId;

        const cluster = new CfnDBCluster(this, "DatabaseCluster", {
            engine: "aurora",
            engineMode: "serverless",
            engineVersion: "5.6",

            dbClusterIdentifier: props.clusterName,

            masterUsername: secret.secretValueFromJson("username").toString(),
            masterUserPassword: secret.secretValueFromJson("password").toString(),

            dbSubnetGroupName: new CfnDBSubnetGroup(this, "db-subnet-group", {
                dbSubnetGroupDescription: `${props.clusterName} database cluster subnet group`,
                subnetIds: props.vpc.selectSubnets(props.subnets).subnetIds
            }).ref,

            vpcSecurityGroupIds: [securityGroup.securityGroupId],

            storageEncrypted: true,

            // Maximum here is 35 days
            backupRetentionPeriod: 35,

            scalingConfiguration: {
                autoPause: true,
                secondsUntilAutoPause: 300,
                minCapacity: 1,
                maxCapacity: props.maxCapacity
            }
        });
        cluster.applyRemovalPolicy(RemovalPolicy.DESTROY, {applyToUpdateReplacePolicy: true});

        this.clusterIdentifier = cluster.ref;
        // create a number token that represents the port of the cluster
        const portAttribute = Token.asNumber(cluster.attrEndpointPort);
        this.clusterEndpoint = new Endpoint(cluster.attrEndpointAddress, portAttribute);

        if (secret) {
            this.secret = secret.addTargetAttachment('AttachedSecret', {target: this});
        }
        const defaultPort = Port.tcp(this.clusterEndpoint.port);
        this.connections = new Connections({securityGroups: [securityGroup], defaultPort});

        // This is currently causing errors when deploying, since it uses a SAM template under the hood
        // Error: "Received malformed response from transform AWS::Serverless-2016-10-31"
        // It also adds in a warning from the CDK:
        // "This stack is using the deprecated `templateOptions.transform` property. Consider switching to `templateOptions.transforms`."
        // Which has notified *all* of our DevOps when we upgraded.
        // this.addRotationSingleUser("Rotation");
    }

    /**
     * Adds the single user rotation of the master password to this cluster.
     */
    public addRotationSingleUser(id: string, options?: SecretRotationOptions): SecretRotation {
        if (!this.secret) {
            throw new Error('Cannot add single user rotation for a cluster without secret.');
        }
        return new SecretRotation(this, id, {
            secret: this.secret,
            application: this.secretRotationApplication,
            vpc: this.vpc,
            vpcSubnets: this.vpcSubnets,
            target: this,
            automaticallyAfter: options ? options.automaticallyAfter : undefined,
        });
    }

    public asSecretAttachmentTarget(): SecretAttachmentTargetProps {
        return {
            targetId: this.clusterIdentifier,
            targetType: AttachmentTargetType.CLUSTER
        };
    }
}
@ryanvade

This comment has been minimized.

Copy link

@ryanvade ryanvade commented Jul 31, 2019

Does #688 not implement this (it adds Aurora RDS)?
(Side note, @eladb your link is no longer valid, have a new one?)

@ApocDev

This comment has been minimized.

Copy link

@ApocDev ApocDev commented Aug 3, 2019

Does #688 not implement this (it adds Aurora RDS)?
(Side note, @eladb your link is no longer valid, have a new one?)

No. The DatabaseCluster doesn't expose the engineMode and scalingConfiguration properties. It also has other property requirements that are only valid for non-serverless instances. (Instance type, size, etc) Those options are not available in the serverless mode.

@ryanvade

This comment has been minimized.

Copy link

@ryanvade ryanvade commented Aug 4, 2019

I see, would it be preferable to add a "Serverless" Database Prop or instead change DatabaseClusterProps to support "Serverless" configurations?

@ApocDev

This comment has been minimized.

Copy link

@ApocDev ApocDev commented Aug 4, 2019

I think creating a separate resource is probably the best option considering most of the options on the current cluster type don't quite apply. E.g only MySQL and Postgres are available, and you can't call lambdas from stored procedures or queries, etc.

Serverless also has the "Web API" feature that may be useful to turn on from some stacks. (Though, I'm not sure that it's currently supported in Cfn)

@seawatts

This comment has been minimized.

Copy link

@seawatts seawatts commented Oct 23, 2019

Thanks for reporting. As a workaround, you should be able to use property overrides

This link doesn't seem to work. Can you give an example of how to do this?

@shivaramani

This comment has been minimized.

Copy link

@shivaramani shivaramani commented Nov 1, 2019

Thanks for reporting. As a workaround, you should be able to use property overrides

This link doesn't seem to work. Can you give an example of how to do this?

I also dont see that link working. Can you please an example of property overrides?

@yargevad

This comment has been minimized.

Copy link

@yargevad yargevad commented Nov 8, 2019

This doc shows how to do property overrides.

@apurvis

This comment has been minimized.

Copy link

@apurvis apurvis commented Nov 12, 2019

any hopes for a release that allows this?

@skinny85

This comment has been minimized.

Copy link
Contributor

@skinny85 skinny85 commented Nov 22, 2019

@jogold is this a matter of adding an EngineMode enum with the values listed in https://docs.aws.amazon.com/AWSCloudFormation/latest/UserGuide/aws-resource-rds-dbcluster.html#cfn-rds-dbcluster-enginemode and an engineMode property to DatabaseClusterProps, or is this more complicated?

@jogold

This comment has been minimized.

Copy link
Contributor

@jogold jogold commented Nov 27, 2019

I think that it's more complicated than that and that it requires a separate class.

@aripalo

This comment has been minimized.

Copy link

@aripalo aripalo commented Nov 27, 2019

@skinny85: It should also support EnableHttpEndpoint boolean property to allow enabling Data API.

Although this is not yet implemented in CfnDBCluster yet. See #5216 for more.

@jogold

This comment has been minimized.

Copy link
Contributor

@jogold jogold commented Nov 27, 2019

@skinny85 the DatabaseCluster was originally authored by @rix0rrr and @eladb I think. I only did the DatabaseInstance and added secret rotation to both. Maybe also check with them.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
You can’t perform that action at this time.