Aurora Serverless v1 as an environment-level storage addon

[Lou1415926]

Today, copilot storage init does not support creating Aurora Serverless v1 template for a storage whose lifecycle is "environment".

For folks that want an Aurora Serverless v1 as an environment addon, I've prepared the steps you can follow below!

Prerequisite

You have already run copilot env init in your workspace. This should create a folder under copilot/ named environments/. Your copilot/ workspace should look something like this:

.
└── copilot/
    └── environments/ 
        └── test/ # The name of an environment that you've initiated using "copilot env init"
            └── manifeset.yml

Steps

1. Create a folder named "addons/" under "environments/". Skip this step if the folder already exists.

.
└── copilot/
    └── environments/ 
      ├── test/ # The name of an environment that you've initiated using "copilot env init"
      │   └── manifeset.yml
      └── addons/

2. Create a file named "addons.parameters.yml" under the "addons/" folder from step 1. Skip this step if the file already exists.
   Your folders should look like this now:

.
└── copilot/
    └── environments/ 
      ├── test/ # The name of an environment that you've initiated using "copilot env init"
      │   └── manifeset.yml
      └── addons/
          └── addons.parameters.yml

3. Write the following ⬇️ in "addons.parameters.yml". Skip this step if the content is already in the "addons.parameters.yml" file.

Parameters:
  VPCID: !Ref VPC
  PrivateSubnets: !Join [ ',', [ !Ref PrivateSubnet1, !Ref PrivateSubnet2 ] ]

4. Create a file named "serverless-v1.yml" under the "addons/" folder from step 1. Skip this step if the file already exists.
   Your folders should look like this now:

.
└── copilot/
    └── environments/ 
      ├── test/ # The name of an environment that you've initiated using "copilot env init"
      │   └── manifeset.yml
      └── addons/
          └── addons.parameters.yml
          └── serverless-v1.yml

5. Write the CloudFormation template that defines an Aurora Serverless v1 storage in the "serverless-v1.yml" file. You can start with the below ⬇️ example. Be sure to replace all <placeholder:> with your desired value.

Parameters:
  App:
    Type: String
    Description: Your application's name.
  Env:
    Type: String
    Description: The environment name your service, job, or workflow is being deployed to.
  # Customize your Aurora Serverless cluster by setting the default value of the following parameters.
  <placeholder:clustername>DBName:
    Type: String
    Description: The name of the initial database to be created in the DB cluster.
    Default: <placeholder:dbname>
    # Cannot have special characters
    # Naming constraints: https://docs.aws.amazon.com/AmazonRDS/latest/UserGuide/CHAP_Limits.html#RDS_Limits.Constraints
  VPCID:
    Type: String
    Description: The ID of the VPC in which to create the Aurora Serverless cluster.
    Default: ""
  PrivateSubnets:
    Type: String
    Description: The IDs of the private subnets in which to create the Aurora Serverless cluster.
    Default: ""
Mappings:
  <placeholder:clustername>EnvScalingConfigurationMap:
    test:
      "DBMinCapacity": 1 # AllowedValues: [1, 2, 4, 8, 16, 32, 64, 128, 256]
      "DBMaxCapacity": 8 # AllowedValues: [1, 2, 4, 8, 16, 32, 64, 128, 256]
    All:
      "DBMinCapacity": 1 # AllowedValues: [1, 2, 4, 8, 16, 32, 64, 128, 256]
      "DBMaxCapacity": 8 # AllowedValues: [1, 2, 4, 8, 16, 32, 64, 128, 256]

Resources:
  <placeholder:clustername>DBSubnetGroup:
    Type: 'AWS::RDS::DBSubnetGroup'
    Properties:
      DBSubnetGroupDescription: Group of Copilot private subnets for Aurora cluster.
      SubnetIds:
        !Split [',', !Ref PrivateSubnets]
  <placeholder:clustername>WorkloadSecurityGroup:
    Metadata:
      'aws:copilot:description': 'A security group for one or more workloads to access the DB cluster <placeholder:clustername>'
    Type: 'AWS::EC2::SecurityGroup'
    Properties:
      GroupDescription: !Sub 'The security group to access DB cluster <placeholder:clustername>.'
      VpcId: !Ref VPCID
      Tags:
        - Key: Name
          Value: !Sub 'copilot-${App}-${Env}-Aurora'
  <placeholder:clustername>DBClusterSecurityGroup:
    Metadata:
      'aws:copilot:description': 'A security group for your DB cluster <placeholder:clustername>'
    Type: AWS::EC2::SecurityGroup
    Properties:
      GroupDescription: The Security Group for the database cluster.
      SecurityGroupIngress:
        - ToPort: 3306
          FromPort: 3306
          IpProtocol: tcp
          Description: Ingress from one or more workloads in the environment.
          SourceSecurityGroupId: !Ref <placeholder:clustername>WorkloadSecurityGroup
      VpcId: !Ref VPCID
      Tags:
        - Key: Name
          Value: !Sub 'copilot-${App}-${Env}-Aurora'
  <placeholder:clustername>AuroraSecret:
    Metadata:
      'aws:copilot:description': 'A Secrets Manager secret to store your DB credentials'
    Type: AWS::SecretsManager::Secret
    Properties:
      Description: !Sub Aurora main user secret for ${AWS::StackName}
      GenerateSecretString:
        SecretStringTemplate: '{"username": "admin"}'
        GenerateStringKey: "password"
        ExcludePunctuation: true
        IncludeSpace: false
        PasswordLength: 16

  <placeholder:clustername>DBClusterParameterGroup:
    Metadata:
      'aws:copilot:description': 'A DB parameter group for engine configuration values'
    Type: 'AWS::RDS::DBClusterParameterGroup'
    Properties:
      Description: !Ref 'AWS::StackName'
      Family: 'aurora-mysql5.7'
      Parameters:
        character_set_client: 'utf8'
  <placeholder:clustername>DBCluster:
    Metadata:
      'aws:copilot:description': 'The <placeholder:clustername> Aurora Serverless database cluster'
    Type: 'AWS::RDS::DBCluster'
    Properties:
      MasterUsername:
        !Join [ "",  [ '{{resolve:secretsmanager:', !Ref <placeholder:clustername>AuroraSecret, ":SecretString:username}}" ]]
      MasterUserPassword:
        !Join [ "",  [ '{{resolve:secretsmanager:', !Ref <placeholder:clustername>AuroraSecret, ":SecretString:password}}" ]]
      DatabaseName: !Ref <placeholder:clustername>DBName
      Engine: 'aurora-mysql'
      EngineVersion: '5.7.mysql_aurora.2.07.1'
      EngineMode: serverless
      DBClusterParameterGroupName: !Ref <placeholder:clustername>DBClusterParameterGroup
      DBSubnetGroupName: !Ref <placeholder:clustername>DBSubnetGroup
      VpcSecurityGroupIds:
        - !Ref <placeholder:clustername>DBClusterSecurityGroup
      ScalingConfiguration:
        AutoPause: true
        # Replace "All" below with "!Ref Env" to set different autoscaling limits per environment.
        MinCapacity: !FindInMap [<placeholder:clustername>EnvScalingConfigurationMap, All, DBMinCapacity]
        MaxCapacity: !FindInMap [<placeholder:clustername>EnvScalingConfigurationMap, All, DBMaxCapacity]
        SecondsUntilAutoPause: 1000
  <placeholder:clustername>SecretAuroraClusterAttachment:
    Type: AWS::SecretsManager::SecretTargetAttachment
    Properties:
      SecretId: !Ref <placeholder:clustername>AuroraSecret
      TargetId: !Ref <placeholder:clustername>DBCluster
      TargetType: AWS::RDS::DBCluster
Outputs:
  <placeholder:clustername>Secret: # injected as {{envVarSecret .ClusterName | toSnakeCase}} environment variable by Copilot.
    Description: "The JSON secret that holds the database username and password. Fields are 'host', 'port', 'dbname', 'username', 'password', 'dbClusterIdentifier' and 'engine'"
    Value: !Ref <placeholder:clustername>AuroraSecret
    Export:
      Name: !Sub ${App}-${Env}-<placeholder:clustername>AuroraSecret
  <placeholder:clustername>SecurityGroup:
    Description: "The security group to attach to the workload."
    Value: !Ref <placeholder:clustername>WorkloadSecurityGroup
    Export:
      Name: !Sub ${App}-${Env}-<placeholder:clustername>SecurityGroup

6. Run copilot env deploy to deploy the environment with the serverless-v1 addon.
7. Update the manifest(s) for the workload(s) that need access to the Aurora Serverless v1 storage:

network:
  vpc:
    security_groups:
      - from_cfn: ${COPILOT_APPLICATION_NAME}-${COPILOT_ENVIRONMENT_NAME}-<placeholder:clustername>SecurityGroup
secrets:
  DB_SECRET:
    from_cfn: ${COPILOT_APPLICATION_NAME}-${COPILOT_ENVIRONMENT_NAME}-<placeholder:clustername>AuroraSecret