AWS::Serverless::SimpleTable Auto-Scaling

[oharaandrew314]

DynamoDB Auto-Scaling via CFN is verbose to say the least. I propose that we add configuration properties to AWS::Serverless::SimpleTable which will implicitly generate all the necessary resources via the SAM translator.

SAM Template:

MyTable:
  Type: AWS::Serverless::SimpleTable
  Properties:
    ...
    AutoScaling: (optional)
      ReadScaling: (optional, disabled if omitted)
        MinCapacity: Integer (default: 5, same as console)
        MaxCapacity: Integer (default: 4000, same as console)
        TargetUtilization: Integer (default: 70, same as console)
      WriteScaling: (optional, disabled if omitted)
        MinCapacity: Integer (default: 5, same as console)
        MaxCapacity: Integer (default: 4000, same as console)
        TargetUtilization: Integer (default: 70, same as console)

Translated CFN Template:

MyTable:
  Type: AWS::DynamoDB::Table
  Properties:
    ...

MyTableScalingRole:
  Type: AWS::IAM::Role
  Properties:
    AssumeRolePolicyDocument:
      Version: "2012-10-17"
      Statement:
      - Effect: Allow
        Principal:
          Service:
          - application-autoscaling.amazonaws.com
        Action:
        - sts:AssumeRole
    Path: "/"
    Policies:
    - PolicyName: root
      PolicyDocument:
        Version: "2012-10-17"
        Statement:
        - Effect: "Allow"
          Action:
          - dynamodb:DescribeTable
          - dynamodb:UpdateTable
          - cloudwatch:PutMetricAlarm
          - cloudwatch:DescribeAlarms
          - cloudwatch:GetMetricStatistics
          - cloudwatch:SetAlarmState
          - cloudwatch:DeleteAlarms
          Resource: !GetAtt MyTable.Arn

MyTableWriteCapacityScalableTarget:
  Type: AWS::ApplicationAutoScaling::ScalableTarget
  Properties:
    MaxCapacity: Integer (provided by SAM template)
    MinCapacity: Integer (provided by SAM template)
    ResourceId: !Sub table/${MyTableScalingRole.Arn}
    ScalableDimension: dynamodb:table:WriteCapacityUnits
    ServiceNamespace: dynamodb

MyTableWriteScalingPolicy:
  Type: AWS::ApplicationAutoScaling::ScalingPolicy
  Properties:
    PolicyName: MyTableWriteAutoScalingPolicy
    PolicyType: TargetTrackingScaling
    ScalingTargetId: !Ref MyTableWriteCapacityScalableTarget
    TargetTrackingScalingPolicyConfiguration:
      TargetValue: Integer  (provided by SAM template)
      ScaleInCooldown: 60
      ScaleOutCooldown: 60
      PredefinedMetricSpecification:
        PredefinedMetricType: DynamoDBWriteCapacityUtilization

MyTableReadCapacityScalableTarget:
  Type: AWS::ApplicationAutoScaling::ScalableTarget
  Properties:
    MaxCapacity: Integer (provided by SAM template)
    MinCapacity: Integer (provided by SAM template)
    ResourceId: !Sub table/${MyTableScalingRole.Arn}
    ScalableDimension: dynamodb:table:ReadCapacityUnits
    ServiceNamespace: dynamodb

MyTableReadScalingPolicy:
  Type: AWS::ApplicationAutoScaling::ScalingPolicy
  Properties:
    PolicyName: MyTableReadAutoScalingPolicy
    PolicyType: TargetTrackingScaling
    ScalingTargetId: !Ref MyTableReadCapacityScalableTarget
    TargetTrackingScalingPolicyConfiguration:
      TargetValue: Integer (provided by SAM template)
      ScaleInCooldown: 60
      ScaleOutCooldown: 60
      PredefinedMetricSpecification:
        PredefinedMetricType: DynamoDBReadCapacityUtilization

I welcome any discussion on this proposal. This proposed implementation would generate a scaling role per table, which would be wasteful. The console creates and reuses a single role for this purpose, but this may be awkward to implement in SAM.

I would be willing to work on this once I become more familiar with the internals of the SAM translator, but it may best be left to someone with more intimate knowledge.

[sanathkr]

This is beautiful! Thanks for the detailed proposal showing SAM template and associated CFN template.

A few things I would encourage you to think:

1. Can we somehow allow customers to provide their own IAM Roles, may be thru a Role property?
2. Do we need to provide overrides for ScalingInCooldown and ScalingUpCooldown in SAM?
3. How do we make this future proof? This is open ended but I would usually think, if DDB adds new functionality such as Scaling based on Throttles or some new scaling configuration, where we might add them in future? Does the syntax you are proposing allow such extensions?

[oharaandrew314]

Can we somehow allow customers to provide their own IAM Roles, may be thru a Role property?

Are you thinking of an optional role to override the generated one? If so, then that may be an option.

Do we need to provide overrides for ScalingInCooldown and ScalingUpCooldown in SAM?

Do we need to? No, the console doesn't even let you choose this. Could we? Sure, do you want to?

How do we make this future proof?

¯_(ツ)_/¯

I guess it's entirely possible for dynamo to add more metrics that you might want to scale based off of, but I think that allowing the user to specify their own TargetTrackingScalingPolicyConfiguration, would defeat the purpose of this implicit configuration. Maybe we should just add more metrics as they become available?

e.g.

ReadScaling: (optional, disabled if omitted)
  MinCapacity: Integer (default: 5, same as console)
  MaxCapacity: Integer (default: 4000, same as console)
  TargetUtilization: Integer (default: 70, same as console, must be undefined if OtherScalingMetric is defined)
  OtherScalingMetric: Integer (optional, must be undefined if TargetUtilization is defined)

I've seen this sort of conditional configuration in other areas of SAM, like the RestApi DefinitionBody and DefinitionUri.

[brysontyrrell]

+1

[mariano-calandra-xp]

I was playing with translator implementing the same feature. I'm currently stuck with an error that I can't figure out, let me know if I can help.

[lorddelicious]

+1

[luispabon]

Just to point out, AWS::ApplicationAutoScaling::ScalableTarget requires the RoleArn property.

[victor-paddle]

Any update on this? It would be really handy to have something like the proposed in here.

[jlhood]

#705 (in the upcoming v1.10.0 release) changed AWS::Serverless::SimpleTable to default to use DynamoDB's recently announced on-demand capacity mode. This gives you similar behavior to auto-scaling, but fully managed through DynamoDB instead of needing to setup separate auto-scaling resources.

[adamwshero]

I would suggest that this issue be reopened as the on-demand capacity mode is expensive and not a one-size-fits-all solution of the aforementioned problem.