(custom-resources): Default logging configuration hits Cloudwatch Logs Resource Policy size limit

[munch9]

Please add your +1 👍 to let us know you have encountered this

Status: RESOLVED

Overview:

A previous PR enabled the ability to configure logging for a state-machine that was managed via a custom resource. This introduced a bug where the logging resource policy became too large because of the name of the log group. This could break the deploys of existing applications that contained this custom resource.

Complete Error Message:

@aws-cdk--aws-dynamodb.ReplicaProvider/Provider/waiter-state-machine (Providerwaiterstatemachine5D4A9DF0) Resource handler returned message: "Invalid Logging Configuration: The CloudWatch Logs Resource Policy size was exceeded. We suggest prefixing your CloudWatch log group name with /aws/vendedlogs/states/. (Service: AWSStepFunctions; Status Code: 400; Error Code: InvalidLoggingConfiguration; Request ID: 4ba96f98-4be5-450c-a069-3d4cbf93271a; Proxy: null)"

Workaround:

Lock your cdk version to v1.115.0 or below.

Solution:

Revert in progress: #28699

Related Issues:

Original Issue:

Title: (custom-resources): Default logging configuration hits Cloudwatch Logs Resource Policy size limit

Describe the bug

Upgrading aws-cdk > 2.115 adds a default logging configuration to custom resources.

When using aws_dynamodb.Table with replication_regions specified this automatically generates a @aws-cdk--aws-dynamodb.ReplicaProvider.NestedStackResource which contains Providerwaiterstatemachine and post upgrade also a new log group

When deploying the following error is returned
@aws-cdk--aws-dynamodb.ReplicaProvider/Provider/waiter-state-machine (Providerwaiterstatemachine5D4A9DF0) Resource handler returned message: "Invalid Logging Configuration: The CloudWatch Logs Resource Policy size was exceeded. We suggest prefixing your CloudWatch log group name with /aws/vendedlogs/states/. (Service: AWSStepFunctions; Status Code: 400; Error Code: InvalidLoggingConfiguration; Request ID: 4ba96f98-4be5-450c-a069-3d4cbf93271a; Proxy: null)"

Expected Behavior

Default logging configuration should be configured in a way to prevent the above error

Current Behavior

Upgrading to aws-cdk >=2.116 generates a logging configuration with what I believe is no name/prefix specified to the log group

"ProviderwaiterstatemachineLogGroupDD693A98": {
   "Type": "AWS::Logs::LogGroup",
   "Properties": {
    "RetentionInDays": 731,
   },
   "UpdateReplacePolicy": "Retain",
   "DeletionPolicy": "Retain",
   "Metadata": {
    "aws:cdk:path": "my-stack/@aws-cdk--aws-dynamodb.ReplicaProvider/Provider/waiter-state-machine/LogGroup/Resource"
   }
  }

Given the nested stack is auto generated I see no way to override the properties here.

Reproduction Steps

Using the below construct with aws-cdk <=2.115 deploys successfully

table = aws_dynamodb.Table(
            self,
            "my-table",
            table_name="my-table",
            partition_key={"name": "pk", "type": aws_dynamodb.AttributeType.STRING},
            removal_policy=RemovalPolicy.RETAIN,
            replication_regions=["eu-central-1"],
            billing_mode=aws_dynamodb.BillingMode.PAY_PER_REQUEST,
            encryption=aws_dynamodb.TableEncryption.AWS_MANAGED,
            point_in_time_recovery=True,
        )

Upgrading aws-cdk > 2.115 causes the above error due to the newly added log group

Possible Solution

Applying the prefix recommended in the above error message /aws/vendedlogs/states/ similar to the recommendation [here] (https://docs.aws.amazon.com/step-functions/latest/dg/cw-logs.html) to the log group name?

Additional Information/Context

No response

CDK CLI Version

2.116.0

Framework Version

No response

Node.js Version

18

OS

Linux

Language

Python

Language Version

No response

Other information

No response

[pahud]

I think it's related to #27310

@go-to-k do you have any ideas about that?

[go-to-k]

Thanks for this report.

That is certainly true. I will work on this issue. It's to create a default log group to start with /aws/vendedlogs/states/.

[MrArnoldPalmer]

Because this issue is a regression, I am reverting the PR that caused this initially. We can roll the new fix into the original change and make sure the tests cover the failure correctly to avoid this in the future.

Revert Pr: #28699

[go-to-k]

@MrArnoldPalmer

Okey, I'm going to add the correct changes in the PR for the one I'm working on now.

[MrArnoldPalmer]

thank you @go-to-k appreciate you taking this up again.

[MrArnoldPalmer]

v2.121.1 is out and should have this solved.

[github-actions]

⚠️COMMENT VISIBILITY WARNING⚠️

Comments on closed issues are hard for our team to see.
If you need more assistance, please either tag a team member or open a new issue that references this one.
If you wish to keep having a conversation with other community members under this issue feel free to do so.