aws-eks: CloudWatch Logs Resource Policy Size Exceeded for Cluster Resource Provider

[peteraws]

Describe the bug

When deploying an EKS cluster, running into error for the Cluster Resource Provider, stating "the CloudWatch Logs Resource Policy size was exceeded. We suggest prefixing your CloudWatch log group name with /aws/vendedlogs/states/"

Expected Behavior

Successful deployment of EKS cluster resource provider

Current Behavior

EKS Cluster is failing

Logical ID - Providerwaiterstatemachine
Status Reason - Resource handler returned message: "Invalid Logging Configuration: The CloudWatch Logs Resource Policy size was exceeded. We suggest prefixing your CloudWatch log group name with /aws/vendedlogs/states/. (Service: AWSStepFunctions; Status Code: 400; Error Code: InvalidLoggingConfiguration;

Reproduction Steps

const cluster = new eks.Cluster(this, "TestCluster", {
version: eks.KubernetesVersion.V1_27,
mastersRole: clusterAdmin,
clusterName: props.clusterName,
defaultCapacity: 0,
vpc: this.vpc,
vpcSubnets: [{ subnetType: ec2.SubnetType.PRIVATE_WITH_NAT }],
securityGroup: TestSecurityGroup,
});

Possible Solution

append CloudWatch log group with /aws/vendedlogs/states/

Additional Information/Context

I have tried multiple deployments int he account and the deployment was previously successful, until the deployment started failing. I've tried several more time int he same account and I run into the same error every time. However, deploying the same code in a different account works.

CDK CLI Version

2.114.0

Framework Version

No response

Node.js Version

20.10.0

OS

macos

Language

TypeScript

Language Version

No response

Other information

No response

[pahud]

related to #19353

[peteraws]

@pahud - The ask is similar, but the difference between this issue and #19353 is there is no workaround for the Cluster Resource Provider by updating the log group name. Since the Cluster Resource Provider is the default provider for a EKS cluster, the user does not have access to modify the CloudWatch log group name for the Cluster Resource Provider.

[go-to-k]

Related to #28577

Are you using CDK with 2.114.0 Version? Perhaps that behavior will happen starting with v2.116.0.

https://github.com/aws/aws-cdk/releases/tag/v2.116.0
#27310

But, if you use v2.121.1 or later versions, I believe it has been resolved by revert.

https://github.com/aws/aws-cdk/releases/tag/v2.121.1
#28699

Because of the background that logging must be given by default to StepFunctions in the Custom resource provider for security reasons, this fundamental solution is being initiated by the PR here, so there may be some movement in the future.

[peteraws]

@go-to-k I saw the same issue with 2.118. However, confirming the issue does not exist in 2.121.1

[github-actions]

⚠️COMMENT VISIBILITY WARNING⚠️

Comments on closed issues are hard for our team to see.
If you need more assistance, please either tag a team member or open a new issue that references this one.
If you wish to keep having a conversation with other community members under this issue feel free to do so.