Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Use function name instead of counter for multiple cloudwatchLogs for avoiding SubscriptionFilter ResourceExceeded error #6263

Closed
rdsedmundo opened this issue Jun 17, 2019 · 0 comments · Fixed by #6471

Comments

@rdsedmundo
Copy link
Contributor

rdsedmundo commented Jun 17, 2019

This is a Bug Report

Description

While adding a cloudwatchLog trigger event (subscription event) to a function that has many of them, we're wrongly relying on a counter to create the CloudFormation's Logical IDs, and if we change the order of by just swapping a couple, deleting, or adding new (that's not in the end), we hit a ResourceExceeded error.

Similar or dependent issues:

Additional Data

Serverless adds a count at the end of the Logical IDs that start with 1. See: https://github.com/serverless/serverless/blob/master/lib/plugins/aws/package/compile/events/cloudWatchLog/index.js#L79

Let's say we have this:

functions:
  performUserExperiments:
    handler: dist/handler.performUserExperiments

Which generates (simplified):

{
   ...
   SumologicLoggingLogsSubscriptionFilterCloudWatchLog1: {
      Type: SubscriptionFilter
      Log: performUserExperimentsLogGroup
   }
}

We add a new fn at the beginning:

functions:
  getUserExperiments:
    handler: dist/handler.getUserExperiments
  performUserExperiments:
    handler: dist/handler.performUserExperiments

Which generates:

{
   ...
   SumologicLoggingLogsSubscriptionFilterCloudWatchLog1: {
      Type: SubscriptionFilter
      Log: getUserExperimentsLogGroup
   },
   SumologicLoggingLogsSubscriptionFilterCloudWatchLog2: {
      Type: SubscriptionFilter
      Log: performUserExperimentsLogGroup
   }
}

Now even though the performUserExperiments fn has not changed at all since its LogicalID changed, CloudFormation will try to create a new resource, in parallel while it applies the update at SumologicLoggingLogsSubscriptionFilterCloudWatchLog1, which before was pointing to performUserExperiments, but now it's pointing to getUserExperiments. This is an update that requires a new resource from scratch, by the way.

Proposed Fix

Get rid of this count completely, and use the name of the log group as part of the Logical ID.

For example, assuming we have:

/aws/lambda/performUserExperiment (log group)
sumologic-logging-dev (function that will be called by the log group)

We can name the Logical ID as:

SumologicLoggingLogsSubscriptionFilterPerformUserExperiment

Instead of:

SumologicLoggingLogsSubscriptionFilterX (X = this serial number we have today)

I'm happy to send a PR if you believe that this approach makes sense. @medikoo @pmuens

  • Serverless Framework Version you're using: v1.45.1
  • Operating System: MacOS
  • Provider Error messages: An error occurred: SumologicLoggingLogsSubscriptionFilterCloudWatchLog2 - Resource limit exceeded. (Service: AWSLogs; Status Code: 400; Error Code: LimitExceededException; Request ID: ***).
@rdsedmundo rdsedmundo changed the title Use function name instead of counter for multiple cloudwatchLogs for avoiding SubscriptionFilter limit exceeded error Use function name instead of counter for multiple cloudwatchLogs for avoiding SubscriptionFilter resourceexceeded error Jun 17, 2019
@rdsedmundo rdsedmundo changed the title Use function name instead of counter for multiple cloudwatchLogs for avoiding SubscriptionFilter resourceexceeded error Use function name instead of counter for multiple cloudwatchLogs for avoiding SubscriptionFilter ResourceExceeded error Jun 17, 2019
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
Development

Successfully merging a pull request may close this issue.

2 participants