[aws-s3-notifications] add_event_notification creates Lambda AND SNS Event Notifications

[ColWillis]

As per example detailed here

I add an SNS destination as detailed above, but after a cdk deploy, a standalone NodeJS Lambda Function is also deployed (with auto-generated IAM Role too)!

Reproduction Steps

My (Python) Code:
testdata_bucket.add_event_notification(s3.EventType.OBJECT_CREATED_PUT, s3n.SnsDestination(thesnstopic), s3.NotificationKeyFilter(prefix=eventprefix, suffix=eventsuffix))

When my code is commented or removed, NO Lambda is present in the cdk.out cfn JSON.
When my code is enabled, in the cdk.out cfn JSON it produces:

1x Custom::S3BucketNotifications (Containing NotificationConfiguration-TopicConfigurations)
1x AWS::IAM::Role (sts:AssumeRole for Lambda), I am also not happy about this as I generate my own IAM roles elsewhere
1x AWS::IAM::Policy (s3:PutBucketNotification, ALLOW, *, Ref above Role)
1x AWS::Lambda::Function (nodejs10.x, ZipFile exports.handler =...)

The Role, Policy and Lambda Function should not be created.

What did you expect to happen?

A SINGLE SNS S3 Notification Event should be created. The Role, Policy and Lambda Function should not be created.

What actually happened?

The SNS S3 Notificattion Event AND a Lambda Function was created

Environment

• CLI Version : 1.57.0 (build 2ccfc50)
• Framework Version: npm 6.14.7
• Node.js Version: v12.16.1
• OS : Windows 10 Enterprise
• Language (Version): Python 3.8.2

Other

Here is the Lambda as produced in the synth cfn json:

"BucketNotificationsHandler050a0587b7544547bf325f094a3db8347ECC3691": { "Type": "AWS::Lambda::Function", "Properties": { "Description": "AWS CloudFormation handler for \"Custom::S3BucketNotifications\" resources (@aws-cdk/aws-s3)", "Code": { "ZipFile": "exports.handler = (event, context) => {\n // eslint-disable-next-line @typescript-eslint/no-require-imports, import/no-extraneous-dependencies\n const s3 = new (require('aws-sdk').S3)();\n // eslint-disable-next-line @typescript-eslint/no-require-imports\n const https = require('https');\n // eslint-disable-next-line @typescript-eslint/no-require-imports\n const url = require('url');\n log(JSON.stringify(event, undefined, 2));\n const props = event.ResourceProperties;\n if (event.RequestType === 'Delete') {\n props.NotificationConfiguration = {}; // this is how you clean out notifications\n }\n const req = {\n Bucket: props.BucketName,\n NotificationConfiguration: props.NotificationConfiguration,\n };\n return s3.putBucketNotificationConfiguration(req, (err, data) => {\n log({ err, data });\n if (err) {\n return submitResponse('FAILED', err.message +\nMore information in CloudWatch Log Stream: ${context.logStreamName});\n }\n else {\n return submitResponse('SUCCESS');\n }\n });\n function log(obj) {\n console.error(event.RequestId, event.StackId, event.LogicalResourceId, obj);\n }\n // tslint:disable-next-line:max-line-length\n // adapted from https://docs.aws.amazon.com/AWSCloudFormation/latest/UserGuide/aws-properties-lambda-function-code.html#cfn-lambda-function-code-cfnresponsemodule\n // to allow sending an error messge as a reason.\n function submitResponse(responseStatus, reason) {\n const responseBody = JSON.stringify({\n Status: responseStatus,\n Reason: reason || 'See the details in CloudWatch Log Stream: ' + context.logStreamName,\n PhysicalResourceId: event.PhysicalResourceId || event.LogicalResourceId,\n StackId: event.StackId,\n RequestId: event.RequestId,\n LogicalResourceId: event.LogicalResourceId,\n NoEcho: false,\n });\n log({ responseBody });\n const parsedUrl = url.parse(event.ResponseURL);\n const options = {\n hostname: parsedUrl.hostname,\n port: 443,\n path: parsedUrl.path,\n method: 'PUT',\n headers: {\n 'content-type': '',\n 'content-length': responseBody.length,\n },\n };\n const request = https.request(options, (r) => {\n log({ statusCode: r.statusCode, statusMessage: r.statusMessage });\n context.done();\n });\n request.on('error', (error) => {\n log({ sendError: error });\n context.done();\n });\n request.write(responseBody);\n request.end();\n }\n};" }, "Handler": "index.handler", "Role": { "Fn::GetAtt": [ "BucketNotificationsHandler050a0587b7544547bf325f094a3db834RoleB6FB88EC", "Arn" ] }, "Runtime": "nodejs10.x", "Timeout": 300 }, "DependsOn": [ "BucketNotificationsHandler050a0587b7544547bf325f094a3db834RoleDefaultPolicy2CF63D36", "BucketNotificationsHandler050a0587b7544547bf325f094a3db834RoleB6FB88EC" ], "Metadata": { "aws:cdk:path": "pylambda-s3buckets-stackid/BucketNotificationsHandler050a0587b7544547bf325f094a3db834/Resource" } }

---

This is 🐛 Bug Report

[iliapolo]

Hi @ColWillis

The lambda you are referring to is not a notification target. It is used internally by CDK to apply configuration notifications on the bucket. That is, its a custom resource that run s3.putNotificationConfiguration on the bucket.

The reason we need this is to avoid circular dependencies between the bucket and the target, where the target depends on the bucket in order to configure strict permission policies (i.e only allow this bucket to send me notifications), and the bucket depends on the target because s3 validates the target of a notification exists before creating the bucket.

By using a custom resource, we allow configuring these strict policies, and avoiding the need for * permissions. So what happens is:

1. Bucket gets created.
2. Target is created with a policy that only allows said bucket to access it.
3. The custom resource now updates the bucket configuration with the relevant notification.

If you look at your bucket configuration, you'll only see one SNS notification configuration, as intended.

I'm closing this issue for now, let us know if something still isn't clear.

Thanks

[ColWillis]

That makes sense now, thanks.

I think the Documentation should be updated to state this though?

(https://docs.aws.amazon.com/cdk/api/latest/python/aws_cdk.aws_s3_notifications.README.html#example)

[iliapolo]

This is a rather internal implementation detail that shouldn't be of much concern to users. Since he haven't seen much confusion around it, we will leave it as to not overload users.

Thanks!

[prafed]

This is no longer needed now that S3 supports EventBridge events. The "addEventNotification" can just create an EventBridge rule and forward any relevant events to the lambda

[namedgraph]

@prafed can you please point to an example of EventBridge?

[miertschink]

@iliapolo
Is there any possibility to define the execution role for this BucketNotificationsHandler Lambda?
In my case I got a pretty restrictive bucket policy.
I got an explizit deny for all actions on the bucket, except for some specific roles.
I want to set one of these roles as the execution role for this BucketNotificationsHandler

Is there any chance do so?

thx