New Serverless Pattern - cdk s3 lambda sns python

cdk-s3-lambda-sns-python/README.md

@@ -0,0 +1,90 @@
+# Amazon S3 to AWS Lambda to publishes Amazon SNS notification\
+
+This pattern creates a Amazon S3 bucket, a Lambda function, and an Amazon SNS topic.
+The Lambda function is triggered by S3 'OBJECT_CREATED' events. The Lambda function can optionally apply business logic to the S3 object. After processing, the Lambda function publishes an SNS notification of the new object.
+
+Important: this application uses various AWS services and there are costs associated with these services after the Free Tier usage - please see the [AWS Pricing page](https://aws.amazon.com/pricing/) for details. You are responsible for any AWS costs incurred. No warranty is implied in this example.
+
+## Requirements
+
+* [Create an AWS account](https://portal.aws.amazon.com/gp/aws/developer/registration/index.html) if you do not already have one and log in. The IAM user that you use must have sufficient permissions to make necessary AWS service calls and manage AWS resources.
+* [AWS CLI](https://docs.aws.amazon.com/cli/latest/userguide/install-cliv2.html) installed and configured
+* [Git Installed](https://git-scm.com/book/en/v2/Getting-Started-Installing-Git)
+* [AWS CDK](https://docs.aws.amazon.com/cdk/v2/guide/cli.html)
+* [Python, pip, virtualenv](https://docs.aws.amazon.com/cdk/latest/guide/work-with-cdk-python.html) installed
+
+## Deployment Instructions
+
+1. Create a new directory, navigate to that directory in a terminal and clone the GitHub repository:
+    ``` 
+    git clone https://github.com/aws-samples/serverless-patterns
+    ```
+2. Change directory to the pattern directory:
+    ```
+    cd cdk-s3-lambda-sns-python
+    ```
+3. Create a virtual environment for Python:
+   ```
+   python3 -m venv .venv
+   ```
+4. Activate the virtual environment
+   ```
+   source .venv/bin/activate
+   ```
+   For a Windows platform, activate the virtualenv like this:
+   ```
+   .venv\Scripts\activate.bat
+5. Install the Python required dependencies:
+   ```
+   pip install -r requirements.txt
+   ```
+6. Review the CloudFormation template the cdk generates for you stack using the following AWS CDK CLI command:
+   ```
+   cdk synth
+   ```
+7. Run the command below to bootstrap your account CDK needs it to deploy
+    ```
+    cdk bootstrap
+    ```
+8. From the command line, use CDK to deploy the AWS resources for the pattern. You'll be prompted to approve security related changes during the deployment.
+    ```
+    cdk deploy --parameters Email=<EMAIL_ID_THAT_YOU_WANT_TO_SUSCRIBE_TO_SNS_TOPIC>
+    ```
+9. Once the deployment is complete, you should receive a email to the email id that was used in the previous step to confirm the Amazon SNS subscription as shown below. Follow the link in the email to confirm the subscription.
+    ![SNS subscription confirmation](images/Subscription_Confirmation.png)
+
+## How it works
+
+* Any object uploaded to the Amazon S3 bucket will trigger the S3 Create Object event and invoke the AWS Lambda associated with the event.
+The AWS Lambdaambda function will get the Amazon S3 bucket name and object key from the event. This informatation is used to read the object and apply business logic if any before triggering the Amazon SNS topic with the required message.
+
+## Testing
+
+1. Navigate to the S3 bucket containing key 'cdks3lambdasnspythonstack-cdks3lambdasns**' along with your stack name and a hash key
+2. Upload any object to the bucket
+3. Wait for the AWS Lambda function to process the S3 event notification and send message to the email subscribed to the SNS topic. It should look like below sample
+
+![Email from SNS subscription](images/SNS_Notification.png)
+
+### Making changes
+
+You can customize the AWS Lambda Functions by editing the code at `./src/lambda_function.py`. To deploy changes, use the `cdk deploy` command.
+
+## Cleanup
+
+1. Delete the stack
+    ```
+    cdk destroy
+    ```
+
+2. Confirm the stack has been deleted. Login to the AWS console and navigate to the AWS Cloudformation service page "CdkServerlessInfraStack" is deleted or run the below 
+    ```bash
+    aws cloudformation list-stacks --query "StackSummaries[?contains(StackName,'CdkS3LambdaSnsPythonStack')].StackStatus"
+    ```
+
+You should expect to see a message confirming `DELETE_COMPLETE`.
+
+----
+Copyright 2024 Amazon.com, Inc. or its affiliates. All Rights Reserved.
+
+SPDX-License-Identifier: MIT-0

cdk-s3-lambda-sns-python/app.py

@@ -0,0 +1,28 @@
+#!/usr/bin/env python3
+import os
+
+import aws_cdk as cdk
+
+from src.cdk_s3_lambda_sns_python_stack import CdkS3LambdaSnsPythonStack
+
+
+app = cdk.App()
+CdkS3LambdaSnsPythonStack(app, "CdkS3LambdaSnsPythonStack",
+    # If you don't specify 'env', this stack will be environment-agnostic.
+    # Account/Region-dependent features and context lookups will not work,
+    # but a single synthesized template can be deployed anywhere.
+
+    # Uncomment the next line to specialize this stack for the AWS Account
+    # and Region that are implied by the current CLI configuration.
+
+    #env=cdk.Environment(account=os.getenv('CDK_DEFAULT_ACCOUNT'), region=os.getenv('CDK_DEFAULT_REGION')),
+
+    # Uncomment the next line if you know exactly what Account and Region you
+    # want to deploy the stack to. */
+
+    #env=cdk.Environment(account='123456789012', region='us-east-1'),
+
+    # For more information, see https://docs.aws.amazon.com/cdk/latest/guide/environments.html
+    )
+
+app.synth()

cdk-s3-lambda-sns-python/cdk-s3-lambda-sns-python.json

@@ -0,0 +1,86 @@
+{
+    "title": "Triggering Serverless Workflows Amazon S3 to AWS Lambda to Amazon SNS",
+    "description": "S3 upload events will trigger Lambda functions to process new objects. After processing the Lambda will publish an SNS notification.",
+    "language": "Python",
+    "level": "200",
+    "framework": "CDK",
+    "introBox": {
+        "headline": "How it works",
+        "text": [
+            "This pattern creates a Amazon S3 bucket, a Lambda function, and an Amazon SNS topic. ",
+            "The Lambda function is triggered by S3 'OBJECT_CREATED' events. The Lambda function can optionally apply business logic to the S3 object. After processing, the Lambda function publishes an SNS notification of the new object.",
+            "Important: this application uses various AWS services and there are costs associated with these services after the Free Tier usage - please see the [AWS Pricing page](https://aws.amazon.com/pricing/) for details. You are responsible for any AWS costs incurred. No warranty is implied in this example."
+        ]
+    },
+    "gitHub": {
+        "template": {
+            "repoURL": "https://github.com/aws-samples/serverless-patterns/tree/main/cdk-s3-lambda-sns-python",
+            "templateURL": "serverless-patterns/cdk-s3-lambda-sns-python",
+            "projectFolder": "cdk-s3-lambda-sns-python",
+            "templateFile": "src/cdk_s3_lambda_sns_python_stack.py"
+        }
+    },
+    "resources": {
+        "bullets": [
+            {
+                "text": "Getting started with the AWS CDK",
+                "link": "https://docs.aws.amazon.com/cdk/v2/guide/getting_started.html"
+            },
+            {
+                "text": "Using an amazon S3 trigger to invoke a Lambda function",
+                "link": "https://docs.aws.amazon.com/lambda/latest/dg/with-s3-example.html"
+            }
+        ]
+    },
+    "deploy": {
+        "text": [
+            "cdk deploy --parameters Email=<EMAIL_ID_THAT_YOU_WANT_TO_SUSCRIBE_TO_SNS_TOPIC>"
+        ]
+    },
+    "testing": {
+        "text": [
+            "See the GitHub repo for detailed testing instructions."
+        ]
+    },
+    "cleanup": {
+        "text": [
+            "Delete the stack: <code>cdk destroy</code>."
+        ]
+    },
+    "authors": [
+        {
+            "name": "Dinesh Balaaji Prabakaran",
+            "image": "https://avatars.githubusercontent.com/u/43818089?s=400&u=8a1e4fabab318016c485049108a2012b760e9ff7&v=4",
+            "bio": "I am a Senior Technical Account Manager with AWS, I specialize in architecting and developing serverless, event-driven solutions on AWS. AWS Storage Technical Field Community member.",
+            "linkedin": "pdineshbalaaji"
+        }
+    ],
+    "patternArch": {
+        "icon1": {
+            "x": 20,
+            "y": 50,
+            "service": "s3",
+            "label": "Amazon S3"
+        },
+        "icon2": {
+            "x": 50,
+            "y": 50,
+            "service": "lambda",
+            "label": "AWS Lambda"
+        },
+        "icon3": {
+            "x": 80,
+            "y": 50,
+            "service": "sns",
+            "label": "Amazon SNS"
+        },
+        "line1": {
+            "from": "icon1",
+            "to": "icon2"
+        },
+        "line2": {
+            "from": "icon2",
+            "to": "icon3"
+        }
+    }
+}

cdk-s3-lambda-sns-python/cdk.json

@@ -0,0 +1,61 @@
+{
+  "app": "python3 app.py",
+  "watch": {
+    "include": [
+      "**"
+    ],
+    "exclude": [
+      "README.md",
+      "cdk*.json",
+      "requirements*.txt",
+      "source.bat",
+      "**/__init__.py",
+      "**/__pycache__",
+      "tests"
+    ]
+  },
+  "context": {
+    "@aws-cdk/aws-lambda:recognizeLayerVersion": true,
+    "@aws-cdk/core:checkSecretUsage": true,
+    "@aws-cdk/core:target-partitions": [
+      "aws",
+      "aws-cn"
+    ],
+    "@aws-cdk-containers/ecs-service-extensions:enableDefaultLogDriver": true,
+    "@aws-cdk/aws-ec2:uniqueImdsv2TemplateName": true,
+    "@aws-cdk/aws-ecs:arnFormatIncludesClusterName": true,
+    "@aws-cdk/aws-iam:minimizePolicies": true,
+    "@aws-cdk/core:validateSnapshotRemovalPolicy": true,
+    "@aws-cdk/aws-codepipeline:crossAccountKeyAliasStackSafeResourceName": true,
+    "@aws-cdk/aws-s3:createDefaultLoggingPolicy": true,
+    "@aws-cdk/aws-sns-subscriptions:restrictSqsDescryption": true,
+    "@aws-cdk/aws-apigateway:disableCloudWatchRole": true,
+    "@aws-cdk/core:enablePartitionLiterals": true,
+    "@aws-cdk/aws-events:eventsTargetQueueSameAccount": true,
+    "@aws-cdk/aws-iam:standardizedServicePrincipals": true,
+    "@aws-cdk/aws-ecs:disableExplicitDeploymentControllerForCircuitBreaker": true,
+    "@aws-cdk/aws-iam:importedRoleStackSafeDefaultPolicyName": true,
+    "@aws-cdk/aws-s3:serverAccessLogsUseBucketPolicy": true,
+    "@aws-cdk/aws-route53-patters:useCertificate": true,
+    "@aws-cdk/customresources:installLatestAwsSdkDefault": false,
+    "@aws-cdk/aws-rds:databaseProxyUniqueResourceName": true,
+    "@aws-cdk/aws-codedeploy:removeAlarmsFromDeploymentGroup": true,
+    "@aws-cdk/aws-apigateway:authorizerChangeDeploymentLogicalId": true,
+    "@aws-cdk/aws-ec2:launchTemplateDefaultUserData": true,
+    "@aws-cdk/aws-secretsmanager:useAttachedSecretResourcePolicyForSecretTargetAttachments": true,
+    "@aws-cdk/aws-redshift:columnId": true,
+    "@aws-cdk/aws-stepfunctions-tasks:enableEmrServicePolicyV2": true,
+    "@aws-cdk/aws-ec2:restrictDefaultSecurityGroup": true,
+    "@aws-cdk/aws-apigateway:requestValidatorUniqueId": true,
+    "@aws-cdk/aws-kms:aliasNameRef": true,
+    "@aws-cdk/aws-autoscaling:generateLaunchTemplateInsteadOfLaunchConfig": true,
+    "@aws-cdk/core:includePrefixInUniqueNameGeneration": true,
+    "@aws-cdk/aws-efs:denyAnonymousAccess": true,
+    "@aws-cdk/aws-opensearchservice:enableOpensearchMultiAzWithStandby": true,
+    "@aws-cdk/aws-lambda-nodejs:useLatestRuntimeVersion": true,
+    "@aws-cdk/aws-efs:mountTargetOrderInsensitiveLogicalId": true,
+    "@aws-cdk/aws-rds:auroraClusterChangeScopeOfInstanceParameterGroupWithEachParameters": true,
+    "@aws-cdk/aws-appsync:useArnForSourceApiAssociationIdentifier": true,
+    "@aws-cdk/aws-rds:preventRenderingDeprecatedCredentials": true
+  }
+}

cdk-s3-lambda-sns-python/example-pattern.json

@@ -0,0 +1,58 @@
+{
+    "title": "Triggering Serverless Workflows Amazon S3 to AWS Lambda to Amazon SNS",
+    "description": "S3 upload events will trigger Lambda functions to process new objects. After processing the Lambda will publish an SNS notification.",
+    "language": "Python",
+    "level": "200",
+    "framework": "CDK",
+    "introBox": {
+      "headline": "How it works",
+      "text": [
+        "This pattern creates a Amazon S3 bucket, a Lambda function, and an Amazon SNS topic. ", 
+        "The Lambda function is triggered by S3 'OBJECT_CREATED' events. The Lambda function can optionally apply business logic to the S3 object. After processing, the Lambda function publishes an SNS notification of the new object.",
+        "Important: this application uses various AWS services and there are costs associated with these services after the Free Tier usage - please see the [AWS Pricing page](https://aws.amazon.com/pricing/) for details. You are responsible for any AWS costs incurred. No warranty is implied in this example."
+      ]
+    },
+    "gitHub": {
+      "template": {
+        "repoURL": "https://github.com/aws-samples/serverless-patterns/tree/main/cdk-s3-lambda-sns-python",
+        "templateURL": "serverless-patterns/cdk-s3-lambda-sns-python",
+        "projectFolder": "cdk-s3-lambda-sns-python",
+        "templateFile": "src/cdk_s3_lambda_sns_python_stack.py"
+      }
+    },
+    "resources": {
+      "bullets": [
+        {
+            "text": "Getting started with the AWS CDK",
+            "link": "https://docs.aws.amazon.com/cdk/v2/guide/getting_started.html"
+        },
+        {
+          "text": "Using an amazon S3 trigger to invoke a Lambda function",
+          "link": "https://docs.aws.amazon.com/lambda/latest/dg/with-s3-example.html"
+        }
+      ]
+    },
+    "deploy": {
+      "text": [
+        "cdk deploy --parameters Email=<EMAIL_ID_THAT_YOU_WANT_TO_SUSCRIBE_TO_SNS_TOPIC>"
+      ]
+    },
+    "testing": {
+      "text": [
+        "See the GitHub repo for detailed testing instructions."
+      ]
+    },
+    "cleanup": {
+      "text": [
+        "Delete the stack: <code>cdk destroy</code>."
+      ]
+    },
+    "authors": [
+        {
+            "name": "Dinesh Balaaji Prabakaran",
+            "image": "https://avatars.githubusercontent.com/u/43818089?s=400&u=8a1e4fabab318016c485049108a2012b760e9ff7&v=4",
+            "bio": "I am a Senior Technical Account Manager with AWS, I specialize in architecting and developing serverless, event-driven solutions on AWS. AWS Storage Technical Field Community member.",
+            "linkedin": "pdineshbalaaji"
+        }
+    ]
+  }

cdk-s3-lambda-sns-python/package.json

@@ -0,0 +1,6 @@
+{
+	"dependencies": {
+		"aws-cdk": "2.103.1"
+	},
+	"Transform": "AWS::Serverless-2016-10-31"
+}

cdk-s3-lambda-sns-python/requirements-dev.txt

@@ -0,0 +1 @@
+pytest==6.2.5

cdk-s3-lambda-sns-python/requirements.txt

@@ -0,0 +1,2 @@
+aws-cdk-lib==2.103.1
+constructs>=10.0.0,<11.0.0

cdk-s3-lambda-sns-python/src/cdk_s3_lambda_sns_python_stack.py

@@ -0,0 +1,49 @@
+import aws_cdk as cdk
+from aws_cdk import (
+    aws_lambda as lambda_,
+    aws_lambda_event_sources as eventsources,
+    aws_s3 as s3
+)
+
+class CdkS3LambdaSnsPythonStack(cdk.Stack):
+
+    def __init__(self, scope: cdk.App, construct_id: str, **kwargs) -> None:
+        super().__init__(scope, construct_id, **kwargs)
+
+        self.email = cdk.CfnParameter(
+            self,
+            "Email",
+            type="String",
+            description="Email address for subscription"
+        )
+
+        #create a S3 bucket
+        bucket = s3.Bucket(self, "cdk-s3-lambda-sns", versioned=True, block_public_access=s3.BlockPublicAccess.BLOCK_ALL,
+            enforce_ssl=True,
+            removal_policy=cdk.RemovalPolicy.DESTROY,
+            auto_delete_objects=True,)
+
+        # Define SNS topic
+        topic = cdk.aws_sns.Topic(self, "S3LambdaSNSTopic")
+
+        # add SNS subscription to SNS topic
+        topic.add_subscription(cdk.aws_sns_subscriptions.EmailSubscription(self.email.value_as_string))
+
+        # Define Lambda function
+        my_function = lambda_.Function(
+            self, "MyFunction",
+            runtime=lambda_.Runtime.PYTHON_3_9,
+            code=lambda_.Code.from_asset("src"),
+            handler="lambda_function.lambda_handler"
+        )
+        # Add environment variable to Lambda function
+        my_function.add_environment("TOPIC_ARN", topic.topic_arn)
+
+        # Grant Lambda permission to access S3
+        bucket.grant_put(my_function) 
+
+        # Grant Lambda permission to publish to SNS topic
+        topic.grant_publish(my_function)
+
+        # add event source to lambda function
+        my_function.add_event_source(eventsources.S3EventSource(bucket, events=[s3.EventType.OBJECT_CREATED]))