diff --git a/custom-http-headers-to-sqs-message-attributes-using-http-api-gateway/Architecture.png b/custom-http-headers-to-sqs-message-attributes-using-http-api-gateway/Architecture.png new file mode 100644 index 000000000..4b2b0add7 Binary files /dev/null and b/custom-http-headers-to-sqs-message-attributes-using-http-api-gateway/Architecture.png differ diff --git a/custom-http-headers-to-sqs-message-attributes-using-http-api-gateway/CODE_OF_CONDUCT.md b/custom-http-headers-to-sqs-message-attributes-using-http-api-gateway/CODE_OF_CONDUCT.md new file mode 100644 index 000000000..5b627cfa6 --- /dev/null +++ b/custom-http-headers-to-sqs-message-attributes-using-http-api-gateway/CODE_OF_CONDUCT.md @@ -0,0 +1,4 @@ +## Code of Conduct +This project has adopted the [Amazon Open Source Code of Conduct](https://aws.github.io/code-of-conduct). +For more information see the [Code of Conduct FAQ](https://aws.github.io/code-of-conduct-faq) or contact +opensource-codeofconduct@amazon.com with any additional questions or comments. diff --git a/custom-http-headers-to-sqs-message-attributes-using-http-api-gateway/CONTRIBUTING.md b/custom-http-headers-to-sqs-message-attributes-using-http-api-gateway/CONTRIBUTING.md new file mode 100644 index 000000000..c4b6a1c50 --- /dev/null +++ b/custom-http-headers-to-sqs-message-attributes-using-http-api-gateway/CONTRIBUTING.md @@ -0,0 +1,59 @@ +# Contributing Guidelines + +Thank you for your interest in contributing to our project. Whether it's a bug report, new feature, correction, or additional +documentation, we greatly value feedback and contributions from our community. + +Please read through this document before submitting any issues or pull requests to ensure we have all the necessary +information to effectively respond to your bug report or contribution. + + +## Reporting Bugs/Feature Requests + +We welcome you to use the GitHub issue tracker to report bugs or suggest features. + +When filing an issue, please check existing open, or recently closed, issues to make sure somebody else hasn't already +reported the issue. Please try to include as much information as you can. Details like these are incredibly useful: + +* A reproducible test case or series of steps +* The version of our code being used +* Any modifications you've made relevant to the bug +* Anything unusual about your environment or deployment + + +## Contributing via Pull Requests +Contributions via pull requests are much appreciated. Before sending us a pull request, please ensure that: + +1. You are working against the latest source on the *main* branch. +2. You check existing open, and recently merged, pull requests to make sure someone else hasn't addressed the problem already. +3. You open an issue to discuss any significant work - we would hate for your time to be wasted. + +To send us a pull request, please: + +1. Fork the repository. +2. Modify the source; please focus on the specific change you are contributing. If you also reformat all the code, it will be hard for us to focus on your change. +3. Ensure local tests pass. +4. Commit to your fork using clear commit messages. +5. Send us a pull request, answering any default questions in the pull request interface. +6. Pay attention to any automated CI failures reported in the pull request, and stay involved in the conversation. + +GitHub provides additional document on [forking a repository](https://help.github.com/articles/fork-a-repo/) and +[creating a pull request](https://help.github.com/articles/creating-a-pull-request/). + + +## Finding contributions to work on +Looking at the existing issues is a great way to find something to contribute on. As our projects, by default, use the default GitHub issue labels (enhancement/bug/duplicate/help wanted/invalid/question/wontfix), looking at any 'help wanted' issues is a great place to start. + + +## Code of Conduct +This project has adopted the [Amazon Open Source Code of Conduct](https://aws.github.io/code-of-conduct). +For more information see the [Code of Conduct FAQ](https://aws.github.io/code-of-conduct-faq) or contact +opensource-codeofconduct@amazon.com with any additional questions or comments. + + +## Security issue notifications +If you discover a potential security issue in this project we ask that you notify AWS/Amazon Security via our [vulnerability reporting page](http://aws.amazon.com/security/vulnerability-reporting/). Please do **not** create a public github issue. + + +## Licensing + +See the [LICENSE](LICENSE) file for our project's licensing. We will ask you to confirm the licensing of your contribution. diff --git a/custom-http-headers-to-sqs-message-attributes-using-http-api-gateway/README.md b/custom-http-headers-to-sqs-message-attributes-using-http-api-gateway/README.md new file mode 100644 index 000000000..06493ecae --- /dev/null +++ b/custom-http-headers-to-sqs-message-attributes-using-http-api-gateway/README.md @@ -0,0 +1,109 @@ +# HTTP API Gateway to SQS for passing custom http headers as message attributes. + +This pattern enables you to pass custom HTTP headers as message attributes when sending messages from HTTP API Gateway to an SQS queue. The headers can be configured either as static values or dynamically passed from the incoming request headers. In the default configuration, the message attribute name is set as 'MessageAttribute1' which maps to the header name 'header1' in the integration request mapping. You can customize these message attribute, header names and static values, according to your requirements by updating the requestParameters section in the SqsIntegration configuration within your SAM template. You can set the name for API Gateway and the SQS queue. + +Important: this application uses various AWS services and there are costs associated with these services after the Free Tier usage - please see the AWS Pricing page for details. You are responsible for any AWS costs incurred. No warranty is implied in this example. + +## Requirements + +Create an AWS account if you do not already have one and log in. The IAM user that you use must have sufficient permissions to make necessary AWS service calls and manage AWS resources. +- AWS CLI installed and configured +- Git Installed +- AWS Serverless Application Model (AWS SAM) installed +- In the default configuration, the message attribute name is set as 'MessageAttribute1' which maps to the header name 'header1' in the integration request mapping. You can customize these message attribute, header names and static values, according to your requirements by updating the requestParameters section in the SqsIntegration configuration within your SAM template. You can set the name for API Gateway and the SQS queue. + +## Deployment Instructions: + +1. Create a new directory, navigate to that directory in a terminal and clone the GitHub repository: + +``` +git clone https://github.com/aws-samples/serverless-patterns +``` +2. Change directory to the pattern directory: + +``` +cd _patterns-model custom-http-headers-to-sqs-message-attributes-using-http-api-gateway +``` +3. From the command line, use AWS SAM to deploy the AWS resources for the pattern as specified in the template.yml file: + +``` +sam deploy --guided --capabilities CAPABILITY_AUTO_EXPAND CAPABILITY_IAM CAPABILITY_NAMED_IAM +``` + +4. During the prompts enter the values corresponding to each field. The values in the square brackets are the default values, which can be overwritten once you enter the inputs. + + Stack Name: + AWS Region: + Parameter ApiGatewayName: + Parameter QueueName: + #Shows you resources changes to be deployed and require a 'Y' to initiate deploy + Confirm changes before deploy [Y/n]: Y + #SAM needs permission to be able to create roles to connect to the resources in your template + Allow SAM CLI IAM role creation [Y/n]: + #Preserves the state of previously provisioned resources when an operation fails + Disable rollback [y/N]: + Save arguments to configuration file [Y/n]: + SAM configuration file [samconfig.toml]: + SAM configuration environment [default]: + + +5. Allow SAM CLI to create IAM roles with the required permissions. +Once you have run sam deploy --guided mode once and saved arguments to a configuration file (samconfig.toml), you can use sam deploy in future to use these defaults. + +6. Note the outputs from the SAM deployment process. These contain the resource names and/or ARNs which are used for later review. + +## How it works: + +This pattern sets up the following resources: + +A HTTP API Gateway with SQS integration with configurations to map the http headers to message attributes for SQS. The HTTP API will have custom logging enabled and the logs will be sent to the Cloudwatch log group created by the pattern. An SQS queue will be created by the pattern and attached to the API Gateway with default parameters. + +In the output section, the SAM deployment returns ProvidedInputs, HttpApiEndpoint, HttpApiArn, HttpApiId, LogGroupArn, QueueArn, ApiUsageInformation, Required headers, Message body sample, RoleArn and QueueUrl. + +## Testing: + +Invoke the API Gateway with required headers and body and see the message being received with the headers as message attributes. + +- Example: (The following example is for the default configuration, you can modify the values based on your custom configuration) + CLI: curl --location 'https://.execute-api..amazonaws.com/sqs' \ + --header 'header1: value for header1 which will go as MessageAttribute1' \ + --header 'header2: value for header2 which will go as MessageAttribute2' \ + --header 'Content-Type: application/json' \ + --data '{ + "MessageBody": "Payload from client via HTTP API Gateway" + }' + +- Use Case requirements: + - Required Headers: + The following headers and their corresponding values are expected to passed along with the request when using this template + - header1's (key and value) is required for MessageAttribute1 + - header2's (key and value) is required for MessageAttribute2 + - 'static_header3' is the static value being configured for MessageAttribute3's header. Since is it configured this header is not mandatory when sending the request. + - Request Body Format: + { + "MessageBody": "Your message here" + } + + - Message received in SQS will have the following attributes. + Attributes (3) + Name: MessageAttribute1 | Type: String | Value: value for header1 which will go as MessageAttribute1 + Name: MessageAttribute2 | Type: String | Value: value for header2 which will go as MessageAttribute2 + Name: MessageAttribute3 | Type: String | Value: static_header3 + +## Cleanup +- Delete the stack: + + ``` + sam delete + ``` + +- Confirm the stack has been deleted + + ``` + aws cloudformation list-stacks --query "StackSummaries[?contains(StackName,'STACK_NAME')].StackStatus" + ``` + + +Copyright 2023 Amazon.com, Inc. or its affiliates. All Rights Reserved. + +SPDX-License-Identifier: MIT-0 \ No newline at end of file diff --git a/custom-http-headers-to-sqs-message-attributes-using-http-api-gateway/example-pattern.json b/custom-http-headers-to-sqs-message-attributes-using-http-api-gateway/example-pattern.json new file mode 100644 index 000000000..706dda10a --- /dev/null +++ b/custom-http-headers-to-sqs-message-attributes-using-http-api-gateway/example-pattern.json @@ -0,0 +1,51 @@ +{ + "title": "HTTP API Gateway to SQS for passing custom http headers as message attributes.", + "description": "HTTP API Gateway to SQS for passing custom http headers as message attributes.", + "level": "300", + "framework": "SAM", + "introBox": { + "headline": "How it works", + "text": [ + "This pattern enables you to pass custom HTTP headers as message attributes when sending messages from HTTP API Gateway to an SQS queue. The headers can be configured either as static values or dynamically passed from the incoming request headers. In the default configuration, the message attribute name is set as 'MessageAttribute1' which maps to the header name 'header1' in the integration request mapping. You can customize these message attribute, header names and static values, according to your requirements by updating the requestParameters section in the SqsIntegration configuration within your SAM template. You can set the name for API Gateway and the SQS queue." + ] + }, + "gitHub": { + "template": { + "repoURL": "https://github.com/aws-samples/serverless-patterns/tree/main/custom-http-headers-to-sqs-message-attributes-using-http-api-gateway", + "templateURL": "serverless-patterns/custom-http-headers-to-sqs-message-attributes-using-http-api-gateway", + "projectFolder": "custom-http-headers-to-sqs-message-attributes-using-http-api-gateway", + "templateFile": "custom-http-headers-to-sqs-message-attributes-using-http-api-gateway/template.yaml" + } + }, + "resources": { + "bullets": [ + { + "text": "HTTP API Gateway to SQS for passing custom http headers as message attributes.", + "link": "https://docs.aws.amazon.com/apigateway/latest/developerguide/http-api-develop-integrations-aws-services.html" + } + ] + }, + "deploy": { + "text": [ + "sam deploy" + ] + }, + "testing": { + "text": [ + "See the GitHub repo for detailed testing instructions." + ] + }, + "cleanup": { + "text": [ + "Delete the stack: sam delete" + ] + }, + "authors": [ + { + "name": "Annangarachari R", + "image": "https://media.licdn.com/dms/image/v2/C5603AQHDdhBrpBtWsg/profile-displayphoto-shrink_800_800/profile-displayphoto-shrink_800_800/0/1651409742725?e=1759363200&v=beta&t=7-EnqmtXjWlH2uA8oSMCwnLfFFvlWtf42-aC8NSZukw", + "bio": "Serverless Enthusiast", + "linkedin": "https://www.linkedin.com/in/annangarachari-r/", + } + ] +} diff --git a/custom-http-headers-to-sqs-message-attributes-using-http-api-gateway/template.yaml b/custom-http-headers-to-sqs-message-attributes-using-http-api-gateway/template.yaml new file mode 100644 index 000000000..e142435ff --- /dev/null +++ b/custom-http-headers-to-sqs-message-attributes-using-http-api-gateway/template.yaml @@ -0,0 +1,165 @@ +AWSTemplateFormatVersion: '2010-09-09' +Transform: AWS::Serverless-2016-10-31 + +Parameters: + ApiGatewayName: + Type: String + Description: Name of the API Gateway + QueueName: + Type: String + Description: Name of the SQS queue + +Resources: + # CloudWatch Log Group + ApiGatewayLogGroup: + Type: AWS::Logs::LogGroup + Properties: + LogGroupName: !Sub '/aws/apigateway/${ApiGatewayName}' + RetentionInDays: 30 + + # SQS Queue + MyQueue: + Type: AWS::SQS::Queue + Properties: + QueueName: !Ref QueueName + + # API Gateway Role + ApiGatewayRole: + Type: AWS::IAM::Role + Properties: + AssumeRolePolicyDocument: + Version: '2012-10-17' + Statement: + - Effect: Allow + Principal: + Service: apigateway.amazonaws.com + Action: sts:AssumeRole + Policies: + - PolicyName: ApiGatewaySQSAndCloudWatchPolicy + PolicyDocument: + Version: '2012-10-17' + Statement: + - Effect: Allow + Action: + - sqs:SendMessage + - sqs:GetQueueUrl + - sqs:SendMessageBatch + Resource: !GetAtt MyQueue.Arn + - Effect: Allow + Action: + - logs:CreateLogGroup + - logs:CreateLogStream + - logs:DescribeLogGroups + - logs:DescribeLogStreams + - logs:PutLogEvents + - logs:GetLogEvents + - logs:FilterLogEvents + Resource: !Sub 'arn:aws:logs:${AWS::Region}:${AWS::AccountId}:log-group:/aws/apigateway/${ApiGatewayName}:*' + + # HTTP API + MyHttpApi: + Type: AWS::Serverless::HttpApi + Properties: + Name: !Ref ApiGatewayName + StageName: $default + Tags: + Name: !Ref ApiGatewayName + AccessLogSettings: + DestinationArn: !GetAtt ApiGatewayLogGroup.Arn + Format: '{"account":"$context.accountId","apiId":"$context.apiId","awsEndpointRequestId":"$context.awsEndpointRequestId","domainName":"$context.domainName","domainPrefix":"$context.domainPrefix","error.message":"$context.error.message","error.responseType":"$context.error.responseType","extendedRequestId":"$context.extendedRequestId","httpMethod":"$context.httpMethod","identity.accountId":"$context.identity.accountId","identity.caller":"$context.identity.caller","identity.sourceIp":"$context.identity.sourceIp","identity.user":"$context.identity.user","identity.userAgent":"$context.identity.userAgent","identity.userArn":"$context.identity.userArn","path":"$context.path","protocol":"$context.protocol","requestId":"$context.requestId","requestTime":"$context.requestTime","requestTimeEpoch":"$context.requestTimeEpoch","resourcePath":"$context.resourcePath","stage":"$context.stage","integrationLatency":"$context.integrationLatency","integrationStatus":"$context.integrationStatus","responseLatency":"$context.responseLatency","responseLength":"$context.responseLength","status":"$context.status","integrationErrorMessage":"$context.integrationErrorMessage", "error.responseType":"$context.error.responseType"}' + DefaultRouteSettings: + ThrottlingBurstLimit: 100 + ThrottlingRateLimit: 50 + DisableExecuteApiEndpoint: false + FailOnWarnings: false + + # API Integration + SqsIntegration: + Type: AWS::ApiGatewayV2::Integration + Properties: + ApiId: !Ref MyHttpApi + IntegrationType: AWS_PROXY + IntegrationSubtype: SQS-SendMessage + CredentialsArn: !GetAtt ApiGatewayRole.Arn + RequestParameters: + QueueUrl: !Ref MyQueue + MessageBody: $request.body.MessageBody + MessageAttributes: >- + { + "MessageAttribute1": { + "DataType": "String", + "StringValue": "${request.header.header1}" + }, + "MessageAttribute2": { + "DataType": "String", + "StringValue": "${request.header.header2}" + }, + "MessageAttribute3": { + "DataType": "String", + "StringValue": "static_header3" + } + } + PayloadFormatVersion: '1.0' + + # Route + SqsRoute: + Type: AWS::ApiGatewayV2::Route + Properties: + ApiId: !Ref MyHttpApi + RouteKey: 'ANY /sqs' + Target: !Join + - / + - - integrations + - !Ref SqsIntegration + +Outputs: + # Resource ARNs and IDs + HttpApiId: + Description: HTTP API ID + Value: !Ref MyHttpApi + + HttpApiArn: + Description: HTTP API ARN + Value: !Sub arn:aws:apigateway:${AWS::Region}::/apis/${MyHttpApi} + + QueueUrl: + Description: URL of the SQS queue + Value: !Ref MyQueue + + QueueArn: + Description: ARN of the SQS queue + Value: !GetAtt MyQueue.Arn + + LogGroupArn: + Description: ARN of the CloudWatch Log Group + Value: !GetAtt ApiGatewayLogGroup.Arn + + RoleArn: + Description: ARN of the IAM Role + Value: !GetAtt ApiGatewayRole.Arn + + # API Endpoint and Usage Information + HttpApiEndpoint: + Description: API Gateway endpoint URL + Value: !Sub https://${MyHttpApi}.execute-api.${AWS::Region}.amazonaws.com + + ApiUsageInformation: + Description: API Usage Information + Value: | + Endpoint: POST https://${MyHttpApi}.execute-api.${AWS::Region}.amazonaws.com/sqs + Required Headers: + - The following headers and their corresponding values are expected to passed along with the request when using this template + - header1's (key and value) is required for MessageAttribute1 header1 + - header2's (key and value) is required for MessageAttribute2 header2 + - 'static_header3' is the static value being configured for MessageAttribute3 header. + Request Body Format: + { + "MessageBody": "Your message here" + } + + # Input Parameters Used + ProvidedInputs: + Description: Input parameters provided for this deployment + Value: !Sub | + API Gateway Name: ${ApiGatewayName} + Queue Name: ${QueueName}