sqs-to-lambda-with-dlq

sqs-to-lambda-with-dlq

This is an example implementation of a Lambda function triggered by an SQS message; if the Lambda function fails, the message is sent to the Dead Letter Queue, and another Lambda function processes the message sent to the Dead Letter Queue.

Resources

Lambda::Function

main

• Runtime: go1.x
• Timeout: 30 sec
• Memory: 128 MB
• Operation:
  • Invoked by sending a message to main SQS Queue
  • If throwError in the message is true, an error is returned. Otherwise, logs a simple message and exits.

handling dlq message

• Runtime: go1.x
• Timeout: 30 sec
• Memory: 128 MB
• Operation:
  • Invoked by sending a message to SQS Queue acts as dead letter queue.
  • Logs a simple message and exits.

SQS::Queue

main

• Events the received message and invokes the main Lambda function.

dlq

• Events the received message and invokes the Lambda function handling dead letter queue message.

Architecture

sequenceDiagram
  participant c as Client
  participant sqsMain as SQS Queue (main)
  participant lMain as Lambda Function (main)
  participant sqsDLQ as SQS Queue (Dead Letter Queue)
  participant lDLQ as Lambda Function (handling DLQ message)
  
  c ->> sqsMain: SendMessage `{throwMessage: bool}`
  sqsMain ->> lMain: Invoke Function
  
  alt throwMessage == false
    lMain ->> lMain: write INFO log
    lMain -->> sqsMain: dequeue
  end
  
  alt throwMessage == true
    lMain ->> lMain: write ERROR log
    
    loop 3 times
      sqsMain ->> lMain: Invoke
      lMain ->> lMain: write ERROR log
    end
    
    sqsMain ->> sqsDLQ: Move Message
    sqsDLQ ->> lDLQ: Invoke
    lDLQ ->> lDLQ: write INFO log
    lDLQ -->> sqsDLQ: dequeue
  end

Usage

Build the function

make build

Deploy

cdk synth

cdk deploy

Send message

• No error

  aws sqs send-message \
  --queue-url "$(aws sqs get-queue-url --queue-name 'aws-cdk-go-example-main-queue' --output text)" \
  --message-body "{\"throwError\":false}"

• Throw error

  aws sqs send-message \
  --queue-url "$(aws sqs get-queue-url --queue-name 'aws-cdk-go-example-main-queue' --output text)" \
  --message-body "{\"throwError\":true}"

Check CloudWatch Logs

• Logs of main function

  aws logs get-log-events \
  --log-group-name /aws/lambda/aws-cdk-go-example-main-function \
  --log-stream-name "$(
    aws logs describe-log-streams \
    --log-group-name /aws/lambda/aws-cdk-go-example-main-function \
    --query 'max_by(logStreams[], &lastEventTimestamp).logStreamName' \
    --output text)" \
  --limit 10 \
  --query 'events[].join(``, [ to_string(timestamp) ,`: `,message])' \
  --output text

• Logs of dlq function

  aws logs get-log-events \
  --log-group-name /aws/lambda/aws-cdk-go-example-dlq-function \
  --log-stream-name "$(
    aws logs describe-log-streams \
    --log-group-name /aws/lambda/aws-cdk-go-example-dlq-function \
    --query 'max_by(logStreams[], &lastEventTimestamp).logStreamName' \
    --output text)" \
  --limit 10 \
  --query 'events[].join(``, [ to_string(timestamp) ,`: `,message])' \
  --output text

Author

michimani210