New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We鈥檒l occasionally send you account related emails.

Already on GitHub? Sign in to your account

Add SQS event integration #5074

Merged
merged 4 commits into from Jul 3, 2018

Conversation

Projects
None yet
7 participants
@alexdebrie
Contributor

alexdebrie commented Jun 25, 2018

What did you implement:

Adds SQS event integration 馃帀

How did you implement it:

EDIT 6/28: SQS integration is live! Full testing instructions below.

I saw Brett Andrews note that Lambda + SQS integration is coming soon. Then I peeked at the SAM implementation of the SQS event. It looks to be very similar to Kinesis / DynamoDB streams as it's a pull-based integration. Like the streams, you can set batch size. There is no need for the startingPosition parameter though.

I found the necessary IAM permissions by looking at the permissions on this role.

How can we verify it:

We need to wait until the integration is actually released. Once it is, you could use the following serverless.yml to test:

# serverless.yml

service: sqs

provider:
  name: aws
  runtime: nodejs6.10

functions:
  hello:
    handler: handler.hello
    events:
      - sqs:
          arn:
            Fn::GetAtt:
              - MyQueue
              - Arn

resources:
  Resources:
    MyQueue:
      Type: "AWS::SQS::Queue"
      Properties:
        QueueName: "MyQueue"

After you deploy, run the following commands with the awscli to put a message into your Queue:

$ QUEUE_URL=`aws sqs get-queue-url --queue-name MyQueue --query 'QueueUrl' --output text`
$ aws sqs send-message --queue-url ${QUEUE_URL} --message-body "test"

Then check to make sure your Lambda fired:

$ sls metrics
Service wide metrics
June 27, 2018 3:16 PM - June 28, 2018 3:16 PM

Invocations: 1
Throttles: 0
Errors: 0
Duration (avg.): 3.63ms

Todos:

  • Write tests
  • Write documentation
  • Fix linting errors
  • Make sure code coverage hasn't dropped
  • Provide verification config / commands / resources
  • Enable "Allow edits from maintainers" for this PR
  • Update the messages below

Is this ready for review?: YES
Is it a breaking change?: NO

@alexdebrie alexdebrie changed the title from [WIP] Add SQS event integration to Add SQS event integration Jun 28, 2018

@@ -46,6 +46,7 @@
"./aws/package/compile/events/cloudWatchEvent/index.js",
"./aws/package/compile/events/cloudWatchLog/index.js",
"./aws/package/compile/events/cognitoUserPool/index.js",
"./aws/package/compile/events/sqs/index.js",

This comment has been minimized.

@jeremydaly

jeremydaly Jun 28, 2018

Contributor

I was wondering why this plugin wasn't running!

This comment has been minimized.

@alexdebrie

alexdebrie Jun 28, 2018

Contributor

馃檲 Yea, took me a while to find this one 馃榿

events:
- sqs:
arn: arn:aws:sqs:region:XXXXXX:myQueue
batchSize: 100

This comment has been minimized.

@jeremydaly

jeremydaly Jun 28, 2018

Contributor

10 is the maximum batch size allowed

This comment has been minimized.

@alexdebrie

alexdebrie Jun 28, 2018

Contributor

Thanks! Updated.

@horike37 horike37 added this to the 1.28 milestone Jun 28, 2018

@domaslasauskas

This comment has been minimized.

domaslasauskas commented Jun 29, 2018

@horike37

@alexdebrie
Great to see working fine 馃帀 馃帀 馃帀
LGTM

@horike37 horike37 merged commit cbc5e3c into master Jul 3, 2018

5 checks passed

continuous-integration/appveyor/branch AppVeyor build succeeded
Details
continuous-integration/appveyor/pr AppVeyor build succeeded
Details
continuous-integration/travis-ci/pr The Travis CI build passed
Details
continuous-integration/travis-ci/push The Travis CI build passed
Details
coverage/coveralls Coverage increased (+0.02%) to 89.254%
Details

@horike37 horike37 deleted the SQS branch Jul 3, 2018

@pmuens

This comment has been minimized.

Member

pmuens commented Jul 3, 2018

This is awesome! Thanks for implementing this @alexdebrie 馃憤 馃帀

@thepauleh thepauleh referenced this pull request Jul 3, 2018

Merged

Here come sqs events #1

@QAnders

This comment has been minimized.

QAnders commented Jul 10, 2018

Any plans to implement the "Extended Client" functionality as in AWS Java SDK to support messages larger than 256kb by storing the payload/body on S3?

E.g.:

resources:
  Resources:
    MyQueue:
      Type: "AWS::SQS::Queue"
      Properties:
        ExtendedLargeMsg: "MyS3Bucket"
        QueueName: "MyQueue"
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment