Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We鈥檒l occasionally send you account related emails.

Already on GitHub? Sign in to your account

Add SQS event integration #5074

Merged
merged 4 commits into from Jul 3, 2018
Merged

Add SQS event integration #5074

merged 4 commits into from Jul 3, 2018

Conversation

@alexdebrie
Copy link
Contributor

alexdebrie commented Jun 25, 2018

What did you implement:

Adds SQS event integration 馃帀

How did you implement it:

EDIT 6/28: SQS integration is live! Full testing instructions below.

I saw Brett Andrews note that Lambda + SQS integration is coming soon. Then I peeked at the SAM implementation of the SQS event. It looks to be very similar to Kinesis / DynamoDB streams as it's a pull-based integration. Like the streams, you can set batch size. There is no need for the startingPosition parameter though.

I found the necessary IAM permissions by looking at the permissions on this role.

How can we verify it:

We need to wait until the integration is actually released. Once it is, you could use the following serverless.yml to test:

# serverless.yml

service: sqs

provider:
  name: aws
  runtime: nodejs6.10

functions:
  hello:
    handler: handler.hello
    events:
      - sqs:
          arn:
            Fn::GetAtt:
              - MyQueue
              - Arn

resources:
  Resources:
    MyQueue:
      Type: "AWS::SQS::Queue"
      Properties:
        QueueName: "MyQueue"

After you deploy, run the following commands with the awscli to put a message into your Queue:

$ QUEUE_URL=`aws sqs get-queue-url --queue-name MyQueue --query 'QueueUrl' --output text`
$ aws sqs send-message --queue-url ${QUEUE_URL} --message-body "test"

Then check to make sure your Lambda fired:

$ sls metrics
Service wide metrics
June 27, 2018 3:16 PM - June 28, 2018 3:16 PM

Invocations: 1
Throttles: 0
Errors: 0
Duration (avg.): 3.63ms

Todos:

  • Write tests
  • Write documentation
  • Fix linting errors
  • Make sure code coverage hasn't dropped
  • Provide verification config / commands / resources
  • Enable "Allow edits from maintainers" for this PR
  • Update the messages below

Is this ready for review?: YES
Is it a breaking change?: NO

@alexdebrie alexdebrie changed the title [WIP] Add SQS event integration Add SQS event integration Jun 28, 2018
@@ -46,6 +46,7 @@
"./aws/package/compile/events/cloudWatchEvent/index.js",
"./aws/package/compile/events/cloudWatchLog/index.js",
"./aws/package/compile/events/cognitoUserPool/index.js",
"./aws/package/compile/events/sqs/index.js",

This comment has been minimized.

Copy link
@jeremydaly

jeremydaly Jun 28, 2018

Contributor

I was wondering why this plugin wasn't running!

This comment has been minimized.

Copy link
@alexdebrie

alexdebrie Jun 28, 2018

Author Contributor

馃檲 Yea, took me a while to find this one 馃榿

events:
- sqs:
arn: arn:aws:sqs:region:XXXXXX:myQueue
batchSize: 100

This comment has been minimized.

Copy link
@jeremydaly

jeremydaly Jun 28, 2018

Contributor

10 is the maximum batch size allowed

This comment has been minimized.

Copy link
@alexdebrie

alexdebrie Jun 28, 2018

Author Contributor

Thanks! Updated.

@horike37 horike37 added this to the 1.28 milestone Jun 28, 2018
@domaslasauskas
Copy link

domaslasauskas commented Jun 29, 2018

Copy link
Member

horike37 left a comment

@alexdebrie
Great to see working fine 馃帀 馃帀 馃帀
LGTM

@horike37 horike37 merged commit cbc5e3c into master Jul 3, 2018
5 checks passed
5 checks passed
continuous-integration/appveyor/branch AppVeyor build succeeded
Details
continuous-integration/appveyor/pr AppVeyor build succeeded
Details
continuous-integration/travis-ci/pr The Travis CI build passed
Details
continuous-integration/travis-ci/push The Travis CI build passed
Details
coverage/coveralls Coverage increased (+0.02%) to 89.254%
Details
@horike37 horike37 deleted the SQS branch Jul 3, 2018
@pmuens
Copy link
Contributor

pmuens commented Jul 3, 2018

This is awesome! Thanks for implementing this @alexdebrie 馃憤 馃帀

@QAnders
Copy link

QAnders commented Jul 10, 2018

Any plans to implement the "Extended Client" functionality as in AWS Java SDK to support messages larger than 256kb by storing the payload/body on S3?

E.g.:

resources:
  Resources:
    MyQueue:
      Type: "AWS::SQS::Queue"
      Properties:
        ExtendedLargeMsg: "MyS3Bucket"
        QueueName: "MyQueue"
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
Linked issues

Successfully merging this pull request may close these issues.

None yet

7 participants
You can鈥檛 perform that action at this time.