Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

feat: Add support for starting position AT_TIMESTAMP to MSK events #12034

Conversation

griffithsbs
Copy link
Contributor

Closes: #12033

  • Add support for a StartingPosition of AT_TIMESTAMP to MSK event source mappings. When this configuration is used,
    StartingPositionTimestamp must also be provided, in Unix Time seconds, which is consistent with the existing support in the kafka event

  • Support for this starting position has recently been added to AWS Lambda event source mappings using MSK
    (https://aws.amazon.com/about-aws/whats-new/2023/06/aws-lambda-starting-timestamp-kafka-sources/)

  • Since runServerless tests are expensive, test the happy path in the existing test of a valid configuration rather than adding an explicit separate test for it, but add a new test to explicitly verify the unhappy path in which a timestamp at which to start is not provided

- Resolves serverless#12033

- Add support for a StartingPosition of AT_TIMESTAMP to MSK event
source mappings. When this configuration is used,
StartingPositionTimestamp must also be provided, in Unix Time seconds,
which is consistent with the existing support in the `kafka` event

- Support for this starting position has recently been added to AWS
Lambda event source mappings using MSK
(https://aws.amazon.com/about-aws/whats-new/2023/06/aws-lambda-starting-timestamp-kafka-sources/)

- Since `runServerless` tests are expensive, test the happy path in the
existing test of a valid configuration rather than adding an explicit
separate test for it, but add a new test to explicitly verify the unhappy
path in which a timestamp at which to start is not provided
@codecov
Copy link

codecov bot commented Jun 22, 2023

Codecov Report

Patch coverage: 100.00% and no project coverage change.

Comparison is base (0382fac) 86.70% compared to head (bf564d0) 86.71%.

❗ Current head bf564d0 differs from pull request most recent head d8a3722. Consider uploading reports for the commit d8a3722 to get more accurate results

Additional details and impacted files
@@           Coverage Diff           @@
##             main   #12034   +/-   ##
=======================================
  Coverage   86.70%   86.71%           
=======================================
  Files         316      316           
  Lines       13513    13520    +7     
=======================================
+ Hits        11717    11724    +7     
  Misses       1796     1796           
Impacted Files Coverage Δ
...ib/plugins/aws/package/compile/events/msk/index.js 100.00% <100.00%> (ø)
...ckage/compile/events/websockets/lib/authorizers.js 100.00% <100.00%> (ø)

☔ View full report in Codecov by Sentry.
📢 Do you have feedback about the report comment? Let us know in this issue.

Copy link
Contributor

@medikoo medikoo left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Looks great 👍

@medikoo medikoo merged commit 483ea16 into serverless:main Jun 22, 2023
5 checks passed
@kaykhan
Copy link

kaykhan commented Oct 12, 2023

@medikoo @griffithsbs I noticed in serverless.ts AT_TIMESTAMP is still not an option, note StartingPositionTimestamp is also not available. https://docs.aws.amazon.com/lambda/latest/dg/with-msk.html
image

@medikoo
Copy link
Contributor

medikoo commented Oct 12, 2023

@kaykhan serverless.ts is maintained by external contributors at https://github.com/serverless/typescript

@kaykhan
Copy link

kaykhan commented Oct 12, 2023

@kaykhan serverless.ts is maintained by external contributors at https://github.com/serverless/typescript

Oh i did not know that, that makes sense why some things are out of sync sometimes, thanks!

khacminh pushed a commit to khacminh/serverless that referenced this pull request Nov 29, 2023
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
Development

Successfully merging this pull request may close these issues.

Support startingPosition: AT_TIMESTAMP for MSK event source mappings
3 participants