AWS Lambda response streaming for Lambda URLs #11907

grakic · 2023-04-09T13:59:13Z

Closes: #11906

dested · 2023-04-17T01:04:16Z

Any thoughts on when/if this will get merged?

Also any thoughts on if its safe enough to just patch on to my serverless project? It looks relatively simple, just not sure if its missing anything I should be aware of

medikoo

@grakic that looks really good! I have just a few mostly cosmetic suggestions

medikoo · 2023-05-09T08:58:24Z

docs/providers/aws/guide/functions.md

@@ -215,7 +215,9 @@ functions:
    url: true
 ```

-Alternatively, you can configure it as an object with the `authorizer` and/or `cors` properties. The `authorizer` property can be set to `aws_iam` to enable AWS IAM authorization on your function URL.
+Alternatively, you can configure it as an object with the `authorizer`, `cors` and/or `invoke` properties.


I'd word it as:

Alternatively, you can configure it as an object, and provide values for authorizer and invokeMode options

medikoo · 2023-05-09T08:59:36Z

docs/providers/aws/guide/functions.md

@@ -287,6 +289,16 @@ functions:
        allowedHeaders: null
 ```

+The `invoke` property can be set to `response_stream` to enable streaming response. If not specified, `buffered` invoke mode is assumed.


As outlined in issue, let's stick to invokeMode.

Let's also support AWS values literally so RESPONSE_STREAM and BUFFERED

medikoo · 2023-05-09T09:02:40Z

lib/plugins/aws/package/compile/functions.js

+    if (invoke) {
+      urlResource.Properties.InvokeMode = invoke;
+    }


I think simpler would be to simply contain whole resolution here as:

if (url.invoke === 'RESPONSE_STREAM) urlResource.Properties.InvokeMode = url.invoke;

medikoo · 2023-05-09T09:04:59Z

test/fixtures/programmatic/function/stream.js

@@ -0,0 +1,9 @@
+'use strict';
+
+// eslint-disable-next-line no-undef


Instead of doing that, let's maybe add awsLambda as global into ESLint config, for the lambda fixtures.

ESlint config is maintained in package.json

medikoo

Thank you @grakic it looks great. I have just one final small organization suggestion and we're ready to go

medikoo · 2023-05-16T06:46:47Z

package.json

@@ -132,6 +132,14 @@
          "sourceType": "module"
        }
      },
+      {


Let's reuse block at L117 (it's dedicated for fixtures)

test/integration/aws/function-url.test.js

harounansari-cj · 2023-05-27T00:49:13Z

Hey guys, Im frustrated that we've had lambda response streaming and SAM has had that feature for a while and Serverless Framework has lagged behind.

I could be missing something, is there another way to add response streaming to this function or do I have to wait for this PR:

functions:
  hello:
    handler: handler.hello
    url:
      # Allow CORS for all requests from any origin
      cors: true

I can do it in my SAM code like this:

  StreamingFunctionUrl:
    Type: AWS::Lambda::Url
    Properties:
      TargetFunctionArn: !Ref StreamingFunction
      AuthType: NONE
      InvokeMode: RESPONSE_STREAM

dested · 2023-05-27T01:03:23Z

Hey guys, Im frustrated that we've had lambda response streaming and SAM has had that feature for a while and Serverless Framework has lagged behind.

I ended up applying a patch using pnpm patching. It's for serverless version 3.30.1 but you should be able to make similar changes in whatever version your using.

serverless@3.30.1.patch:

diff --git a/lib/plugins/aws/package/compile/functions.js b/lib/plugins/aws/package/compile/functions.js
index 7de85631abe83463672973bef50f5cb1a264589f..0c0afe5ccc75c2f3039ed0f986f0aa9ed0058762 100644
--- a/lib/plugins/aws/package/compile/functions.js
+++ b/lib/plugins/aws/package/compile/functions.js
@@ -642,6 +642,7 @@ class AwsCompileFunctions {
 
     let auth = 'NONE';
     let cors = null;
+    let invoke=null
     if (url.authorizer === 'aws_iam') {
       auth = 'AWS_IAM';
     }
@@ -675,7 +676,13 @@ class AwsCompileFunctions {
 
       cors.maxAge = url.cors.maxAge;
     }
-
+    if (url.invoke) {
+      if (url.invoke === 'buffered') {
+        invoke = 'BUFFERED';
+      } else if (url.invoke === 'response_stream') {
+        invoke = 'RESPONSE_STREAM';
+      }
+    }
     const urlResource = {
       Type: 'AWS::Lambda::Url',
       Properties: {
@@ -696,6 +703,9 @@ class AwsCompileFunctions {
       };
     }
 
+    if (invoke) {
+      urlResource.Properties.InvokeMode = invoke;
+    }
     const logicalId = this.provider.naming.getLambdaFunctionUrlLogicalId(functionName);
     cfTemplate.Resources[logicalId] = urlResource;
     cfTemplate.Outputs[this.provider.naming.getLambdaFunctionUrlOutputLogicalId(functionName)] = {
diff --git a/lib/plugins/aws/provider.js b/lib/plugins/aws/provider.js
index c737ae300234fe733cdbdcdfcc5297bafe772401..5a8791d2f711b668658724a8086859948d772ca9 100644
--- a/lib/plugins/aws/provider.js
+++ b/lib/plugins/aws/provider.js
@@ -1495,6 +1495,10 @@ class AwsProvider {
                         },
                       ],
                     },
+                    invoke: {
+                      type: 'string',
+                      enum: ['buffered', 'response_stream'],
+                    },
                   },
                   additionalProperties: false,
                 },

In your package.json:

  "pnpm": {
    "patchedDependencies": {
      "serverless@3.30.1": "patches/serverless@3.30.1.patch"
    }
  },

Best of luck!

harounansari-cj · 2023-05-27T01:08:45Z

Hey guys, Im frustrated that we've had lambda response streaming and SAM has had that feature for a while and Serverless Framework has lagged behind.

I ended up applying a patch using pnpm patching. It's for serverless version 3.30.1 but you should be able to make similar changes in whatever version your using.

serverless@3.30.1.patch:

diff --git a/lib/plugins/aws/package/compile/functions.js b/lib/plugins/aws/package/compile/functions.js
index 7de85631abe83463672973bef50f5cb1a264589f..0c0afe5ccc75c2f3039ed0f986f0aa9ed0058762 100644
--- a/lib/plugins/aws/package/compile/functions.js
+++ b/lib/plugins/aws/package/compile/functions.js
@@ -642,6 +642,7 @@ class AwsCompileFunctions {
 
     let auth = 'NONE';
     let cors = null;
+    let invoke=null
     if (url.authorizer === 'aws_iam') {
       auth = 'AWS_IAM';
     }
@@ -675,7 +676,13 @@ class AwsCompileFunctions {
 
       cors.maxAge = url.cors.maxAge;
     }
-
+    if (url.invoke) {
+      if (url.invoke === 'buffered') {
+        invoke = 'BUFFERED';
+      } else if (url.invoke === 'response_stream') {
+        invoke = 'RESPONSE_STREAM';
+      }
+    }
     const urlResource = {
       Type: 'AWS::Lambda::Url',
       Properties: {
@@ -696,6 +703,9 @@ class AwsCompileFunctions {
       };
     }
 
+    if (invoke) {
+      urlResource.Properties.InvokeMode = invoke;
+    }
     const logicalId = this.provider.naming.getLambdaFunctionUrlLogicalId(functionName);
     cfTemplate.Resources[logicalId] = urlResource;
     cfTemplate.Outputs[this.provider.naming.getLambdaFunctionUrlOutputLogicalId(functionName)] = {
diff --git a/lib/plugins/aws/provider.js b/lib/plugins/aws/provider.js
index c737ae300234fe733cdbdcdfcc5297bafe772401..5a8791d2f711b668658724a8086859948d772ca9 100644
--- a/lib/plugins/aws/provider.js
+++ b/lib/plugins/aws/provider.js
@@ -1495,6 +1495,10 @@ class AwsProvider {
                         },
                       ],
                     },
+                    invoke: {
+                      type: 'string',
+                      enum: ['buffered', 'response_stream'],
+                    },
                   },
                   additionalProperties: false,
                 },

In your package.json:

  "pnpm": {
    "patchedDependencies": {
      "serverless@3.30.1": "patches/serverless@3.30.1.patch"
    }
  },

Best of luck!

thank you ill give this a shot

harounansari-cj · 2023-05-27T01:36:29Z

@dested Hmm I tried your approach and added it to my package json. I also tried manually updating my node_modules/.bin/serverless and ran node_modules/.bin/serverless but it doesnt seem to be changing the invoke mode:

medikoo · 2023-05-27T11:21:34Z

@harounansari-cj @dested it looks that this PR got abandoned, are you interested in finalizing this work?

grakic · 2023-05-27T21:10:44Z

@medikoo Can you review proposed updates and resolve conversations?

medikoo

Thank you @grakic, looks great 👍

test/integration/aws/function-url.test.js

codecov · 2023-05-29T10:50:39Z

Codecov Report

Patch coverage: 97.56% and project coverage change: +0.04 🎉

Comparison is base (a6bcc62) 86.68% compared to head (5f47a39) 86.73%.

❗ Current head 5f47a39 differs from pull request most recent head 41cb47e. Consider uploading reports for the commit 41cb47e to get more accurate results

Additional details and impacted files

@@            Coverage Diff             @@
##             main   #11907      +/-   ##
==========================================
+ Coverage   86.68%   86.73%   +0.04%     
==========================================
  Files         316      316              
  Lines       13440    13502      +62     
==========================================
+ Hits        11651    11711      +60     
- Misses       1789     1791       +2

Impacted Files	Coverage Δ
lib/cli/interactive-setup/console-dev-mode-feed.js	`88.96% <ø> (ø)`
...s/aws/package/compile/events/event-bridge/index.js	`97.72% <ø> (ø)`
lib/plugins/aws/provider.js	`94.78% <ø> (ø)`
lib/plugins/aws/package/compile/events/stream.js	`97.60% <83.33%> (-0.72%)`	⬇️
lib/plugins/aws/package/compile/events/schedule.js	`97.82% <97.87%> (-0.48%)`	⬇️
lib/cli/commands-schema/aws-service.js	`100.00% <100.00%> (ø)`
...b/cli/interactive-setup/console-enable-dev-mode.js	`98.70% <100.00%> (+0.03%)`	⬆️
lib/plugins/aws/deploy-function.js	`96.53% <100.00%> (ø)`
lib/plugins/aws/invoke-local/index.js	`69.87% <100.00%> (ø)`
lib/plugins/aws/lib/naming.js	`97.51% <100.00%> (+0.07%)`	⬆️
... and 3 more

☔ View full report in Codecov by Sentry.
📢 Do you have feedback about the report comment? Let us know in this issue.

jayarjo · 2024-03-09T10:01:30Z

So is it invokeMode or invoke?

sainiankit · 2024-03-25T11:27:58Z

What could be causing:

ERROR	TypeError: responseStream.setContentType is not a function
    at Object.<anonymous> (/var/task/dist/server.js:11:20)
    at Generator.next (<anonymous>)
    at /var/task/dist/server.js:7:71
    at new Promise (<anonymous>)
    at __awaiter (/var/task/dist/server.js:3:12)
    at /var/task/dist/server.js:10:85
    at /var/task/serverless_sdk/index.js:24:10716
    at /opt/sls-sdk-node/wrapper.js:13368:92
    at Runtime.handler (/opt/sls-sdk-node/wrapper.js:13375:11)
    at Runtime.handleOnceNonStreaming (file:///var/runtime/index.mjs:1173:29)

Lambda Handler looks like:

    declare const awslambda: any;

exports.handler = awslambda.streamifyResponse(
  async (_event: any, responseStream: any, _context: any) => {
    responseStream.setContentType("text/event-stream");
    setTimeout(() => responseStream.write("data: Hello, \n\n"), 1000);
    setTimeout(() => responseStream.write("data: World!\n\n"), 2000);
    setTimeout(() => responseStream.end(), 3000);
  }
);

serverless.yml liiks like:

service: streaming-lambda

provider:
  name: aws
  runtime: nodejs18.x
  stage: production
  region: ap-south-1
  memorySize: 256

functions:
  app:
    handler: dist/server.handler
    url:
      cors: true
      invokeMode: RESPONSE_STREAM
    timeout: 60

grakic · 2024-03-25T12:03:23Z

@sainiankit It seems that you are using serverless_sdk wrapper for your Lambda function.

I don't think serverless_sdk is supporting stream invoke mode, please see discussion here: #11906 (comment)

sainiankit · 2024-03-25T12:26:50Z

@grakic Sorry, Please could you explain it a little:

seems that you are using serverless_sdk wrapper for your Lambda function.

I have serverless.yml defined
I use github actions to call serverless deploy.
Where does serverless_sdk come into action ?

Is there a work around ?


      - name: serverless deploy
        uses: serverless/github-action@master
        with:
          args: deploy
        env:
          SERVERLESS_ACCESS_KEY: ${{ secrets.SERVERLESS_ACCESS_KEY }}
    ```

tobilg · 2024-03-25T14:26:08Z

You could have a look at my working example: https://github.com/tobilg/serverless-duckdb/blob/main/src/functions/streamingQuery.ts

grakic changed the title ~~Closes #11906 - Support AWS Lambda response streaming for Lambda URL~~ Closes #11906 - AWS Lambda response streaming for Lambda URLs Apr 9, 2023

grakic changed the title ~~Closes #11906 - AWS Lambda response streaming for Lambda URLs~~ AWS Lambda response streaming for Lambda URLs Apr 9, 2023

GrahamCampbell previously approved these changes Apr 23, 2023

View reviewed changes

medikoo suggested changes May 9, 2023

View reviewed changes

medikoo assigned grakic May 9, 2023

medikoo added enhancement cat/lambda labels May 9, 2023

grakic dismissed GrahamCampbell’s stale review via 420a590 May 14, 2023 13:53

grakic force-pushed the aws-lambda-streaming branch from feef40c to 420a590 Compare May 14, 2023 13:53

grakic requested a review from medikoo May 14, 2023 13:54

medikoo suggested changes May 16, 2023

View reviewed changes

Support AWS Lambda response streaming for Lambda URL

41cb47e

grakic force-pushed the aws-lambda-streaming branch from 420a590 to 41cb47e Compare May 27, 2023 21:30

medikoo approved these changes May 29, 2023

View reviewed changes

test/integration/aws/function-url.test.js Show resolved Hide resolved

medikoo merged commit 3afb71e into serverless:main May 29, 2023
5 checks passed

thekid mentioned this pull request Jun 17, 2023

Support streaming xp-forge/lambda#22

Closed

khacminh pushed a commit to khacminh/serverless that referenced this pull request Nov 29, 2023

feat(AWS Lambda): Response streaming for Lambda URL (serverless#11907)

2fce7f8

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

AWS Lambda response streaming for Lambda URLs #11907

AWS Lambda response streaming for Lambda URLs #11907

grakic commented Apr 9, 2023

dested commented Apr 17, 2023

medikoo left a comment

medikoo May 9, 2023

medikoo May 9, 2023

medikoo May 9, 2023

medikoo May 9, 2023

medikoo left a comment

medikoo May 16, 2023

harounansari-cj commented May 27, 2023 •

edited

dested commented May 27, 2023

harounansari-cj commented May 27, 2023

harounansari-cj commented May 27, 2023 •

edited

medikoo commented May 27, 2023

grakic commented May 27, 2023

medikoo left a comment

codecov bot commented May 29, 2023

jayarjo commented Mar 9, 2024

sainiankit commented Mar 25, 2024

grakic commented Mar 25, 2024

sainiankit commented Mar 25, 2024

tobilg commented Mar 25, 2024

		@@ -0,0 +1,9 @@
		'use strict';

		// eslint-disable-next-line no-undef

AWS Lambda response streaming for Lambda URLs #11907

AWS Lambda response streaming for Lambda URLs #11907

Conversation

grakic commented Apr 9, 2023

dested commented Apr 17, 2023

medikoo left a comment

Choose a reason for hiding this comment

medikoo May 9, 2023

Choose a reason for hiding this comment

medikoo May 9, 2023

Choose a reason for hiding this comment

medikoo May 9, 2023

Choose a reason for hiding this comment

medikoo May 9, 2023

Choose a reason for hiding this comment

medikoo left a comment

Choose a reason for hiding this comment

medikoo May 16, 2023

Choose a reason for hiding this comment

harounansari-cj commented May 27, 2023 • edited

dested commented May 27, 2023

harounansari-cj commented May 27, 2023

harounansari-cj commented May 27, 2023 • edited

medikoo commented May 27, 2023

grakic commented May 27, 2023

medikoo left a comment

Choose a reason for hiding this comment

codecov bot commented May 29, 2023

Codecov Report

jayarjo commented Mar 9, 2024

sainiankit commented Mar 25, 2024

grakic commented Mar 25, 2024

sainiankit commented Mar 25, 2024

tobilg commented Mar 25, 2024

harounansari-cj commented May 27, 2023 •

edited

harounansari-cj commented May 27, 2023 •

edited