Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Node.js 20.x. Longer cold starts than prior versions (18,16) with 3X top values compared to the average #126

Closed
adriantanasa opened this issue Nov 21, 2023 · 2 comments

Comments

@adriantanasa
Copy link

adriantanasa commented Nov 21, 2023

The recently released nodejs20.x runtime for AWS Lambda is establishing a concerning performance degradation trend for Node runtime and larger variation in the cold start values with maximum values 3X more than previous node version.

Based on running a benchmark over 100 points for a function (1KB code size, 256MB memory, us-east-1) with nodejs20.x runtime the average value is 240ms (compared to 171ms node18.x and 159ms for node16.x).

There is an increased variation in the initDuration for this runtime compared to the previous version, with the maximum values increased by several times. For the same 1KB-sized function, its maximum cold start value is now 793ms compared to 248ms (node18), and the P90 is 584ms compared to 182ms on node18.

This "instability" is confirmed for functions with variable package sizes (1K to 64MB) over 1000 cold start executions.

Screenshot 2023-11-21 at 12 52 38

Gist results for all Nodejs runtimes:

Details of how the benchmark is designed and executed here: Size is (almost) all that matters for optimizing AWS Lambda cold starts

This issue was raised also by @o-alexandrov on the Nodejs20.x release topic:

@jtuliani I was very hyped about Node.js 20 after your comment above on what AWS did to decrease cold starts.
Unfortunately, the cold starts are a lot worse than Node.js 18/16/14 for non-Graviton (non-ARM) and pretty much the same as Node.js 16 for Graviton.
Originally posted by @o-alexandrov in #91 (comment)

Hopefully, this is something that can be tuned in the next while to fulfill the high community expectations from the announcement of cold starts optimization actions such as the removal of extra root CA certs, lazy-loading of non-essential modules, and further tuning of the AWS SDK 3 library.

runtime-nodejs20

@adriantanasa adriantanasa changed the title Node.js 20.x. Longer cold starts than prior versions (18x,16x) with 3X top values compared to the average Node.js 20.x. Longer cold starts than prior versions (18,16) with 3X top values compared to the average Nov 21, 2023
@electricsheep
Copy link

I have experienced the same issue.

Here is a graph of my lambda duration - red line indicates the upgrade from nodejs18.x to nodejs20.x:

image

Runtime average for v18: 195ms
Runtime average for v20: 310ms

@adriantanasa
Copy link
Author

I have run the benchmark recently and it seems the peak Init Duration issues are fixed now. I am curios what caused them though. Also the overall performance is improved almost matching the nodejs16.x runtime, You can see here the latest results for nodejs20.x. Great job, AWS team for addressing this ;)

screencapture-us-east-1-console-aws-amazon-cloudwatch-home-2024-03-14-16_00_26

PS @electricsheep please create another issue for your case. It might be something from the libraries used or other factor. This issue here was only for Init Duration with fairly plain functions with no npm libraries or AWS SDK calls.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

No branches or pull requests

2 participants