Improve SDK cold-start latency

[millems]

With the rise of serverless frameworks, library's cold-start latency has become increasingly important to minimize. Libraries that rely heavily on reflection, class-path scanning, needlessly-large thread pools, etc. greatly increase the time it takes to get a lambda JVM to a state that it can process requests.

Cold-start latency of the SDK should be benchmarked and expensive libraries (I'm looking at you, Jackson databind) should be moved out of the critical startup path.

[cherrydev]

Yep! I put together a very basic Lambda that handles an S3 Event, performs a couple operations on S3 objects and then puts some messages on an SQS queue. Using the default Lambda timeouts of 6s I get timeouts. It turns out first-time invocation actually takes about 7.5s. Doing some experimentation with different data serializers leads me to attribute most of this to Jackson.
The data serializer library that had the fastest startup times was square/moshi, which had startup times that were about 10% of Jackson's when it was using the reflection serializers and 7% when using compile-time generation.
I realize that changing data serializer libraries at this point is maybe not feasible.
For reference, a cold invocation of the same lambda written in node.js was 1100ms.

[millems]

We've found big improvements with Jackson-jr as well. Once we can switch over to it for region metadata loading, we'll be able to get using Jackson off of our startup path.

We'll still be using it for unmarshalling JSON exceptions, but removing that is also on our pre-GA plan.

[lwis]

@millems do you have any progress share regarding this? I'm amongst many users that are currently suffering unacceptable cold start times for serverless.

[spfink]

@lwis, have you tested with the latest version? Configured correctly, it should show improvement over v1.

To minimize cold start you should:

• Use the UrlConnectionHttpClient
• Specify a region
• Specify the environment variables credential provider

We are still working to remove Jackson from two more places which should give better cold start times once done.

[lwis]

@spfink thanks for the response!

Something like;

SnsClient.builder()
            .region(Region.EU_WEST_2)
            .httpClient(UrlConnectionHttpClient.builder().build())
            .credentialsProvider(EnvironmentVariableCredentialsProvider.create())
            .build()

I'm unfortunately still seeing 3.5s request to response 😞

Edit: I'm aware that UrlConnection eats up to a second of that time, and network maybe accounts for a small chunk.

[spfink]

What size of a Lambda are you getting 3.5 seconds on?

Removing the last few Jackson use cases should help us cut another second or so off.

[lwis]

@spfink 1536MB. I can put together a little test project if that's helpful?

[dagnir]

related:

• Loading endpoints.json takes a while #748
• AwsDefaultClientBuilder.mergeChildDefaults() instantiates DefaultCredentialsProvider which is expensive #749

[Sandmania]

This is very promising. We are seeing over 50% improvement compared to the old lambda client when initializing the new like so:

LambdaClient.builder()
                .region(Region.EU_CENTRAL_1)
                .httpClient(UrlConnectionHttpClient.builder().build())
                .build();

with sdk v2 and a lambda with 1024MB of memory that takes around 1000ms. Instantiating the old one with AWSLambdaClientBuilder.standard().withRegion("eu-central-1").build() and the same amount of memory takes about 2300ms.

[millems]

Glad that you're seeing some of the benefits already! We still have a few more changes planned, so hopefully we'll be able to get it even lower.

[spfink]

@Sandmania which version of the SDK are you using and seeing those results on?

[Sandmania]

@spfink this is with 2.0.0-preview-12 vs 1.11.443. I'm guessing most of the improvement comes from using UrlConnectionHttpClient, as just instantiating the AmazonHttpClient in 1.11.443 (which I think relies on ApacheHttpClient?) takes 1700ms.

[davidmoten]

see #2527 for details of cold-start improvements with a lightweight java client.