Spring Lambda load time issue

[gmaruti]

Apologies if this is wrong place for this query.

I am building Spring Lambda function with and with out Spring Boot. I observe Lambda function takes considerable load time and at times timing out (taking more than 15secs).

Any suggestions on improving the startup times?

[sapessi]

Hi, do you have VPC configured in your Lambda function? Connecting to a VPC can introduce additional latency when the Lambda function first starts.

Having said that, Spring is definitely the slowest framework to start. I have a few ideas on how to speed things up but they will require a lot of testing and likely won't be included in the 0.5 release.

Try to increase the amount of memory allocated to the Lambda function. Giving more memory to the function also allocates from CPU, which should speed things up during startup.

[mattcorey]

Can you summarize your ideas succinctly enough for someone in the community to take a shot at the updates?

[sapessi]

@mattcorey Haven't fully thought this through, but I can see three options that are not necessarily mutually exclusive:

1. Ask the Spring team for input on how to configure the application context to skip some of the initialization and introspection steps
2. Instead of relying on dependency injection, write some AbstractProcessor classes that people can run at compile time that generate bytecode to hardwire some of the annotation's dependencies
3. Write a parallel class loader and pass it to Spring to speed up introspection (you would break the Java specs with this)

[joeyvmason]

+1

[thepont]

I don't know if this will be helpful, but I managed to reduce the cold start time of my application from 50 seconds to 10 seconds by changing a few things around.

The biggest success for me was setting lazyInit in @ComponentScan to true, since not all my requests require access to the same beans, I also reduced the basePackages of the component scan a little ( this mostly reduced my cold start time because I was using a spring boot application to run the service locally and spring boots @EnableAutoConfiguration was being used and increasing the startup time significantly).

I also attempted manually adding all the beans to the JavaConfig as opposed to using the ComponentScan annotation ( Still lazy loading ) but never managed to get this working correctly (kept getting 406 errors when attempting to serialize the response), this also didn't seem to reduce the cold boot time significantly ~9 seconds

I also got significant cold boot speed ups by increasing the memory of the lambda function up to the maxium.

I wonder if the @ComponentScan in the example should use lazyInit ?

[sapessi]

Thanks @thepont, super helpful tips. I will definitely look at updating our Spring sample and try some of your suggestions. I plan to move the documentation for the repo to the wiki, I will create a tips page for each implementation and add your content there!

[nithril]

You may look too at 1.10.9. Generating an index of candidate components

While classpath scanning is very fast, it is possible to improve the startup performance of large applications by creating a static list of candidates at compilation time.

[sapessi]

Thanks @nithril - I will link to this in the docs.

[kyeotic]

This isssue should have stopped the release of the official blog post. I'm shocked this thing was released; you are going to send a wave of Java engineers off of a cliff.

[sapessi]

Hey @tyrsius Unfortunately the spring load time issue was highlighted in the blog post. There's only so much we can do to mitigate this in the serverless-java-container-library.

We have customers running Spring in Lambda now - while the cold start time is high, it only affects the 99th/99.9th percentile of requests (1 out of 1000) with steady state production traffic. For some applications, that threshold is acceptable.

Do you think we should add a warning to the Spring documentation here?

[kyeotic]

It was highlighted and then swept away as "not a real issue". Many production system don't have steady request rates. In fact that is one of lambda's big selling points: you don't pay when its not in use and scaling is automatic. If you have a steady request rate you don't need scaling, and your always using it. Cost analysis has shown that these cases are bad fits for lambda. So the time when this is not a problem is also the time when you should put it on a real server and not use lambda.

I think by encouraging people to bloat up a Java lambda with both an HTTP server and your wrapper you have encouraged a very bad practice, one you will have a hard time reversing.

[sapessi]

@tyrsius This library is not an HTTP server, it does not start one. Simply goes straight into the framework. This point aside, I will update the documentation and perhaps the archetype to use the indexer component.

The point of this library is to help developers work with tools they are already familiar with. If they judge Spring to be too slow at cold start for their use-case, it will be very easy for them to run the exact same code on EC2/ECS or switch to use Jersey/Spark if they want to stay within Lambda. I trust them to be able to make this decision themselves.

[sapessi]

I'll use this issue to track adding the indexer dependency in the Spring and Spring Boot archetypes as well as samples.

[kyeotic]

You're right, its not a full HTTP server, just an HTTP server framework.

[joeyvmason]

Is there an open issue for adding support for the spring-context-indexer? This would be improve performance greatly.

[sapessi]

@joeyvmason I'm using this issue to track the spring-context-indexer. My thinking now is that it won't be enforced b the framework, I'll just add it to the samples and the archetype. I've already updated the Spring documentation to recommend use of the indexer.

[marinr]

Java lambdas generally suffer high latency at cold start. So you may consider keeping it in warm state. That can be done by exposing /ping service just keep you lambda warm. That will reduce load time drastically.

[sapessi]

This is now addressed in #131 and #142. Closing this one.