New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Reduce time for serverless cold-start of .Net/C# code #1060

Closed
James2397 opened this Issue Nov 2, 2017 · 22 comments

Comments

Projects
None yet
9 participants
@James2397

James2397 commented Nov 2, 2017

Reduce time for serverless cold-start of .Net/C# code

At the moment .Net Core is one of the slowest loading languages you can use for serverless development in Azure Functions of Aws Lambda. Each time you start a new function written in .Net Core there is significant "cold-start" time which means users experience an extended time waiting compared to using other languages. With serverless becoming a core architectural approach for many companies it would make sense to look into this problem now. If the "cold start" of .net core programs could be reduced to low milliseconds it would significantly increase the appeal of the language for serverless development.

The second part of this problem is the artifact that is generated from a dotnet core publish is large compared to other languages which also attributes to slow "cold-start" times. It would be good if the IL Linker could be brought into the normal publishing process to reduce dll sizes (https://github.com/dotnet/core/blob/master/samples/linker-instructions.md). If namespaces or classes are not being used they should be removed from the published dlls. Or combine dlls into a single binary that only contains used classes/methods/properties. Regardless, just make it super small and fast to work the best out of all languages with serverless.

@Petermarcu

This comment has been minimized.

Show comment
Hide comment
@Petermarcu
Member

Petermarcu commented Nov 2, 2017

@Petermarcu

This comment has been minimized.

Show comment
Hide comment
@Petermarcu

Petermarcu Nov 2, 2017

Member

Can you share any number you have for cold start times? There are things that can be done to make cold start better and some of it depends on how AWS has built and configured the runtime to work.

Member

Petermarcu commented Nov 2, 2017

Can you share any number you have for cold start times? There are things that can be done to make cold start better and some of it depends on how AWS has built and configured the runtime to work.

@James2397

This comment has been minimized.

Show comment
Hide comment
@James2397

James2397 Nov 2, 2017

Here is a comparison by a tech blogger and it summarizes what we have been experiencing: http://theburningmonk.com/2017/06/aws-lambda-compare-coldstart-time-with-different-languages-memory-and-code-sizes/

James2397 commented Nov 2, 2017

Here is a comparison by a tech blogger and it summarizes what we have been experiencing: http://theburningmonk.com/2017/06/aws-lambda-compare-coldstart-time-with-different-languages-memory-and-code-sizes/

@James2397

This comment has been minimized.

Show comment
Hide comment
@James2397

James2397 Nov 3, 2017

The below blog article also shows how .Net Core serverless code is significantly slower than all other languages, and the main language it is most similar too - Java - is actually the quickest for cold starts:

https://read.acloud.guru/comparing-aws-lambda-performance-when-using-node-js-java-c-or-python-281bef2c740f
Summary: .Net Core was over 5 times slower than Java in those tests for cold start serverless invocation.

Hopefully, this can be looked at soon because it is bad for our customers if we use .Net Core and this happens regularly. Java being the fastest shows that it should be possible for a language like C# to achieve similar positive results.

James2397 commented Nov 3, 2017

The below blog article also shows how .Net Core serverless code is significantly slower than all other languages, and the main language it is most similar too - Java - is actually the quickest for cold starts:

https://read.acloud.guru/comparing-aws-lambda-performance-when-using-node-js-java-c-or-python-281bef2c740f
Summary: .Net Core was over 5 times slower than Java in those tests for cold start serverless invocation.

Hopefully, this can be looked at soon because it is bad for our customers if we use .Net Core and this happens regularly. Java being the fastest shows that it should be possible for a language like C# to achieve similar positive results.

@Petermarcu

This comment has been minimized.

Show comment
Hide comment
@Petermarcu

Petermarcu Nov 3, 2017

Member

I noticed that article is using .NET Core 1.0. I know there have been significant investments in both startup and and throughput since. Particularly on Linux. I think it would be useful to do the exercise again with the latest bits to see where things stand. Overall I think this is a good scenario to try to tackle and improve.

On the comment for the size of a published application. How large is the published size of your serverless application?

Member

Petermarcu commented Nov 3, 2017

I noticed that article is using .NET Core 1.0. I know there have been significant investments in both startup and and throughput since. Particularly on Linux. I think it would be useful to do the exercise again with the latest bits to see where things stand. Overall I think this is a good scenario to try to tackle and improve.

On the comment for the size of a published application. How large is the published size of your serverless application?

@James2397

This comment has been minimized.

Show comment
Hide comment
@James2397

James2397 Nov 5, 2017

A lot of customers are using .Net Core 1.0 (.Net Stardard 1.6) because that is what AWS supports for most features (like Lambda serverless) where cold-start time is critical. No serverless provider currently has support for .Net 2.0 yet. In Azure you can manually hack the configs to get .Net Core 2.0 working but I don't think it would be operating in an optimized way when it's not fully supported in the UI yet. Nobody can do the .Net 2.0 numbers yet because AWS and Azure does not fully support .Net Core 2.0 yet. What I do know from working on a few non-serverless .Net Core 2.0 projects recently though, is the startup time is better than .Net 1.0 but not significantly better - the cold start time is still an issue.

James2397 commented Nov 5, 2017

A lot of customers are using .Net Core 1.0 (.Net Stardard 1.6) because that is what AWS supports for most features (like Lambda serverless) where cold-start time is critical. No serverless provider currently has support for .Net 2.0 yet. In Azure you can manually hack the configs to get .Net Core 2.0 working but I don't think it would be operating in an optimized way when it's not fully supported in the UI yet. Nobody can do the .Net 2.0 numbers yet because AWS and Azure does not fully support .Net Core 2.0 yet. What I do know from working on a few non-serverless .Net Core 2.0 projects recently though, is the startup time is better than .Net 1.0 but not significantly better - the cold start time is still an issue.

@James2397

This comment has been minimized.

Show comment
Hide comment
@James2397

James2397 Nov 5, 2017

In regards to the size, in Node we can have some scripts that are a few KB to the do the same job in .Net is 10-20mb because of the tree of dependencies that are pulled in Microsoft .Net Core framework and externally. For example, we may only use 1 tiny enum from a dependency dll and it bings in the whole 5mb dll instead of only the code for the 1 enum that is < 1KB. Our function sizes in .Net Core are on average 100's of times the size of the equivalent in Node because of the 'publish' command bringing in whole dlls, and things we don't need or will ever use - and the code never references. Traditional systems where space was not a concern this is fine, however for serverless where cold-start is most important and every large chunk of unrequired code slows down the loading of the function, it would be good to get dll's cut-down and optimized to be smaller and faster, or removed all together if the code never uses classes/methods/properties from that dll (I am particularly talking about dependencies of dependencies).

James2397 commented Nov 5, 2017

In regards to the size, in Node we can have some scripts that are a few KB to the do the same job in .Net is 10-20mb because of the tree of dependencies that are pulled in Microsoft .Net Core framework and externally. For example, we may only use 1 tiny enum from a dependency dll and it bings in the whole 5mb dll instead of only the code for the 1 enum that is < 1KB. Our function sizes in .Net Core are on average 100's of times the size of the equivalent in Node because of the 'publish' command bringing in whole dlls, and things we don't need or will ever use - and the code never references. Traditional systems where space was not a concern this is fine, however for serverless where cold-start is most important and every large chunk of unrequired code slows down the loading of the function, it would be good to get dll's cut-down and optimized to be smaller and faster, or removed all together if the code never uses classes/methods/properties from that dll (I am particularly talking about dependencies of dependencies).

@Petermarcu

This comment has been minimized.

Show comment
Hide comment
@Petermarcu

Petermarcu Nov 5, 2017

Member

On the size thing, we have our linker on the way which will allow the code in your "app" to be trimmed down to only what you need. https://github.com/dotnet/core/blob/master/samples/linker-instructions-advanced.md

We're going to take a deeper look at the serverless startup problem. We know how fast runtime startup is locally. We'll need to determine what other aspects of the serverless scenario are contributing to the coldstart problem.

We definitely want this to be great. Any additional insight is appreciated.

Member

Petermarcu commented Nov 5, 2017

On the size thing, we have our linker on the way which will allow the code in your "app" to be trimmed down to only what you need. https://github.com/dotnet/core/blob/master/samples/linker-instructions-advanced.md

We're going to take a deeper look at the serverless startup problem. We know how fast runtime startup is locally. We'll need to determine what other aspects of the serverless scenario are contributing to the coldstart problem.

We definitely want this to be great. Any additional insight is appreciated.

@Petermarcu

This comment has been minimized.

Show comment
Hide comment
@Petermarcu

Petermarcu Nov 8, 2017

Member

We are having someone reproduce the results so we can dig into more of the details.

Member

Petermarcu commented Nov 8, 2017

We are having someone reproduce the results so we can dig into more of the details.

@James2397 James2397 changed the title from Reduce time for serverless cold-start time of .Net/C# code to Reduce time for serverless cold-start of .Net/C# code Nov 13, 2017

@kannank

This comment has been minimized.

Show comment
Hide comment
@kannank

kannank Jan 17, 2018

Is there any update in the issue. Has there been any noticeable improvements in 2.0

kannank commented Jan 17, 2018

Is there any update in the issue. Has there been any noticeable improvements in 2.0

@dotnetchris

This comment has been minimized.

Show comment
Hide comment
@dotnetchris

dotnetchris Feb 26, 2018

FWIW AWS supports Core 2.0 now in lambdas, added support early 2018.

dotnetchris commented Feb 26, 2018

FWIW AWS supports Core 2.0 now in lambdas, added support early 2018.

@jansabbe

This comment has been minimized.

Show comment
Hide comment
@jansabbe

jansabbe Mar 2, 2018

I ran the benchmarks from theburningmonk again using .net core 2.0 (and adding golang as well). See my results here: https://plot.ly/%7Ejansabbe/6/csharp-java-python-nodejs6-golang/ .Net core 2.0 is faster than java, but still way slower than nodejs/python or Go.

Interestingly, filesize doesn't matter that much. The .net core 2.0 deployment package was around 200K, while the golang deployment package was close to 3MB. Go was still an order of a magnitude faster for simply responding "hello". Not sure if this is a .net issue, seems more like something is weird at AWS.

jansabbe commented Mar 2, 2018

I ran the benchmarks from theburningmonk again using .net core 2.0 (and adding golang as well). See my results here: https://plot.ly/%7Ejansabbe/6/csharp-java-python-nodejs6-golang/ .Net core 2.0 is faster than java, but still way slower than nodejs/python or Go.

Interestingly, filesize doesn't matter that much. The .net core 2.0 deployment package was around 200K, while the golang deployment package was close to 3MB. Go was still an order of a magnitude faster for simply responding "hello". Not sure if this is a .net issue, seems more like something is weird at AWS.

@Petermarcu

This comment has been minimized.

Show comment
Hide comment
@Petermarcu

Petermarcu Mar 8, 2018

Member

Just saw this article from yesterday showing .NET Core 2.0 as the top dog https://read.acloud.guru/comparing-aws-lambda-performance-of-node-js-python-java-c-and-go-29c1163c2581 .

Member

Petermarcu commented Mar 8, 2018

Just saw this article from yesterday showing .NET Core 2.0 as the top dog https://read.acloud.guru/comparing-aws-lambda-performance-of-node-js-python-java-c-and-go-29c1163c2581 .

@asabla

This comment has been minimized.

Show comment
Hide comment
@asabla

asabla Mar 8, 2018

Sadly @Petermarcu they're excluding cold startup time

Similar to the original performance tests, we’ll ignore the initial cold start time — and focus only on the duration metric to compare runtime performance between the different languages

asabla commented Mar 8, 2018

Sadly @Petermarcu they're excluding cold startup time

Similar to the original performance tests, we’ll ignore the initial cold start time — and focus only on the duration metric to compare runtime performance between the different languages

@James2397

This comment has been minimized.

Show comment
Hide comment
@James2397

James2397 Mar 8, 2018

@Petermarcu the article states a few sentences in "we’ll ignore the initial cold start time — and focus only on the duration metric to compare runtime". There was no questions about the run-time performance of .Net, which we know is good - what was, was the lengthy cold-start duration that is affecting a lot of web platforms that want to scale up serverless fast. In serverless, sometimes you will get a request that takes 90ms, then another time because it is going through a cold start it takes multiples of that (plus the time the host times to start the new VM). The other blog posts I listed demonstrated the slower cold-start time of .net core.

James2397 commented Mar 8, 2018

@Petermarcu the article states a few sentences in "we’ll ignore the initial cold start time — and focus only on the duration metric to compare runtime". There was no questions about the run-time performance of .Net, which we know is good - what was, was the lengthy cold-start duration that is affecting a lot of web platforms that want to scale up serverless fast. In serverless, sometimes you will get a request that takes 90ms, then another time because it is going through a cold start it takes multiples of that (plus the time the host times to start the new VM). The other blog posts I listed demonstrated the slower cold-start time of .net core.

@James2397

This comment has been minimized.

Show comment
Hide comment
@James2397

James2397 Mar 8, 2018

@asabla you beat me to it ;)

James2397 commented Mar 8, 2018

@asabla you beat me to it ;)

@Petermarcu

This comment has been minimized.

Show comment
Hide comment
@Petermarcu

Petermarcu Mar 8, 2018

Member

Understood. Thanks for pointing that out. Just to clarify, the "other article" is this one: http://theburningmonk.com/2017/06/aws-lambda-compare-coldstart-time-with-different-languages-memory-and-code-sizes/ right?

In general, I would expect that .NET and Java would be in the same ballpark. We did make improvements to coldstart in .NET Core 2.0 and have more in 2.1 and have more ideas for beyond that. I'd be interested in seeing how things are looking today with .NET Core 2.0 vs Java.

There are strategies that can be employed by the infrastructure whether its Azure or AWS to keep the process warm. You can see this in the efforts and variations the author had to go through to ensure they were getting cold start time.

Member

Petermarcu commented Mar 8, 2018

Understood. Thanks for pointing that out. Just to clarify, the "other article" is this one: http://theburningmonk.com/2017/06/aws-lambda-compare-coldstart-time-with-different-languages-memory-and-code-sizes/ right?

In general, I would expect that .NET and Java would be in the same ballpark. We did make improvements to coldstart in .NET Core 2.0 and have more in 2.1 and have more ideas for beyond that. I'd be interested in seeing how things are looking today with .NET Core 2.0 vs Java.

There are strategies that can be employed by the infrastructure whether its Azure or AWS to keep the process warm. You can see this in the efforts and variations the author had to go through to ensure they were getting cold start time.

@leecow

This comment has been minimized.

Show comment
Hide comment
@leecow

leecow Jun 22, 2018

Member

Lots of performance work went into 2.1 and we're not resting on our laurels. Closing this conversation as it has gone quiet. Definitely open another if needed.

Member

leecow commented Jun 22, 2018

Lots of performance work went into 2.1 and we're not resting on our laurels. Closing this conversation as it has gone quiet. Definitely open another if needed.

@leecow leecow closed this Jun 22, 2018

@James2397

This comment has been minimized.

Show comment
Hide comment
@James2397

James2397 Jun 24, 2018

Could we get a comparison between the different .Net Core versions to see if their is an improvement instead of just closing the issue? This issue is still a concern to a lot of people.

James2397 commented Jun 24, 2018

Could we get a comparison between the different .Net Core versions to see if their is an improvement instead of just closing the issue? This issue is still a concern to a lot of people.

@neilgallagher

This comment has been minimized.

Show comment
Hide comment
@neilgallagher

neilgallagher Aug 24, 2018

Second that. Would love to see a comparison on startup times especially between 2.0 and 2.1. Also are there any articles on what the cold start improvements are in 2.1? Thanks

neilgallagher commented Aug 24, 2018

Second that. Would love to see a comparison on startup times especially between 2.0 and 2.1. Also are there any articles on what the cold start improvements are in 2.1? Thanks

@jansabbe

This comment has been minimized.

Show comment
Hide comment
@jansabbe

jansabbe Aug 26, 2018

I ran the cold-start benchmark from theburningmonk again for 2.0 and 2.1.

There seems to be an improvement in the case of 512mb, but other than that 2.0 and 2.1 are pretty similar for cold starts. Both versions are faster than Java. Still a long road ahead to compete with NodeJS, Go or Python 😉
No idea if it is possible to deploy crossgenned, ReadyToRun images to AWS. Might help for cold-starts.

jansabbe commented Aug 26, 2018

I ran the cold-start benchmark from theburningmonk again for 2.0 and 2.1.

There seems to be an improvement in the case of 512mb, but other than that 2.0 and 2.1 are pretty similar for cold starts. Both versions are faster than Java. Still a long road ahead to compete with NodeJS, Go or Python 😉
No idea if it is possible to deploy crossgenned, ReadyToRun images to AWS. Might help for cold-starts.

@sinapis

This comment has been minimized.

Show comment
Hide comment
@sinapis

sinapis Sep 20, 2018

Please reopen this issue. Cold start times just cost us a customer that went with Node instead, and honestly I completely understand his choice. The difference in cold start is x100 with .Net Core 2.1 vs Node/Python/Go

sinapis commented Sep 20, 2018

Please reopen this issue. Cold start times just cost us a customer that went with Node instead, and honestly I completely understand his choice. The difference in cold start is x100 with .Net Core 2.1 vs Node/Python/Go

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment