New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Nancy: What is Nancy.Hosting.Event2.dll? + Add discussion summary to README #877

Closed
phillip-haydon opened this Issue Jun 22, 2014 · 13 comments

Comments

Projects
None yet
7 participants
@phillip-haydon

phillip-haydon commented Jun 22, 2014

What is this host?

https://github.com/TechEmpower/FrameworkBenchmarks/blob/master/nancy/lib/Nancy.Hosting.Event2.dll

The benchmarks for any .NET stuff are pretty terrible so I'm trying to figure out what this junk is and see if we can get some better benchmarks in.

Is there any source code?

Also why does this exist? https://github.com/TechEmpower/FrameworkBenchmarks/blob/master/nancy/src/Global.asax.cs

@bhauer

This comment has been minimized.

Member

bhauer commented Jun 23, 2014

Your second question appears to be the same as #879, which in turn I believe is related to #325.

However, the Event2.dll is not something I have any knowledge of. Let me see if I can loop in past contributors @kppullin and @MalcolmEvershed for their opinions.

@kppullin

This comment has been minimized.

Contributor

kppullin commented Jun 23, 2014

Event2.dll

I believe this is based on https://github.com/kekekeks/evhttp-sharp . @kekekeks should be able to fill in the details. IIRC either the pull request or the evhttp repo had sample results showing the net effect.


Global.asax thread configuration

I based much of the Nancy and ServiceStack configurations on the then-existing ASP.NET setup, which also set these values in a similar manner. To keep things consistent I carried the idea over and also because they provide better end results (based on previous production tuning at work and testing via the FrameworkBenchmarks suite).

There are two issues with the default values (which also vary by framework version, but a test against 4.5 just showed 1 min thread per core, and 2048 max per core [though the max might be a hard cap regardless of # of cores]).

Min

On the min value side, once the number of threads allocated to the threadpool exceeds the min value, subsequent threads are spun up at the rate of 1 every 500ms, up to the max limit.

Let's consider that the 256 concurrent request benchmark. Based on the default settings it will take 64 seconds to allocate 256 threads on the server side (256 requests / 2 cores * .5 seconds). This is certainly non-optimal for the worse case latency metric (whether we should optimize for this aspect is a separate topic). It affects the other metrics as well, but to a lesser extent as the average settles out over time.

Max

On the max value side, as the number of threads allocated grows, we first start running into diminishing returns and, shortly thereafter, negative returns - particularly for the non-IO dependent benchmarks. So setting a reasonable max value helps prevent run away thread allocations.

Conclusion

After a bit of testing the values in the config provided the "best" overall result. However, this was on a couple medium EC2 nodes. Other settings are likely more optimal for other configurations, which is difficult to determine without having access to such configurations.

Such is the crux of performance tuning : )! Perhaps it makes sense to revert to the defaults due to this setting being 'non-standard' (though I'd argue should be a best practice).

For more details check the links referenced here: https://github.com/TechEmpower/FrameworkBenchmarks/blob/master/aspnet/src/Application.cs#L67


With all that, I'd love to see some more love put in to all of the .NET tests, both on Windows and Linux, and also with the new ASP.NET vNext bits! I had a baby 5 months ago and am just starting to find time to breathe, but not enough yet to help contribute at the moment :)

Let me know if you have further questions and I'll help the best I'm able.

@kekekeks

This comment has been minimized.

Contributor

kekekeks commented Jun 23, 2014

The main problems with .NET serverside running on Mono are:

  • slow slow cross-AppDomain calls
  • crappy implementation of ASP.NET stack. It has several bottlenecks and non-optimal code. There is https://github.com/xplicit/HyperFastCgi that has some workarounds, but its still slow.
  • non-optimal System.Net.Sockets (it follows Windows model that is designed for IOCP with thread pool waiting for events, while for Linux it's better to have several worker threads using epoll and handle events in the same thread that got it from the kernel)

While to trying to get Nancy up and running, I found out that self-hosting of it works way faster than hosting it using mono-xsp. Then I've created a wrapper around libevent2's built-in HTTP-server that can handle twice more requests per second.

ASP.NET vNext will eliminate only the second problem. For Mono they use NOwin, which is still limited by Mono's network subsystem. It can be eliminated by creating Mono.Sockets on top of libuv or something like that (See http://lists.ximian.com/pipermail/mono-devel-list/2014-May/041484.html )

@phillip-haydon

This comment has been minimized.

phillip-haydon commented Jun 23, 2014

Right now none of the Nancy tests run.

The Windows one has such bad results despite testing I did last year which (without async) is twice as fast as MVC/WebAPI, and ServiceStack is faster again. Yet the results from this Framework Benchmark show super poor results.

While the results shown for Mono pale in comparison to tests I did with fastcgi-mono-server4 and nginx. However I was unable to get async working with this method.

@kekekeks

This comment has been minimized.

Contributor

kekekeks commented Jun 23, 2014

AFAIK, Mono ASP.NET stack implementation doesn't support async/await pattern yet.

@phillip-haydon

This comment has been minimized.

phillip-haydon commented Jun 23, 2014

So I'm told, but it does work if you push the requests from nginx to a self host which I think is described on the Nancy wiki.

@kekekeks

This comment has been minimized.

Contributor

kekekeks commented Jun 23, 2014

Self host uses HttpListener and own request management code, so async/await works fine. It also works fine with evhttp-sharp for the same reason. However, its useless for benchmarks on Mono, since neither Mysql.Data.dll and Npgsql.dll support *Async methods. Such functionality is scheduled for Npgsql, but isn't done yet.

@phillip-haydon

This comment has been minimized.

phillip-haydon commented Jun 23, 2014

True, but EF 6 supports async and MySQL. So can async not be done with that?

Also you could just wrap the call yourself for the purpose of the benchmark test, even if it's run as a separate test. I have no Mono benchmarks to know if there is any gain but it would be interesting to see if there is.

@kekekeks

This comment has been minimized.

Contributor

kekekeks commented Jun 23, 2014

DbCommand has default implementations of *Async methods that run synchronously, for example https://github.com/mono/mono/blob/master/mcs/class/System.Data/System.Data.Common/DbCommand.cs#L172

@jchannon

This comment has been minimized.

Contributor

jchannon commented Jun 23, 2014

So thats a mono issue? FYI : Dapper supports async which is what is being used in these tests but until mono make them truely async then i guess theres no gain

@kekekeks

This comment has been minimized.

Contributor

kekekeks commented Jun 23, 2014

It's MySql/Npgsql issue, Mono just provides default implementation.

@jchannon

This comment has been minimized.

Contributor

jchannon commented Jun 23, 2014

Ah I see because MySql/Npgsql need to inherit that abstract class

@hamiltont

This comment has been minimized.

Contributor

hamiltont commented Aug 12, 2014

@phillip-haydon Can we close this?

For everyone else involved here ( @jchannon , @kekekeks @kppullin ) -- I would to see a pull request that moves a summary of this discussion into the README for the affected frameworks so it's easy to find for anyone interested

@hamiltont hamiltont added the question label Aug 12, 2014

@LadyMozzarella LadyMozzarella added this to the Round 11 milestone Mar 24, 2015

@LadyMozzarella LadyMozzarella changed the title from Nancy: What is Nancy.Hosting.Event2.dll to Nancy: What is Nancy.Hosting.Event2.dll? + Add discussion summary to README Mar 24, 2015

@LadyMozzarella LadyMozzarella self-assigned this Mar 25, 2015

LadyMozzarella added a commit to LadyMozzarella/FrameworkBenchmarks that referenced this issue Apr 14, 2015

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment