ASP.NET Update tests

[pdonald]

I refactored aspnet (582 additions and 269,238 deletions) and implemented Update tests for all frameworks/databases.

[pfalls1]

When I run the mono tests, the build process seems to stall. Here's the output:

Project "/home/pfalls/FrameworkBenchmarks/aspnet/src/Benchmarks.sln" (default target(s)):
    Target ValidateSolutionConfiguration:
        Building solution configuration "Release|x64".
    Target Build:
        Project "/home/pfalls/FrameworkBenchmarks/aspnet/src/Benchmarks.AspNet.csproj" (default target(s)):
            Target RestorePackages:
                Executing: mozroots --import && mono --runtime=v4.0.30319 /home/pfalls/FrameworkBenchmarks/aspnet/src/../lib/.nuget/NuGet.exe install "/home/pfalls/FrameworkBenchmarks/aspnet/src/../lib/packages.config" -source ""    -OutputDirectory "/home/pfalls/FrameworkBenchmarks/aspnet/src/../lib"
                Mozilla Roots Importer - version 3.0.10.0
                Download and import trusted root certificates from Mozilla's MXR.
                Copyright 2002, 2003 Motus Technologies. Copyright 2004-2008 Novell. BSD licensed.
                Downloading from 'http://mxr.mozilla.org/seamonkey/source/security/nss/lib/ckfw/builtins/certdata.txt?raw=1'...
                Importing certificates into user store...
                Issuer: C=US, O="RSA Data Security, Inc.", OU=Secure Server Certification Authority
                Serial number: C0-DD-5E-19-98-3C-6F-57-5E-FE-45-4E-7E-66-AD-02
                Valid from 11/9/1994 12:00:00 AM to 1/7/2010 11:59:59 PM
                Thumbprint SHA-1: 44-63-C5-31-D7-CC-C1-00-67-94-61-2B-B6-56-D3-BF-82-57-84-6F
                Thumbprint MD5:   74-7B-82-03-43-F0-00-9E-6B-B3-EC-47-BF-85-A5-93

[pdonald]

Sorry about that. I will add mozroots --import to the installer.py file (where it belongs) and test it tomorrow.

[pdonald]

Should be fixed now.

[pdonald]

I'd like to point out that this pull request disabled the debug mode which people are complaning about.

[bhauer]

Thanks @pdonald. We'll aim to get this merged soon to address the debug mode complaint.

[micahasmith]

@pdonald

I have a tentative async version of this project setup at https://github.com/micahasmith/FrameworkBenchmarks. I do not however have a linux box to test on... if you'd like to try it out the work is there. I feel as though it would be remiss to not have an Async version of the asp.net tests up, especially when concurrency will be in play.

I tested on Windows successfully, however i am no mono expert.

If there's anything i can do, let me know.

[pdonald]

Did you get noticeable performance gains?

From what I remember, Controllers are already called asynchronously (but I could be mistaken).

In ADO.NET you are using ExecuteReaderAsync but not OpenAsync or ReadAsync. I was already using these async methods but then peaked at the source of MySQL and PostgreSQL providers and it turns out they have not implemented asynchronous methods and are calling the synchronous methods so it's kind of pointless to use them and should even be slower because of overhead.

[micahasmith]

From what I remember, Controllers are already called asynchronously (but I could be mistaken).

Interesting. What I read seems to say otherwise, but i've def been wrong before. Regardless, the proof is "in the pudding" and hopefully these tests will give us some "deductive light" on the subject.

From what I know of the .NET world, any time concurrency can push past 50 concurrent calls you need to start using async methods (unless you've modified web/machine config).

Did you get noticeable performance gains?

On the /json call when concurrency was at -c 100 (i used ab) i could get the response down to 100 ms less than the minimum i got the regular, sync one too. I'm assuming this was in response to the aforementioned 50-concurrent-call-rule. But that was on my machine, and in general i was extremely disappointed by the variance in the response times. This is the first time my observation of a .NET site lead me to believe that it may be worthwhile to "warm up" by making a few requests before doing the actual test, JVM-style.

As a side note, I feel strongly that we should also make a aspnet test profile that relies soley on IAsyncHttpHandler-- from experience i've always cranked the most out of my .net web projects by relying on it. Especially given that there is a java servlet test, I think it could be argued.

it turns out they have not implemented asynchronous methods

Hmm. Good to know. Well when they start testing against SQL Server hopefully we'll get some wins there....

[pdonald]

I am no expert but if you throw new Exception() in an action method, you can see in the stack trace that it's using IHttpAsyncHandler and action methods are invoked asynchronously. Here you can see that AsyncController is actually empty.

A note about testing from Using Asynchronous Methods in ASP.NET MVC 4:

You'll need a Windows Server operating system to see the benefits of asynchronous methods under high load.

I don't think using Task will make this application magically faster especially since we're not using any asynchronous code. JSON serialization is CPU intensive and the data providers don't implement async methods.

As a side note, I feel strongly that we should also make a aspnet test profile that relies soley on IAsyncHttpHandler-- from experience i've always cranked the most out of my .net web projects by relying on it. Especially given that there is a java servlet test, I think it could be argued.

I think the goal of this project is to benchmark frameworks in somewhat realistic scenarios. I had my own non-scientific benchmarks with an async HttpListener and even TcpClient and I still couldn't beat node.js.

[micahasmith]

Here you can see that AsyncController is actually empty.

Wow. Thanks for showing me that!

I don't think using Task will make this application magically faster especially since we're not using any asynchronous code. JSON serialization is CPU intensive and the data providers don't implement async methods.

Understandable. I only initially added it to the /json one in order for it to be consistent across the whole proj w/ ones that i thought would be able to be faster because of it (sql tests i expected to benefit from async).

I think the goal of this project is to benchmark frameworks in somewhat realistic scenarios. I had my own non-scientific benchmarks with an async HttpListener and even TcpClient and I still couldn't beat node.js.

I do run IAsyncHttpHandler only projects in production due to my lack of need for full-out MVC stack. I guess "realistic scenarios" is subjective.

Thanks again for pointing that AsyncController stuff out... mind=blown.

[rafaelsc]

I have a suggestion to add in this Pull Request.
I still see a opportunity to optimize the web.config.

• Remove unnecessary httpHandlers (Like WebResource.axd, ScriptResource.axd)
• Remove unnecessary httpModules (Like *Authentication, Profile)
• Remove unnecessary assemblies (Like System.WorkflowServices)
• Disable the healthMonitoring ( http://www.asp.net/web-forms/tutorials/deployment/deploying-web-site-projects/logging-error-details-with-asp-net-health-monitoring-cs )
• Add httpRuntime in Web.config ( http://msdn.microsoft.com/en-us/library/e1f13641(v=vs.100).aspx )
• Try change the theadpool size
  • applicationPool ( http://msdn.microsoft.com/en-us/library/dd560842.aspx )
  • processModel.Threads ( http://msdn.microsoft.com/en-us/library/7w2sway1(v=vs.100).aspx )

More Info:

• http://support.microsoft.com/default.aspx?scid=kb;en-us;821268