Skip to content

HTTPS clone URL

Subversion checkout URL

You can clone with HTTPS or Subversion.

Download ZIP

Loading…

ASP.NET Update tests #272

Merged
merged 5 commits into from

5 participants

@pdonald

I refactored aspnet (582 additions and 269,238 deletions) and implemented Update tests for all frameworks/databases.

pdonald added some commits
@pdonald pdonald Refactored ASP.NET
Removed the ugly Global.asax and switched to IHttpModule
Switched to NuGet: removed dependencies and now downloading them at build time
Fixed views that were not working on mono
Added an index page with a list of all tests
82bcde9
@pdonald pdonald ASP.NET: Update test + Fortunes test for MongoDB 51f36eb
@pdonald pdonald Fixed build errors on Windows Server 3d8d46c
@pdonald pdonald Updated README and benchmark_config a454ab5
@pfalls1

When I run the mono tests, the build process seems to stall. Here's the output:

Project "/home/pfalls/FrameworkBenchmarks/aspnet/src/Benchmarks.sln" (default target(s)):
    Target ValidateSolutionConfiguration:
        Building solution configuration "Release|x64".
    Target Build:
        Project "/home/pfalls/FrameworkBenchmarks/aspnet/src/Benchmarks.AspNet.csproj" (default target(s)):
            Target RestorePackages:
                Executing: mozroots --import && mono --runtime=v4.0.30319 /home/pfalls/FrameworkBenchmarks/aspnet/src/../lib/.nuget/NuGet.exe install "/home/pfalls/FrameworkBenchmarks/aspnet/src/../lib/packages.config" -source ""    -OutputDirectory "/home/pfalls/FrameworkBenchmarks/aspnet/src/../lib"
                Mozilla Roots Importer - version 3.0.10.0
                Download and import trusted root certificates from Mozilla's MXR.
                Copyright 2002, 2003 Motus Technologies. Copyright 2004-2008 Novell. BSD licensed.
                Downloading from 'http://mxr.mozilla.org/seamonkey/source/security/nss/lib/ckfw/builtins/certdata.txt?raw=1'...
                Importing certificates into user store...
                Issuer: C=US, O="RSA Data Security, Inc.", OU=Secure Server Certification Authority
                Serial number: C0-DD-5E-19-98-3C-6F-57-5E-FE-45-4E-7E-66-AD-02
                Valid from 11/9/1994 12:00:00 AM to 1/7/2010 11:59:59 PM
                Thumbprint SHA-1: 44-63-C5-31-D7-CC-C1-00-67-94-61-2B-B6-56-D3-BF-82-57-84-6F
                Thumbprint MD5:   74-7B-82-03-43-F0-00-9E-6B-B3-EC-47-BF-85-A5-93
@pdonald

Sorry about that. I will add mozroots --import to the installer.py file (where it belongs) and test it tomorrow.

@pdonald

Should be fixed now.

@pdonald

I'd like to point out that this pull request disabled the debug mode which people are complaning about.

@bhauer
Owner

Thanks @pdonald. We'll aim to get this merged soon to address the debug mode complaint.

@micahasmith

@pdonald

I have a tentative async version of this project setup at https://github.com/micahasmith/FrameworkBenchmarks. I do not however have a linux box to test on... if you'd like to try it out the work is there. I feel as though it would be remiss to not have an Async version of the asp.net tests up, especially when concurrency will be in play.

I tested on Windows successfully, however i am no mono expert.

If there's anything i can do, let me know.

@pdonald

Did you get noticeable performance gains?

From what I remember, Controllers are already called asynchronously (but I could be mistaken).

In ADO.NET you are using ExecuteReaderAsync but not OpenAsync or ReadAsync. I was already using these async methods but then peaked at the source of MySQL and PostgreSQL providers and it turns out they have not implemented asynchronous methods and are calling the synchronous methods so it's kind of pointless to use them and should even be slower because of overhead.

@micahasmith

From what I remember, Controllers are already called asynchronously (but I could be mistaken).

Interesting. What I read seems to say otherwise, but i've def been wrong before. Regardless, the proof is "in the pudding" and hopefully these tests will give us some "deductive light" on the subject.

From what I know of the .NET world, any time concurrency can push past 50 concurrent calls you need to start using async methods (unless you've modified web/machine config).

Did you get noticeable performance gains?

On the /json call when concurrency was at -c 100 (i used ab) i could get the response down to 100 ms less than the minimum i got the regular, sync one too. I'm assuming this was in response to the aforementioned 50-concurrent-call-rule. But that was on my machine, and in general i was extremely disappointed by the variance in the response times. This is the first time my observation of a .NET site lead me to believe that it may be worthwhile to "warm up" by making a few requests before doing the actual test, JVM-style.

As a side note, I feel strongly that we should also make a aspnet test profile that relies soley on IAsyncHttpHandler-- from experience i've always cranked the most out of my .net web projects by relying on it. Especially given that there is a java servlet test, I think it could be argued.

it turns out they have not implemented asynchronous methods

Hmm. Good to know. Well when they start testing against SQL Server hopefully we'll get some wins there....

@pdonald

I am no expert but if you throw new Exception() in an action method, you can see in the stack trace that it's using IHttpAsyncHandler and action methods are invoked asynchronously. Here you can see that AsyncController is actually empty.

A note about testing from Using Asynchronous Methods in ASP.NET MVC 4:

You'll need a Windows Server operating system to see the benefits of asynchronous methods under high load.

I don't think using Task will make this application magically faster especially since we're not using any asynchronous code. JSON serialization is CPU intensive and the data providers don't implement async methods.

As a side note, I feel strongly that we should also make a aspnet test profile that relies soley on IAsyncHttpHandler-- from experience i've always cranked the most out of my .net web projects by relying on it. Especially given that there is a java servlet test, I think it could be argued.

I think the goal of this project is to benchmark frameworks in somewhat realistic scenarios. I had my own non-scientific benchmarks with an async HttpListener and even TcpClient and I still couldn't beat node.js.

@micahasmith

Here you can see that AsyncController is actually empty.

Wow. Thanks for showing me that!

I don't think using Task will make this application magically faster especially since we're not using any asynchronous code. JSON serialization is CPU intensive and the data providers don't implement async methods.

Understandable. I only initially added it to the /json one in order for it to be consistent across the whole proj w/ ones that i thought would be able to be faster because of it (sql tests i expected to benefit from async).

I think the goal of this project is to benchmark frameworks in somewhat realistic scenarios. I had my own non-scientific benchmarks with an async HttpListener and even TcpClient and I still couldn't beat node.js.

I do run IAsyncHttpHandler only projects in production due to my lack of need for full-out MVC stack. I guess "realistic scenarios" is subjective.

Thanks again for pointing that AsyncController stuff out... mind=blown.

@rafaelsc

I have a suggestion to add in this Pull Request.
I still see a opportunity to optimize the web.config.

More Info:

@pfalls1 pfalls1 merged commit 9dd6dce into from
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Commits on May 11, 2013
  1. @pdonald

    Refactored ASP.NET

    pdonald authored
    Removed the ugly Global.asax and switched to IHttpModule
    Switched to NuGet: removed dependencies and now downloading them at build time
    Fixed views that were not working on mono
    Added an index page with a list of all tests
  2. @pdonald
  3. @pdonald
  4. @pdonald
Commits on May 16, 2013
  1. @pdonald
Something went wrong with that request. Please try again.