Low(er) Allocations ASP.NET Core #3

Open
davidfowl opened this Issue Oct 19, 2016 · 0 comments

Projects

None yet

1 participant

@davidfowl
Member

Themes

  • Share more memory (try to avoid framework code using their own pool when data is already pooled)
  • Reduce the number of intermediate translations from byte[] -> char[] -> resulting object
  • String parsing should avoid creating as much garbage as possible
  • UTF8 is the most common encoding in our scenarios, so consider fast paths for performance critical code.

Areas we can improve today

  • StaticFiles - Today we use our own implementation of CopyToAsync (https://github.com/aspnet/HttpAbstractions/blob/dev/src/Microsoft.AspNetCore.Http.Extensions/StreamCopyOperation.cs) which uses pooled buffers. We should measure it against this
    dotnet/corefx#11569 as we're still paying the per allocation cost for reads to the FileStream. This fixes those.

  • FormReader - Today we allocate a few objects, a pool char[], a StringBuilder, a StreamReader which allocates a byte[] with specified buffer size.
    We should have a fast path for utf8 so we don't need to look at chars at all. See the prototype here https://github.com/aspnet/HttpAbstractions/blob/davidfowl/push-reader/src/Microsoft.AspNetCore.WebUtilities/Utf8FormReader.cs.

  • JsonInputFormatter - Today MVC uses a custom TextReader (https://github.com/aspnet/HttpAbstractions/blob/dev/src/Microsoft.AspNetCore.WebUtilities/HttpRequestStreamReader.cs) that uses the ArrayPool<byte> and ArrayPool<char> which is then wrapped in a JsonTextReader which shares the same pool for parsing
    . Even though the TextReader and JsonTextReader share the same pool, they both allocate their own sets of buffers which are copied multiple times. The current flow looks like this (https://github.com/aspnet/Mvc/blob/dev/src/Microsoft.AspNetCore.Mvc.Formatters.Json/JsonInputFormatter.cs#L99-L105).

    • Allocate memory from Kestrel's pool to read the body make it available via the HttpRequest.Body property
    • Allocate byte[] and char[] in HttpRequestStreamReader
    • Call HttpRequest.Body.ReadAsync into byte[] then decode into char[] (based on the specified encoding)
    • JsonTextReader allocates a char[] from the pool and calls TextReader.ReadAsync pass in the char[] which data
      is copied into from HttpRequestStreamReader char[].

    Instead we the flow would look like this (we're assuming a fast path for UTF8):

    • Use CopyToAsync to on the request body to get incoming bytes from Kestrel.
    • Use a Utf8JsonReader directly over the body to parse incoming bytes as UTF8 creating primitive objects directly as we see them come in from the byte[] payload.
  • JsonOutputFormatter - Today MVC uses a custom TextWriter (https://github.com/aspnet/HttpAbstractions/blob/dev/src/Microsoft.AspNetCore.WebUtilities/HttpResponseStreamWriter.cs) that uses the ArrayPool<byte> and ArrayPool<char> which is then wrapped in a JsonTextWriter which shares the same pool for parsing
    . Even though the TextWriter and JsonTextWriter share the same pool, they both allocate their own sets of buffers which are copied multiple times (same as reads). The outgoing pipeline isn't as allocatey, as the JsonTextWriter tries to write directly to the underlying TextWriter whenever possible. Today it converts a few primitive types to string before writing to the underlying text writer. This means you end up with primitive -> string -> char[] -> byte[] conversion. We can instead encode directly to byte[] from all primitive values. We might need to tap into the work in corefxlab (https://github.com/dotnet/corefxlab/tree/master/src/System.Text.Primitives/System/Text) to make this happen.

  • Microsoft.Net.HttpHeaders - The parsers here need to be as non allocating as possible. These components are used in a bunch of different middleware (StaticFiles/ResponseCaching) and are usually allocated once per request. Unfortunately we need to parse the existing strings for the forseeable future, so in the mean time, we need to do a pass and reduce allocations as much as we can.

  • WebSockets - @anurse has a prototype of WebSockets implemented on top of Channels. We will use this in SignalR (when we ship SignalR) and we'd also expose this API to end users (2.0).

Primitives

Future

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment