Excessive allocations in HttpContent.ReadAsByteArrayAsync

[dotMorten]

Description

When using HttpClient.SendAsync and reading the HttpResponse.Content into a byte-array, it results in allocations in my testing 4 times that of the actual http response. If using HttpClient.GetByteArrayAsync allocations much closer to the actual http response is made. However this API call only allows for basic get requests, and no way to look at response headers etc.

I'm trying to find ways to heavily reduce memory allocations while still having full access to the power SendAsync provides over the simplified Get* methods.

Configuration

.NET 7.0.101, Windows 11 Pro, build 22621, x64

Regression?

Same thing observed with .NET 6, allocations almost identical.

Data

Benchmark:

   [MemoryDiagnoser]
    public class HttpTests
    {
        private readonly HttpClient client;
        private const string url = "http://sampleserver6.arcgisonline.com/arcgis/rest/services/Census/MapServer/3/query?where=1%3D1&text=&objectIds=&time=&timeRelation=esriTimeRelationOverlaps&geometry=&geometryType=esriGeometryEnvelope&inSR=&spatialRel=esriSpatialRelIntersects&distance=&units=esriSRUnit_Foot&relationParam=&outFields=*&returnGeometry=true&returnTrueCurves=false&maxAllowableOffset=&geometryPrecision=&outSR=&havingClause=&returnIdsOnly=false&returnCountOnly=false&orderByFields=&groupByFieldsForStatistics=&outStatistics=&returnZ=false&returnM=false&gdbVersion=&historicMoment=&returnDistinctValues=false&resultOffset=&resultRecordCount=&returnExtentOnly=false&sqlFormat=none&datumTransformation=&parameterValues=&rangeValues=&quantizationParameters=&featureEncoding=esriDefault&f=pjson";
        public HttpTests()
        {
            client = new HttpClient();
        }

        [Benchmark] 
        public async Task<byte[]> Client_GetByteArrayAsync()
        {
            // Allocates ~1mb, about size of response
            var response = await client.GetByteArrayAsync(url).ConfigureAwait(false);
            return response;
        }
        [Benchmark]
        public async Task<byte[]> Client_Send_GetByteArrayAsync()
        {
            // Allocates ~4mb, about 4 times the response size
            var request = new HttpRequestMessage(HttpMethod.Get, url);
            var response = await client.SendAsync(request);
            var bytes = await response.Content.ReadAsByteArrayAsync().ConfigureAwait(false);
            return bytes;
        }
   }

Method	Gen0	Gen1	Gen2	Allocated
Client_GetByteArrayAsync	-	-	-	1.02 MB
Client_Send_GetByteArrayAsync	1000.0000	1000.0000	1000.0000	4.87 MB

Now I was able to reduce memory by about 1mb, using this approach, but still quite short of the simple GetByteArrayAsync:

        public async Task<int> ReadAsSpanAsync()
        {
            var request = new HttpRequestMessage(HttpMethod.Get, url);
            var response = await client.SendAsync(request, HttpCompletionOption.ResponseHeadersRead).ConfigureAwait(false);
            using MemoryStream stream = new MemoryStream();
            await response.Content.CopyToAsync(stream);
            ReadOnlyMemory<byte> bytes;
            if (!stream.TryGetBuffer(out ArraySegment<byte> dataSegment))
                throw new Exception("Can't get buffer");
            bytes = dataSegment;
            if(bytes.Span.Length == 0)
                throw new Exception("Buffer empty");
            return bytes.Span.Length;
        }

Method	Gen0	Gen1	Gen2	Allocated
ReadAsSpanAsync	750.0000	750.0000	750.0000	3.87 MB

Analysis

Looking at the HttpContent, lots of byte array copying is performed to ensure no access to the internal memorystream that was buffered to. It might be helpful to create some access to this data with ReadOnlyMemory<byte> or similar.

Tagging subscribers to this area: @dotnet/ncl
See info in area-owners.md if you want to be subscribed.

Issue Details

---

Description

When using HttpClient.SendAsync and reading the HttpResponse.Content into a byte-array, it results in allocations in my testing 4 times that of the actual http response. If using HttpClient.GetByteArrayAsync allocations much closer to the actual http response is made. However this API call only allows for basic get requests, and no way to look at response headers etc.

I'm trying to find ways to heavily reduce memory allocations while still having full access to the power SendAsync provides over the simplified Get* methods.

Configuration

.NET 7.0.101, Windows 11 Pro, build 22621, x64

Regression?

Same thing observed with .NET 6, allocations almost identical.

Data

Benchmark:

   [MemoryDiagnoser]
    public class HttpTests
    {
        private readonly HttpClient client;
        private const string url = "http://sampleserver6.arcgisonline.com/arcgis/rest/services/Census/MapServer/3/query?where=1%3D1&text=&objectIds=&time=&timeRelation=esriTimeRelationOverlaps&geometry=&geometryType=esriGeometryEnvelope&inSR=&spatialRel=esriSpatialRelIntersects&distance=&units=esriSRUnit_Foot&relationParam=&outFields=*&returnGeometry=true&returnTrueCurves=false&maxAllowableOffset=&geometryPrecision=&outSR=&havingClause=&returnIdsOnly=false&returnCountOnly=false&orderByFields=&groupByFieldsForStatistics=&outStatistics=&returnZ=false&returnM=false&gdbVersion=&historicMoment=&returnDistinctValues=false&resultOffset=&resultRecordCount=&returnExtentOnly=false&sqlFormat=none&datumTransformation=&parameterValues=&rangeValues=&quantizationParameters=&featureEncoding=esriDefault&f=pjson";
        public HttpTests()
        {
            client = new HttpClient();
        }

        [Benchmark] 
        public async Task<byte[]> Client_GetByteArrayAsync()
        {
            // Allocates ~1mb, about size of response
            var response = await client.GetByteArrayAsync(url).ConfigureAwait(false);
            return response;
        }
        [Benchmark]
        public async Task<byte[]> Client_Send_GetByteArrayAsync()
        {
            // Allocates ~4mb, about 4 times the response size
            var request = new HttpRequestMessage(HttpMethod.Get, url);
            var response = await client.SendAsync(request);
            var bytes = await response.Content.ReadAsByteArrayAsync().ConfigureAwait(false);
            return bytes;
        }
   }

Method	Gen0	Gen1	Gen2	Allocated
Client_GetByteArrayAsync	-	-	-	1.02 MB
Client_Send_GetByteArrayAsync	1000.0000	1000.0000	1000.0000	4.87 MB

Now I was able to reduce memory by about 1mb, using this approach, but still quite short of the simple GetByteArrayAsync:

        public async Task<int> ReadAsSpanAsync()
        {
            var request = new HttpRequestMessage(HttpMethod.Get, url);
            var response = await client.SendAsync(request, HttpCompletionOption.ResponseHeadersRead).ConfigureAwait(false);
            using MemoryStream stream = new MemoryStream();
            await response.Content.CopyToAsync(stream);
            ReadOnlyMemory<byte> bytes;
            if (!stream.TryGetBuffer(out ArraySegment<byte> dataSegment))
                throw new Exception("Can't get buffer");
            bytes = dataSegment;
            if(bytes.Span.Length == 0)
                throw new Exception("Buffer empty");
            return bytes.Span.Length;
        }

Method	Gen0	Gen1	Gen2	Allocated
ReadAsSpanAsync	750.0000	750.0000	750.0000	3.87 MB

Analysis

Looking at the HttpContent, lots of byte array copying is performed to ensure no access to the internal memorystream that was buffered to. It might be helpful to create some access to this data with ReadOnlyMemory<byte> or similar.

Author:	dotMorten
Assignees:	-
Labels:	area-System.Net.Http, tenet-performance, untriaged
Milestone:	-

[stephentoub]

I'm unclear as to the issue being reported. The title says that HttpContent.GetByteArrayAsync had excessive allocation, but there isn't such a method, and the text of the issue says HttpClient.GetByteArrayAsync had the desired allocation. Is this issue asking how to write code that reads the response stream and achieves the same allocations as HttpClient.GetByteArrayAsync? Or did you intend to ask about HttpContent.ReadAsByteArrayAsync?

[LanceMcCarthy]

@stephentoub this screen explains the issue in a glance https://twitter.com/dotMorten/status/1621672844760592385?t=mpRkFiS_w25HzqUxGsL1Tg&s=19

(Sorry for doing a Kramer entrance, but I'd like to improve my networking stack too 😉)

[dotMorten]

@stephentoub sorry for the typo. Fixed the title.
Yes I want to read the data from HttpContent just as efficient as I can with the shortcut method on HttpClient.

[davidfowl]

If you want this code to be efficient, don't allocate the byte[] and instead use the Stream (I know it's unrelated to the issue but at the end of the day the above code can't really be efficient).

[dotMorten]

I do want to allocate the byte array for the contents (it’ll be sent to native code in my case as is) but I don’t want to allocate it 4 times over.
I was trying to optimize a little and move to more Span/Memory use and was shocked to see a 1mb download becomes a 4mb allocation.

As shown above I was able to use stream and reduce a little bit but it is still 3 times as inefficient

[stephentoub]

The difference comes from a few places:

• HttpContent for whatever reason was designed to buffer its contents and be readable multiple times, e.g. you can call ReadAsByteArrayAsync multiple times on the same content and it'll succeed. That means it doesn't hand out the same array each time, because the consumer could change the contents and thereby affect the behavior of future calls, which means HttpContent buffers everything once and returns a copy. If you only call it once, though, you end up with twice the allocation needed. That explains 1x of the overhead.
• The response is chunked rather than having a content length, so the client has no idea how big the response will be. As such, via MemoryStream, it ends up using a standard doubling algorithm on the buffer size as data arrives. The way the data ends up arriving in chunks, it ends up doubling ~10 times. The way the chunks are sent by the server relative to the total amount sent, the server ends up sending about 1MB but in a way that causes the buffer to have doubled to ~2MB (enough data was sent in just the right (wrong) way that it forced another double but didn't use most of it). That difference is another 1x of the overhead. But we allocated doubling each time to get there, so along the way it also allocated another ~2MB in temporary arrays... that explains another ~2x of the overhead.

In contrast, GetByteArrayAsync doesn't have to buffer in a way that's reusable, so it buffers into ArrayPool memory, creates a byte[] from that, and returns the temporary arrays to the pool. ReadAsByteArrayAsync could also use temporary ArrayPool buffers, but it needs to store the resulting temporary for any future calls (because of the "allow multiple future reads" design), which either it would need to hold onto the last ArrayPool array it got indefinitely and hope that the developer disposes of the response message appropriately (note your repro doesn't), or it would need to copy the final ArrayPool array into yet another temporary; this would enable pooling the temporaries along the way, but would still require copying the final one.

We could explore doing using ArrayPool in that manner. We could also explore using a stream that uses a linked list of chunks rather than keeping the data contiguous. But frankly if you care about allocation, I'm with David here: consume the Stream yourself. At that point you can avoid allocating any temporary buffers for the response data.

[dotMorten]

@stephentoub thanks for the detailed answer. I’m more than happy to use stream but I still don’t see how I can use SendAsync and consume the HttpContent’s stream and still get the same low allocation that GetByteArrayAsync does. As shown in my 3rd benchmark I can reduce it by 30% with stream still far from the 75% reduction that might be possible (and an array pool added that might become 50%). From what I can tell the buffered stream will always be there, even when returning as soon as headers have been read.

(Good calll on the dispose - my real code does that or when using in .net framework it ends up exhausting all connections if hitting the same server a lot in parallel)

[stephentoub]

From what I can tell the buffered stream will always be there, even when returning as soon as headers have been read.

It won't. If you use ResponseHeadersRead and get the stream with ReadAsStream, you can read from that and there won't be any additional buffering by the HttpContent. GetBytesArrayAsync isn't doing anything magical; it's just copying from that same response stream into a stream backed by array pool arrays.

[dotMorten]

Thanks again! I lifted out LimitArrayPoolWriteStream and was able to get the performance I wanted. In fact I actually see a way now with the rented pool to make it never create extra copies and make it almost allocation free, since I only need the array for a brief moment before returning it to the pool, so just need to expose the rented array directly.

I think we can close this, but I gotta admit that it might be nice with some new overload/method to get the contents out with low allocation without all those hoops.

[ManickaP]

I think this has been answered and resolved, so I'm closing this.