Skip to content

Use exponential buffer growth in LoopbackServer test helper#127779

Merged
agocke merged 2 commits intomainfrom
copilot/optimize-networking-test-loopback
May 5, 2026
Merged

Use exponential buffer growth in LoopbackServer test helper#127779
agocke merged 2 commits intomainfrom
copilot/optimize-networking-test-loopback

Conversation

Copy link
Copy Markdown
Contributor

Copilot AI commented May 5, 2026

Description

LoopbackServer.Connection grows its read buffers by a fixed +BufferSize (4000 bytes) on each resize, causing O(n²) copying for large payloads. Changed to *2 doubling in both ReadToEndAsync and ReadLineBytesAsync using Array.Resize.

ReadToEndAsync:

  • Fixed the resize trigger from bytesRead == buffer.Length to offset == buffer.Length (the previous check would stop triggering after the first resize, since subsequent reads only fill the remaining buffer space)
  • Replaced manual new byte[] + CopyTo with Array.Resize(ref buffer, buffer.Length * 2)
  • Removed the redundant totalLength variable; the final GetString now uses offset directly

ReadLineBytesAsync:

  • Replaced manual new byte[] + Array.Copy (into new buffer) with in-place Array.Copy to compact live data, followed by Array.Resize(ref _readBuffer, _readBuffer.Length * 2)
  • Index resets (_readStart, _readEnd, startSearch) are now always applied unconditionally after the compact step

Copilot AI review requested due to automatic review settings May 5, 2026 00:15
Copilot AI review requested due to automatic review settings May 5, 2026 00:16
@agocke agocke marked this pull request as ready for review May 5, 2026 00:16
Copilot AI review requested due to automatic review settings May 5, 2026 00:16
@dotnet-policy-service
Copy link
Copy Markdown
Contributor

Tagging subscribers to this area: @karelz, @dotnet/ncl
See info in area-owners.md if you want to be subscribed.

Copy link
Copy Markdown
Contributor

Copilot AI left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Pull request overview

This PR updates the HTTP/1.1 loopback test helper used across System.Net.Http functional tests to grow two internal read buffers exponentially instead of by fixed 4 KB increments, aiming to reduce repeated copying when handling large payloads.

Changes:

  • Changes Connection.ReadToEndAsync to resize its temporary buffer with * 2 growth.
  • Changes Connection.ReadLineBytesAsync to resize _readBuffer with * 2 growth.
  • Keeps the change scoped to the shared LoopbackServer test infrastructure used by networking tests.
Comments suppressed due to low confidence (1)

src/libraries/Common/tests/System/Net/Http/LoopbackServer.cs:643

  • When dataLength is 0 (for example, the previous line ended exactly at the end of _readBuffer), this branch allocates a larger array but never installs it or resets the indices. The subsequent read then uses a count of 0 and ReadLineBytesAsync treats that as EOF, so the helper can stop parsing even though more request data is still available.
                            byte[] newBuffer = new byte[_readBuffer.Length * 2];
                            int dataLength = _readEnd - _readStart;
                            if (dataLength > 0)
                            {
                                Array.Copy(_readBuffer, _readStart, newBuffer, 0, dataLength);
                                _readStart = 0;
                                _readEnd = dataLength;
                                _readBuffer = newBuffer;
                                startSearch = dataLength;

Comment thread src/libraries/Common/tests/System/Net/Http/LoopbackServer.cs Outdated
…n ReadToEndAsync/ReadLineBytesAsync

Agent-Logs-Url: https://github.com/dotnet/runtime/sessions/64def83f-7ce6-49e3-8227-26dd9cd45de5

Co-authored-by: MihaZupan <25307628+MihaZupan@users.noreply.github.com>
@agocke
Copy link
Copy Markdown
Member

agocke commented May 5, 2026

/ba-g infra failures: Remote machine provider issue: The remote provider was unable to process the request.

@agocke agocke merged commit 9dced0c into main May 5, 2026
85 of 91 checks passed
@github-project-automation github-project-automation Bot moved this to Done in AppModel May 5, 2026
@agocke agocke deleted the copilot/optimize-networking-test-loopback branch May 5, 2026 20:56
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Projects

Status: Done

Development

Successfully merging this pull request may close these issues.

4 participants