Enable ArrayPool buffer pooling in Apple/Android SslStream PALs

[artl93]

I am running an experiment - full analysis to follow. Contact @artl93

Motivation

ProtocolToken already supports pooled (ArrayPool<byte>.Shared) payload buffers via a RentBuffer flag, consumed correctly by ReleasePayload(). The Windows (SChannel) and OpenSSL/Unix (Linux) SslStream PALs already set this flag. The Apple (Secure Transport: macOS/iOS/tvOS/Mac Catalyst) and Android PALs never did — every encrypted handshake/write token on those two PALs was allocated fresh instead of rented, for no apparent functional reason (confirmed via git blame/history: RentBuffer simply wasn't threaded through when those two PALs were added, #87874 per @bartonjs; @wfurt confirmed these PALs "were never in center of attention" and flagged buffer-lifetime/leak risk as the reason to be cautious about rental in general).

Change

Two lines, two files — set the existing RentBuffer = true flag at the two token-allocation call sites in:

• src/libraries/System.Net.Security/src/System/Net/Security/SslStreamPal.OSX.cs
• src/libraries/System.Net.Security/src/System/Net/Security/SslStreamPal.Android.cs

No new code path is introduced — ProtocolToken.ReleasePayload() already gates pool-return on this per-token flag and is already exercised by the Windows/OpenSSL PALs in production today. This only activates a pre-existing, already-tested pattern for the two PALs that were missing it.

Evidence

Exact-parent A/B (this session — fully-matched Release config, both host/libs/runtime)

• Baseline (exact parent, pre-fix): c1b09d933f7d720f7ae50f80b4ec6980bb93d96f
• Candidate (fix commit): 982805085b2b7225b2fe5ff113e0a6a65d5084e6
• Hardware: Apple M5 Pro (18 cores), macOS 26.5.2 (Darwin 25.5.0) arm64, 48 GiB RAM
• SDK/runtime: 11.0.100-preview.6.26352.110 SDK, 11.0.0 runtime, both sides built with identical ./build.sh clr+libs -rc release -lc release -hc release (this corrects an earlier internal pass that mixed a Debug testhost/host config with Release libraries — matching all three configs removes that variable)
• Method: separate git worktree per commit, isolated .dotnet/artifacts; a self-contained console harness executed via <testhost>/dotnet exec against each side's own testhost (swaps the shared-framework System.Net.Security.dll cleanly — confirmed different SHA-256 hashes, 2afb0fb4... candidate vs 0bfb9c8d... baseline); 3 process launches per side, 5 in-process warm-up iterations + 10 measured trials each = n=30 trials/scenario/side, baseline/candidate runs alternated per round; allocation measured via GC.GetTotalAllocatedBytes(precise: true) (process-wide — GetAllocatedBytesForCurrentThread is unreliable across await continuations that resume on a different thread-pool thread, confirmed by a discarded first attempt yielding nonsensical negative deltas)

Scenario	Base ms (median)	Cand ms (median)	Time Δ	Base B (median)	Cand B (median)	Bytes Δ
Handshake	12.14	11.76	−3.1%	40,736	39,072	−4.1%
MutualHandshake (mTLS)	21.48	21.34	−0.6%	65,792	63,072	−4.1%
Record round-trip 1B	11.46	11.89	+3.7%†	41,024	39,304	−4.2%
Record round-trip 1KiB	11.44	11.78	+3.0%†	43,064	40,320	−6.4%
Record round-trip 16KiB	11.42	11.80	+3.3%†	106,664	88,560	−17.0%
Record round-trip 64KiB	11.55	11.91	+3.1%†	221,720	154,344	−30.4%
Pooled c=1, 4KiB	11.43	11.33	−0.9%	57,656	51,840	−10.1%
Pooled c=16, 4KiB	140.75	142.06	+0.9%†	920,976	827,692	−10.1%
Pooled c=64, 4KiB	543.25	552.35	+1.7%†	3,685,936	3,317,028	−10.0%

† Time deltas of a few percent at these absolute magnitudes (TCP-loopback + RSA/AES/SHA-dominated) are within run-to-run noise for a whole-connection benchmark; this change targets allocations, not raw handshake/record latency. Allocation reduction is the reproducible signal, scales with record size as expected (bigger records ⇒ bigger rented buffers ⇒ proportionally larger savings), and the pooled/concurrent scenarios (representative of a reused, HttpClient-style connection) show a consistent ~10% reduction.

These end-to-end whole-connection percentages are intentionally more conservative than an earlier isolated-component pass (which measured the touched allocation call sites directly and saw −69%/−97%/−98% at 1 KiB/16 KiB/64 KiB, and ~−89% pooled). Both are correct for what they measure: the isolated number answers "how much did the touched code improve," the end-to-end number above answers "how much does a full connection improve" once diluted by everything else a handshake/record round-trip allocates (sockets, the SslStream object graph, cert parsing, TLS state machine, task continuations). Both point the same direction and are mutually consistent.

EgorBot

Requested (-amd -osx_arm64) in a follow-up comment with a self-contained BenchmarkDotNet harness covering the same 9 scenarios (Handshake, MutualHandshake, Record_1B/1KiB/16KiB/64KiB, Pooled_C1/C16/C64_4KiB). Results pending — will be posted to this PR by the bot once its run completes.

Validation

• Functional (System.Net.Security.Tests, candidate HEAD): 4978 total, 0 failed, 20 skipped (macOS: TLS 1.3/renegotiation/null-encryption tests correctly skip — platform doesn't support those, unrelated to this change).
• Unit (System.Net.Security.Unit.Tests): 132 total, 0 failed, 3 skipped.
• Added a differential regression test (SslStreamStreamToStreamTest.SslStream_StreamToStream_RepeatedVariableSizeWrites_RoundTripsCorrectly) exercising many back-to-back writes across a range of payload sizes — including large payloads that force different rented-buffer sizes — verifying data integrity across pooled-buffer reuse, so accidental data bleed between reused buffers, truncation, or double-return corruption would be caught.

Risks / limitations

• Buffer lifetime risk (raised by @wfurt): mitigated by reusing the existing, already-production-tested RentBuffer/ReleasePayload() mechanism rather than introducing new pooling logic — the same code path Windows/OpenSSL PALs rely on today. No new disposal/ownership contract is introduced.
• Platform coverage: Apple-PAL numbers above are measured directly on macOS/Apple Silicon. Android uses the identical code pattern but has not been independently measured on-device in this pass — same-pattern parity with the tested Apple/Windows/OpenSSL PALs is the basis for extending the change there, not independent Android measurement.
• TLS 1.3: this local macOS Secure Transport testhost does not negotiate TLS 1.3 in the harness as written; not investigated further in this pass (tracked as a known gap, not a blocker for this specific buffer-pooling change, which is protocol-version-independent).
• As with any ArrayPool consumer, if a caller ever accessed a ProtocolToken's buffer past its ReleasePayload() call, that would now retrieve pool-recycled memory rather than a discarded array — this pattern is already relied upon by the other two PALs and the test suite exercises repeated encrypt/decrypt cycles, but this is the class of risk @wfurt flagged and worth explicit reviewer attention.

Note

This PR description, evidence, and validation were prepared with AI assistance (GitHub Copilot CLI) under human supervision. Still a work in progress — not yet ready for review (see final PR comment for current status).

[dotnet-policy-service]

Tagging subscribers to this area: @dotnet/ncl, @bartonjs, @vcsjones
See info in area-owners.md if you want to be subscribed.

[Copilot]

Pull request overview

Enables ProtocolToken payload buffer pooling (ArrayPool<byte>) for the Apple (SecureTransport) and Android SslStream PAL implementations, aligning them with the existing Windows and Unix/OpenSSL PAL behavior and reducing steady-state TLS allocation churn.

Changes:

• Set token.RentBuffer = true in Apple and Android PALs for EncryptMessage and HandshakeInternal.
• Add a functional regression test that performs repeated variable-size writes and validates round-trip integrity.

Show a summary per file

File	Description
src/libraries/System.Net.Security/src/System/Net/Security/SslStreamPal.OSX.cs	Enables pooled ProtocolToken payload buffers for encrypted-write and handshake output tokens on Apple PAL.
src/libraries/System.Net.Security/src/System/Net/Security/SslStreamPal.Android.cs	Enables pooled ProtocolToken payload buffers for encrypted-write and handshake output tokens on Android PAL.
src/libraries/System.Net.Security/tests/FunctionalTests/SslStreamStreamToStreamTest.cs	Adds regression coverage for repeated variable-size application-data writes with end-to-end integrity validation.

Copilot's findings

• Files reviewed: 3/3 changed files
• Comments generated: 1

[bartonjs]

Looks like that came in with #87874, so @wfurt would be the one to ask. I assume the answer is that they're just not "as relevant" so they got overlooked, rather than intentionally omitted.

[wfurt]

I think the macOS and Android was never in center of the attention and I feel our test coverage is not that great. Renting comes with higher risk of leaks where the buffers need to be managed explicitly and correctly instead of relying on GC. But in some ways that ship has already sailed. The best thing would be eliminate the intermediate buffers on platforms where we can. Something that come up recently within our team and something I had PoC for long time ago ... but never finished.

[Copilot]

Copilot's findings

• Files reviewed: 3/3 changed files
• Comments generated: 3

[artl93]

Good suggestion, but deferring for now to keep this PR's diff minimal and easy to review/revert (it's already a 2-line, 2-file fix mirroring the existing Windows/OpenSSL pattern). A shared ProtocolToken pooled-init factory would be a good separate follow-up refactor across all 4 PALs, but I'd rather not fold a cross-cutting refactor into a targeted perf fix. Filing this as a mental note for a future PR rather than blocking this one on it.

[Copilot]

Copilot's findings

• Files reviewed: 3/3 changed files
• Comments generated: 1

[artl93]

@EgorBot -amd -osx_arm64

using System.Net;
using System.Net.Security;
using System.Net.Sockets;
using System.Security.Cryptography;
using System.Security.Authentication;
using System.Security.Cryptography.X509Certificates;
using BenchmarkDotNet.Attributes;

namespace SslStreamPoolingBenchmarks;

[MemoryDiagnoser]
public class SslStreamPoolingBench
{
    private X509Certificate2 _serverCert = null!;
    private X509Certificate2 _clientCert = null!;
    private byte[] _payload1B = new byte[1];
    private byte[] _payload1KiB = new byte[1024];
    private byte[] _payload16KiB = new byte[16 * 1024];
    private byte[] _payload64KiB = new byte[64 * 1024];
    private byte[] _payload4KiB = new byte[4 * 1024];

    [GlobalSetup]
    public void Setup()
    {
        _serverCert = CreateSelfSignedCert("CN=sslstream-bench-server");
        _clientCert = CreateSelfSignedCert("CN=sslstream-bench-client");
        Random.Shared.NextBytes(_payload1B);
        Random.Shared.NextBytes(_payload1KiB);
        Random.Shared.NextBytes(_payload16KiB);
        Random.Shared.NextBytes(_payload64KiB);
        Random.Shared.NextBytes(_payload4KiB);
    }

    [GlobalCleanup]
    public void Cleanup()
    {
        _serverCert?.Dispose();
        _clientCert?.Dispose();
    }

    private static X509Certificate2 CreateSelfSignedCert(string subject)
    {
        using RSA rsa = RSA.Create(2048);
        var req = new CertificateRequest(subject, rsa, HashAlgorithmName.SHA256, RSAExtensions());
        req.CertificateExtensions.Add(new X509BasicConstraintsExtension(false, false, 0, false));
        req.CertificateExtensions.Add(new X509KeyUsageExtension(
            X509KeyUsageFlags.DigitalSignature | X509KeyUsageFlags.KeyEncipherment, false));
        X509Certificate2 cert = req.CreateSelfSigned(DateTimeOffset.UtcNow.AddDays(-1), DateTimeOffset.UtcNow.AddDays(30));
        return X509CertificateLoader.LoadPkcs12(cert.Export(X509ContentType.Pfx), (string?)null,
            X509KeyStorageFlags.Exportable);
    }

    private static System.Security.Cryptography.RSASignaturePadding RSAExtensions() =>
        System.Security.Cryptography.RSASignaturePadding.Pkcs1;

    private static (NetworkStream client, NetworkStream server, Socket cs, Socket ss) CreateConnectedSockets()
    {
        using var listener = new Socket(AddressFamily.InterNetwork, SocketType.Stream, ProtocolType.Tcp);
        listener.Bind(new IPEndPoint(IPAddress.Loopback, 0));
        listener.Listen(1);
        var clientSocket = new Socket(AddressFamily.InterNetwork, SocketType.Stream, ProtocolType.Tcp);
        var connectTask = clientSocket.ConnectAsync((IPEndPoint)listener.LocalEndPoint!);
        Socket serverSocket = listener.Accept();
        connectTask.GetAwaiter().GetResult();
        return (new NetworkStream(clientSocket, true), new NetworkStream(serverSocket, true), clientSocket, serverSocket);
    }

    private async Task<(SslStream client, SslStream server)> HandshakeAsync(bool mutualAuth, SslProtocols protocols = SslProtocols.None)
    {
        var (clientNs, serverNs, _, _) = CreateConnectedSockets();
        var client = new SslStream(clientNs, false, (s, c, ch, e) => true);
        var server = new SslStream(serverNs, false, mutualAuth ? (s, c, ch, e) => true : null);

        var serverOptions = new SslServerAuthenticationOptions
        {
            ServerCertificate = _serverCert,
            ClientCertificateRequired = mutualAuth,
            EnabledSslProtocols = protocols,
        };
        var clientOptions = new SslClientAuthenticationOptions
        {
            TargetHost = "sslstream-bench-server",
            EnabledSslProtocols = protocols,
            ClientCertificates = mutualAuth ? new X509CertificateCollection { _clientCert } : null,
        };

        Task serverTask = server.AuthenticateAsServerAsync(serverOptions, CancellationToken.None);
        Task clientTask = client.AuthenticateAsClientAsync(clientOptions, CancellationToken.None);
        await Task.WhenAll(serverTask, clientTask);
        return (client, server);
    }

    [Benchmark]
    public async Task Handshake()
    {
        var (client, server) = await HandshakeAsync(mutualAuth: false);
        client.Dispose();
        server.Dispose();
    }

    [Benchmark]
    public async Task MutualHandshake()
    {
        var (client, server) = await HandshakeAsync(mutualAuth: true);
        client.Dispose();
        server.Dispose();
    }

    private async Task RecordRoundTripAsync(byte[] payload)
    {
        var (client, server) = await HandshakeAsync(mutualAuth: false);
        try
        {
            byte[] recvBuf = new byte[payload.Length];
            Task writeTask = client.WriteAsync(payload, 0, payload.Length);
            int total = 0;
            while (total < payload.Length)
            {
                total += await server.ReadAsync(recvBuf, total, payload.Length - total);
            }
            await writeTask;
        }
        finally
        {
            client.Dispose();
            server.Dispose();
        }
    }

    [Benchmark]
    public Task Record_1B() => RecordRoundTripAsync(_payload1B);

    [Benchmark]
    public Task Record_1KiB() => RecordRoundTripAsync(_payload1KiB);

    [Benchmark]
    public Task Record_16KiB() => RecordRoundTripAsync(_payload16KiB);

    [Benchmark]
    public Task Record_64KiB() => RecordRoundTripAsync(_payload64KiB);

    private async Task PooledConnectionAsync(int concurrency)
    {
        var tasks = new Task[concurrency];
        for (int i = 0; i < concurrency; i++)
        {
            tasks[i] = RecordRoundTripAsync(_payload4KiB);
        }
        await Task.WhenAll(tasks);
    }

    [Benchmark]
    public Task Pooled_C1_4KiB() => PooledConnectionAsync(1);

    [Benchmark]
    public Task Pooled_C16_4KiB() => PooledConnectionAsync(16);

    [Benchmark]
    public Task Pooled_C64_4KiB() => PooledConnectionAsync(64);
}

[artl93]

Status: still a work in progress — not ready for review.

Since the last status update:

• Rewrote the PR title/description with consolidated, corrected evidence (exact-parent A/B with fully-matched Release build config across host/libs/runtime).
• Requested @EgorBot -amd -osx_arm64 above — results pending.
• Addressed Copilot reviewer feedback: Assert.Equal for byte-array comparisons, added a dedicated handshake-token-pooling regression test, rescoped test comments for accuracy. Deferred (not blocking) a suggested shared-token-init-helper refactor across PALs — noted as a good future follow-up, out of scope for this targeted fix.
• Local functional (4983 total / 0 failed / 20 skipped) and unit (132 total / 0 failed / 3 skipped) suites pass on the latest commit.

Remaining before this could be considered for review:

• EgorBot corroboration on independent hardware (amd64 + osx_arm64) — requested, awaiting results.
• Full CI matrix completion on the latest push (in progress at time of writing, no failures observed yet).
• Android PAL numbers are still only inferred from code-pattern parity with the tested Apple/Windows/OpenSSL PALs, not independently measured on-device.

Note

This comment was prepared with AI assistance (GitHub Copilot CLI) under human supervision.

[Copilot]

Copilot's findings

• Files reviewed: 3/3 changed files
• Comments generated: 5

[artl93]

@wfurt - what are your thoughts on the merits and / or timing? (I'm still working on the PR as a background task.)

[rzikm]

@wfurt - what are your thoughts on the merits and / or timing? (I'm still working on the PR as a background task.)

area co-maintainer here, I would be okay with merging this even this late in the cycle. The risk is low since the other PAL implementation already rent the buffer (so any bugs in the shared logic would have already surfaced).

[rzikm]

I don't think either of the new tests adds any value. There are enough existing tests (thousands) that if the buffer reuse logic was flawed, it would surface as nondeterministic failures in random tests across whole test suite runs.

[wfurt]

making platforms differences smaller is generally ok -> this is not big new rewrite, I see it more as consolidation.