avoid allocations for TLS handshake

[wfurt]

There are several places where we use Span.ToArray() to get data returned by native layer to something we can write to underlying stream. One day, we may write Memory<byte> wrapper but for now I replaced them with renting buffer from buffer pool and Span.CopyTo.

Since the PAL already deals with renting, I also updated Encrypt method to simplify it. It is already renting buffer but it will guess upfront and than we would pass it in via ref and then get actually length via extra variable. I changed that to simple out where all PAL flavors would rent just what is needed and and we would always return it up in generic SslStream code.

We already return SecurityStatusPal in most cases so we would now return ProtoclToken (that also includes SecurityStatusPal). The is needed as we cannot longer depend on size of the array as renting may give us more than asked for.
The structure is little bit bigger but I don't think it would would be significant. We could pass is via out/ref parameter if this is concern.

contributes to #68951

[ghost]

Tagging subscribers to this area: @dotnet/ncl, @bartonjs, @vcsjones
See info in area-owners.md if you want to be subscribed.

Issue Details

---

There are several places where we use Span.ToArray() to get data returned by native layer to something we can write to underlying stream. One day, we may write Memory<byte> wrapper but for now I replaced them with renting buffer from buffer pool and Span.CopyTo.

Since the PAL already deals with renting, I also updated Encrypt method to simplify it. It is already renting buffer but it will guess upfront and than we would pass it in via ref and then get actually length via extra variable. I changed that to simple out where all PAL flavors would rent just what is needed and and we would always return it up in generic SslStream code.

We already return SecurityStatusPal in most cases so we would now return ProtoclToken (that also includes SecurityStatusPal). The is needed as we cannot longer depend on size of the array as renting may give us more than asked for.
The structure is little bit bigger but I don't think it would would be significant. We could pass is via out/ref parameter if this is concern.

contributes to #68951

Author:	wfurt
Assignees:	wfurt
Labels:	area-System.Net.Security
Milestone:	8.0.0

[rzikm]

looks promising so far, I am looking forward for measurements.

[davidfowl]

This change makes me so happy? This will nuke the remaining byte[] allocations yes?

[wfurt]

Please take a look @stephentoub if you have chance. Not critical for 8.0 IMHO. I wish we have some longevity tests or way to verify that we do not leak the rented buffers. From top, this is only called in to places (Renegotiate & ForceHandshake) and fallback should be covered by the Finally clause.

[karelz]

@stephentoub will you get a chance to take look? (not 8.0 critical)

[wfurt]

this is ready for another round @rzikm @stephentoub.

The change got bigger but hopefully easier to maintain and understand. Most of the operations are now part of the ProtocolToken. It is now passed as ref through PAL laters to places where new buffer is needed.
That also hopefully simplified the logic in Windows safe handle where may get data in two tries.

[rzikm]

LGTM

[stephentoub]

We should ensure we've got good stress test coverage of this. Thanks!