.NET 10: HTTPS connection establishment latency regression compared to .NET 9, triggering downstream timeout policies

[perkops]

Description

After upgrading from .NET 9 to .NET 10 (no code or package changes), HTTPS requests through SocketsHttpHandler more frequently require connection times that exceed 500ms — a threshold that was consistently met on .NET 9.

This is causing failures in the Azure Cosmos DB .NET SDK, which uses a hard-coded 500ms first-attempt timeout for internal metadata/address resolution requests.

We have filed Azure/azure-cosmos-dotnet-v3#5642 for the SDK side, but the underlying question remains: what changed in .NET 10's SocketsHttpHandler/SslStream pipeline that increases HTTPS request latency compared to .NET 9?

We cloned the Cosmos DB SDK and changed the first-attempt timeout from 500ms to 5 seconds. All errors disappeared completely. This confirms that requests which completed within 500ms on .NET 9 now intermittently exceed 500ms on .NET 10 — with no code or package changes, only the runtime upgrade.

See the detailed reproduction steps and screenshots in azure-cosmos-dotnet-v3#5642.

Reproduction Steps

See the detailed reproduction steps and screenshots in azure-cosmos-dotnet-v3#5642, which includes before/after comparisons with traffic-shaped latency on both .NET 9 and .NET 10.

Expected behavior

HTTPS requests through SocketsHttpHandler should have comparable connection establishment latency to .NET 9. Requests that consistently completed within 500ms on .NET 9 should not intermittently exceed 500ms on .NET 10.

Actual behavior

• Intermittent TaskCanceledException on HTTPS requests that take slightly over 500ms
• The same requests consistently complete within 500ms on .NET 9
• We can reproduce this locally using traffic shaping and introducing artificial delays
• It happens intermittently inside of Azure

Regression?

No response

Known Workarounds

No response

Configuration

• .NET 9: No issues (identical code and packages)
• .NET 10.0.x: Reproducible
• OS: Linux (Azure App Services) and Windows
• Downstream library: Azure Cosmos DB SDK 3.46.0

Other information

Potentially relevant .NET 10 changes

• #112383 — Disposed HTTP/1.1 connections are no longer returned to the pool, potentially reducing pool hit rate and forcing more fresh TCP+TLS connection establishments
• #110744 — Race condition fix in connection timeout CTS assignment, changes connection establishment timing

Stack trace

  System.Threading.Tasks.TaskCanceledException: The operation was canceled.
   ---> System.IO.IOException: Unable to read data from the transport
        connection: Operation canceled.
   ---> System.Net.Sockets.SocketException (125): Operation canceled
     at System.Net.Sockets.Socket.AwaitableSocketAsyncEventArgs
        .ThrowException(...)
     at System.Net.Security.SslStream
        .EnsureFullTlsFrameAsync...
     at System.Net.Security.SslStream
        .ReadAsyncInternal...
     at System.Net.Http.HttpConnection.SendAsync(...)
     at System.Net.Http.HttpConnectionPool
        .SendWithVersionDetectionAndRetryAsync(...)
     at System.Net.Http.Metrics.MetricsHandler
        .SendAsyncWithMetrics(...)
     at System.Net.Http.DiagnosticsHandler
        .SendAsyncCore(...)

Impact

The Azure Cosmos DB .NET SDK has an internal 500ms first-attempt timeout for control plane operations that worked reliably on .NET 9. After upgrading to .NET 10, these requests intermittently exceed 500ms, causing recurring TaskCanceledException errors across multiple microservices in production (Azure App Services).

While the immediate fix belongs in the Cosmos SDK (azure-cosmos-dotnet-v3#5642),
the latency regression in SocketsHttpHandler may affect other libraries with similar internal timeout policies.

[ManickaP]

I'm looking at the screenshots from the cosmos repo issue and I have few follow up questions:

• Do you also have screens without the artificial delay?
• Is the 4-5ms duration for .NET 9? Or are those just requests on already established connection?
• What was the normal duration for connection establishment on .NET 9 vs now on .NET 10?
  • Is the difference significant or was it always >400 ms and now we're just few ms slower and thus sometimes over the threshold?
• How often do these timeouts occur in comparison to connections that succeed to be established?

[perkops]

I'm looking at the screenshots from the cosmos repo issue and I have few follow up questions:

• Do you also have screens without the artificial delay?

Yes - The first screenshot in the Azure/azure-cosmos-dotnet-v3#5642 shows the scenario without artificial delay — no errors with normal latency.

• Is the 4-5ms duration for .NET 9? Or are those just requests on already established connection?

All screenshots are from .NET 10 running locally and using a Cosmos Emulator.

The 4-5 ms are when no artifical delay was introduced.

We added the artifical delay to simulate, what we see in Azure, when our PaaS service communicates with CosmosDB, where we often see spikes on communication from 4-5ms to above 500ms (after upgrading to .NET 10 - services still running .NET 9 are not affected by these issues).

• What was the normal duration for connection establishment on .NET 9 vs now on .NET 10?

  • Is the difference significant or was it always >400 ms and now we're just few ms slower and thus sometimes over the threshold?

We dont have those numbers, we can just see that after upgrading to .NET 10 we frequently exceed the 500ms and hence throws exceptions and triggers our alert systems.

• How often do these timeouts occur in comparison to connections that succeed to be established?

In production (Azure App Services, no artificial delay): The timeouts occur intermittently — not on every cache refresh, but often enough to trigger our alerting systems. We suspect connections occasionally exceeds 500ms.

[perkops]

@ManickaP Just for reference, heres a screenshot from actual production environment with the issue.

[ManickaP]

Yeah, this is a one instance of the failure. But right now we don't have the info about the mean duration, the duration for .NET 9 etc. for comparison. At this point and state, there's just not enough info for us to make this actionable. I quickly checked the changes that went into .NET 10 in HTTP and Security and I don't see anything suspicious.
On top of that, it would be good to retry with .NET 9, to make sure this really is a regression in .NET 10 and not just a coincidence with some environmental change that happened at the same time.

[pinkfloydx33]

After upgrade to net10 we're experiencing similar with token requests through Azure Identity (ie. for Azure Postgres Workload Identity Auth). It only seems to be happening during initial application startup. We resolved by extending timeouts and adding a retry (which we were fortunate enough to be able to control) but a downgrade back to net9 doesn't exhibit the behavior.

[ManickaP]

If it's related only to start up, it could be #115688. But still we'd need some hard data (measurements, metrics, traces etc.) and narrow it down specifically to this change.
cc @MihaZupan