What is the reason for keeping HttpClientCacheSize to number Environment.ProcessorCount on Linux

[madhub]

The Question

Currently we are investigating slowness of S3 access on Linux deployment, upon investigation we found number of cache HttpClient varies between Windows & Linux relevant section of the code is ClientConfig.cs

What is the reason of having different number for cached HttpClients ?
From .NET Core 2.1 onwards HttpClient uses SocketsHttpHandler on both Windows & Linux and their should not be any behavior changes.

Here is the sample scenario to show the difference
On 10 CPU systems make 10 S3 request to same object on the same bucket.

• On Windows - First request takes time, and all subsequent request are fast
• On Linux - All request request takes equal amount of time which includes connection establish cost

So should we set the AmazonS3Config.HttpClientCacheSize to 1 on Linux ? , any recommendation ?

Environment

• SDK Version: 3.7.3.12
• Package Version: Awssdk.s3 ver: 3.7.3.12, Awssdk.core ver:3.7.3.19
• OS Info: Windows 10 , Centos 8
• Build Environment :Terminal dotnet
• Targeted .NET Platform: .NET 3.1

---

This is a ❓ general question

[ashishdhingra]

Hi @madhub,

Good morning.

Thanks for posting the guidance question. Looking at the documentation comment for HttpClientCacheSize, it mentions that On Windows the default value is 1 since the underlying native implementation does not have throttling constraints like the non Windows Curl based implementation. For non Windows based platforms the default is the value return from System.Environment.ProcessorCount.. Based on the value of the cache size, it creates number of HttpClient instances at HttpRequestMessageFactory.CreateHttpClientCache(). The HttpClient is retrieved from client cache via HttpClientCache.GetNextClient(). If you notice that if the number of HttpClient(s) in cache is 1, it just returns the first client, else it uses Interlocked.Increment() to randomly determine the client to be used from cache. This might cause some lag due to thread safe locking.

While we cannot advise if you should set AmazonS3Config.HttpClientCacheSize to 1 on Linux since as mentioned above, the cache is used to prevent throttling issues. Changing the default behavior could result in throttling issues in non-Windows environments as comment mentions. Hope this helps.

Thanks,
Ashish

[madhub]

Thanks for the response,

.NET Core 2.1 on wards implementation is same for Windows & Non Windows platforms .See the documentation here HttpClient Sockets improvements.
Both Windows & non Windows platform uses the same implementation SocketsHttpHandler

.NET Core 2.1 Release
consistent-cross-platform-implementation

More information about how HttpClient internal implemented before .NET Core 2.1 and after 2.x

[normj]

@madhub when this code was written .NET Core was using a different implementation for windows and linux. We do need to revise this code now that both platforms use SocketsHttpHandler.

I suspect you are right and we should just use one cached HttpClient with the modern versions of .NET Core. We will need to do some validation first before making that change.

[madhub]

Note that HttpClient uses SocketsHttpHandler common implementation from .NET Core 2.1 onwards , previous versions ( .NET Core 2.0) still uses different implementation for Windows(WinHttpHandler) & Linux (CurlHandler) , since AWS SDK is .NETStandard 2.0 library same can be used deployed in .NET Core 2.0 , .NET Core 2.1,.NET Core 3.1 and NET 5 runtime hence AmazonS3Config.HttpClientCacheSize should be set to one only if the runtime is >.NET Core 2.1

[ozziepeeps]

We got bitten by this exact issue, combined with our use of the new DOTNET_PROCESSOR_COUNT environment variable in NET6 which exacerbated things. We can confirm that setting HttpClientCacheSize to 1 improved performance in our setup.

It would be good to get this fixed in the SDK @normj.

[rsaunderson88-kw]

I was also just bit by this when testing on Linux. Requests under 1 MB went from taking almost 200 ms to finish GetObjectAsync and another 200 to execute CopyToAsync to 50ms round trip. This is a pretty large difference even for a simple example.

[danielmarbach]

Why hasn't this value been streamlined? I mean we are now alread in .NET 8 preview mode, which will be the next LTS and all of what has been written here in this issue is valid for multiple years now.

[danielmarbach]

See #2549

[ashishdhingra]

Closing issue since PR #2549 is merged and released per #2549 (comment).

[github-actions]

⚠️COMMENT VISIBILITY WARNING⚠️

Comments on closed issues are hard for our team to see.
If you need more assistance, please either tag a team member or open a new issue that references this one.
If you wish to keep having a conversation with other community members under this issue feel free to do so.