ClientConfig.EndpointDiscoveryEnabled is checked on every request, causing poor performance #1270
Description
Hi. We are using the AWS sdk for DynamoDBv2.
Explicitly setting EndpointDiscoveryEnabled on the AmazonDynamoDBConfig to any value (true OR false) uses 40% less CPU per request. With this setting the sdk also allocates 45% less garbage per request (900kb -> 500kb).
Profiling reveals that FallbackEndpointDiscoveryEnabledFactory.Enabled is called on every dynamoDB request. It uses CredentialProfileStoreChain to look for config files (I assume), which is expensive. The result of this operation is never cached.
default AmazonDynamoDBConfig (10 concurrent PutItem requests, see code below)
same test as above, but EndpointDiscoveryEnabled set to any value
(please note that the absolute values reported in the profiler are much higher than usual. this is due to deep-profiling. Running the test without deep profiling yields the same results)
Expected Behavior
Expensive configuration checks are cached
Current Behavior
FallbackEndpointDiscoveryEnabledFactory uses CredentialProfileStoreChain to check config files on every single request.
Possible Solution
Cache configuration variable after retrieving the result. I'm unsure how the sdk usually caches values like these. One option would be to cache it in ClientConfig.cs
public bool EndpointDiscoveryEnabled
{
get
{
if (!this.endpointDiscoveryEnabled.HasValue)
{
var endpointDiscoveryEnabled = FallbackEndpointDiscoveryEnabledFactory.GetEnabled();
if (endpointDiscoveryEnabled != null)
{
this.endpointDiscoveryEnabled.Value = endpointDiscoveryEnabled.Value; // SUGGESTED PATCH
// return endpointDiscoveryEnabled.Value; // COMMENTED OUT
}
else
{
this.endpointDiscoveryEnabled.Value = false; // SUGGESTED PATCH
// return false; // COMMENTED OUT
}
}
return this.endpointDiscoveryEnabled.Value;
}
set { this.endpointDiscoveryEnabled = value; }
}Steps to Reproduce (for bugs)
code used for benchmark:
using System;
using System.Collections.Generic;
using System.Net;
using System.Threading;
using System.Threading.Tasks;
using Amazon;
using Amazon.DynamoDBv2;
using Amazon.DynamoDBv2.Model;
using UnityEngine;
public class DynamoTest : MonoBehaviour {
AmazonDynamoDBClient dy;
void Start() {
dy = new AmazonDynamoDBClient("key", "secretkey", CreateDynamoConfig());
InvokeRepeating(nameof(PutMany), 0f, 5f);
}
public static AmazonDynamoDBConfig CreateDynamoConfig() {
var config = new AmazonDynamoDBConfig();
config.RegionEndpoint = RegionEndpoint.GetBySystemName("us-east-1");
config.ServiceURL = "http://localhost:8000/";
config.EndpointDiscoveryEnabled = true; // <--- 50% CPU reduction if set to any value!
return config;
}
async void PutMany() {
var tasks = new List<Task<PutItemResponse>>();
for (int i = 0; i < 10; i++) {
tasks.Add(Put());
}
await Task.WhenAll(tasks);
}
Task<PutItemResponse> Put() {
return dy.PutItemAsync(new PutItemRequest {
TableName = "MyTable",
Item = new Dictionary<string, AttributeValue> {
{ "pk", new AttributeValue("1") },
{ "sk", new AttributeValue(Guid.NewGuid().ToString()) },
}
});
}
}Context
High-performance server-application (>100 dynamodb requests per second). Heavy CPU-usage and GC allocations are avoided as much as possible in the application-code.
Your Environment
- AWSSDK.Core version used: 0585ea6 (HEAD -> master, tag: 3.3.492.0)
- Service assembly and version used: DynamoDBv2 netstandard2.0 (built on commit above)
- Operating System and version: Windows10 64bit
- Visual Studio version: (not using Visual Studio)
- Targeted .NET platform: .net standard 2.0
.NET Core Info
running on Unity 2019.1.0f2 . note that we are using NetStandard 2.0 (not unity dlls!)