Getting SdkClientException (Unable to execute HTTP request: Timeout waiting for connection from pool)' after several days of running the application

[raviagarwal101]

Upcoming End-of-Support

• I acknowledge the upcoming end-of-support for AWS SDK for Java v1 was announced, and migration to AWS SDK for Java v2 is recommended.

Describe the bug

We are using AWS SDK for interacting with AWS DynamoDB.

Using com.amazonaws.services.dynamodbv2.datamodeling.DynamoDBTableMapper class and .save(entity) method to save objects to DynamoDB.

The application runs fine for several days, and then suddenly starts throwing below exceptions

exception: 'SdkClientException (Unable to execute HTTP request: Timeout waiting for connection from pool)' that was caused by: org.apache.http.impl.conn.PoolingHttpClientConnectionManager.leaseConnection(PoolingHttpClientConnectionManager.java:316)

The only way to get rid of it is by restarting the application.

SDK version: 1.12.777

Regression Issue

• Select this option if this issue appears to be a regression.

Expected Behavior

The expectation is to not throw the SDKClientException

Current Behavior

exception: 'SdkClientException (Unable to execute HTTP request: Timeout waiting for connection from pool)' that was caused by: org.apache.http.impl.conn.PoolingHttpClientConnectionManager.leaseConnection(PoolingHttpClientConnectionManager.java:316)

Reproduction Steps

Using the provided methods in DynamoDBTableMapper class to save, delete, query

Possible Solution

No response

Additional Information/Context

No response

AWS Java SDK version used

1.12.777

JDK version used

openjdk version "21.0.1" 2023-10-17 LTS

Operating System and version

AWS ECS Fargate 1.4.0

[debora-ito]

@raviagarwal101

Timeout waiting for connection from pool indicates that all the connections from the pool are in use, and a new incoming request timed out while waiting for a connection to be available.

For timeout investigations we recommend to generate SDK client metrics and to monitor HttpClientPool and ClientExecuteTime metrics over time. And also take a look at this blog post that shows how to interpret the metrics.