Using DynamoDBLocal with DynamoDB api client causes Credential Provider is closed crash

[jurmous]

Describe the bug

When I update to 0.20.0-beta or 0.20.1-beta I get a Credential Provider closed exception as soon as I try to communicate with a local DynamoDB instance.

Probably it is related to #824?

Expected behavior

No crashes on closed Credential Providers

Current behavior

Crash:

java.lang.IllegalStateException: Credentials provider is closed
	at aws.sdk.kotlin.runtime.auth.credentials.CachedCredentialsProvider.getCredentials(CachedCredentialsProvider.kt:61)
	at aws.sdk.kotlin.runtime.auth.credentials.DefaultChainCredentialsProvider.getCredentials(DefaultChainCredentialsProvider.kt:70)
	at aws.sdk.kotlin.runtime.auth.credentials.internal.ManagedCredentialsProvider.getCredentials(ManagedCredentialsProvider.kt)
	at aws.smithy.kotlin.runtime.auth.awssigning.DefaultAwsSignerImpl.sign(DefaultAwsSigner.kt:28)
	at aws.smithy.kotlin.runtime.auth.awssigning.AwsHttpSigner.sign(AwsHttpSigner.kt:154)
	at aws.smithy.kotlin.runtime.http.operation.HttpAuthHandler.call(SdkOperationExecution.kt:251)
	at aws.smithy.kotlin.runtime.http.operation.HttpAuthHandler.call(SdkOperationExecution.kt:241)
	at aws.sdk.kotlin.runtime.http.middleware.AwsRetryHeaderMiddleware.handle(AwsRetryHeaderMiddleware.kt:35)
	at aws.sdk.kotlin.runtime.http.middleware.AwsRetryHeaderMiddleware.handle(AwsRetryHeaderMiddleware.kt:22)
	at aws.smithy.kotlin.runtime.io.middleware.DecoratedHandler.call(Middleware.kt:40)
	at aws.smithy.kotlin.runtime.io.middleware.Phase.handle(Phase.kt:64)
	at aws.smithy.kotlin.runtime.io.middleware.DecoratedHandler.call(Middleware.kt:40)
	at aws.smithy.kotlin.runtime.http.middleware.RetryMiddleware.tryAttempt-BWLJW6A(RetryMiddleware.kt:79)
	at aws.smithy.kotlin.runtime.http.middleware.RetryMiddleware.access$tryAttempt-BWLJW6A(RetryMiddleware.kt:32)
	at aws.smithy.kotlin.runtime.http.middleware.RetryMiddleware$handle$result$outcome$1$invokeSuspend$$inlined$withChildTraceSpan$1.invokeSuspend(CoroutineContextUtils.kt:238)
	at aws.smithy.kotlin.runtime.http.middleware.RetryMiddleware$handle$result$outcome$1$invokeSuspend$$inlined$withChildTraceSpan$1.invoke(CoroutineContextUtils.kt)
	at aws.smithy.kotlin.runtime.http.middleware.RetryMiddleware$handle$result$outcome$1$invokeSuspend$$inlined$withChildTraceSpan$1.invoke(CoroutineContextUtils.kt)
	at kotlinx.coroutines.intrinsics.UndispatchedKt.startUndispatchedOrReturn(Undispatched.kt:89)
	at kotlinx.coroutines.BuildersKt__Builders_commonKt.withContext(Builders.common.kt:169)
	at kotlinx.coroutines.BuildersKt.withContext(Unknown Source)
	at aws.smithy.kotlin.runtime.http.middleware.RetryMiddleware$handle$result$outcome$1.invokeSuspend(RetryMiddleware.kt:117)
	at aws.smithy.kotlin.runtime.http.middleware.RetryMiddleware$handle$result$outcome$1.invoke(RetryMiddleware.kt)
	at aws.smithy.kotlin.runtime.http.middleware.RetryMiddleware$handle$result$outcome$1.invoke(RetryMiddleware.kt)
	at aws.smithy.kotlin.runtime.retries.StandardRetryStrategy.doTryLoop(StandardRetryStrategy.kt:51)
	at aws.smithy.kotlin.runtime.retries.StandardRetryStrategy.retry(StandardRetryStrategy.kt:32)
	at aws.smithy.kotlin.runtime.http.middleware.RetryMiddleware.handle(RetryMiddleware.kt:47)
	at aws.smithy.kotlin.runtime.http.middleware.RetryMiddleware.handle(RetryMiddleware.kt:32)
	at aws.smithy.kotlin.runtime.io.middleware.DecoratedHandler.call(Middleware.kt:40)
	at aws.smithy.kotlin.runtime.http.operation.MutateHandler.call(SdkOperationExecution.kt:238)
	at aws.smithy.kotlin.runtime.http.operation.MutateHandler.call(SdkOperationExecution.kt:235)
	at aws.smithy.kotlin.runtime.io.middleware.ModifyRequestMiddleware.handle(ModifyRequest.kt:24)
	at aws.smithy.kotlin.runtime.io.middleware.DecoratedHandler.call(Middleware.kt:40)
	at aws.smithy.kotlin.runtime.io.middleware.ModifyRequestMiddleware.handle(ModifyRequest.kt:24)
	at aws.smithy.kotlin.runtime.io.middleware.DecoratedHandler.call(Middleware.kt:40)
	at aws.smithy.kotlin.runtime.io.middleware.ModifyRequestMiddleware.handle(ModifyRequest.kt:24)
	at aws.smithy.kotlin.runtime.io.middleware.DecoratedHandler.call(Middleware.kt:40)
	at aws.smithy.kotlin.runtime.io.middleware.Phase.handle(Phase.kt:64)
	at aws.smithy.kotlin.runtime.io.middleware.DecoratedHandler.call(Middleware.kt:40)
	at aws.smithy.kotlin.runtime.http.operation.SerializeHandler.call(SdkOperationExecution.kt:231)
	at aws.smithy.kotlin.runtime.http.operation.SerializeHandler.call(SdkOperationExecution.kt:209)
	at aws.smithy.kotlin.runtime.http.operation.InitializeHandler.call(SdkOperationExecution.kt:206)
	at aws.smithy.kotlin.runtime.io.middleware.Phase.handle(Phase.kt:60)
	at aws.smithy.kotlin.runtime.io.middleware.DecoratedHandler.call(Middleware.kt:40)
	at aws.smithy.kotlin.runtime.http.operation.OperationHandler.call(SdkOperationExecution.kt:190)
	at aws.smithy.kotlin.runtime.http.operation.OperationHandler.call(SdkOperationExecution.kt:183)
	at aws.smithy.kotlin.runtime.http.operation.SdkHttpOperationKt.execute(SdkHttpOperation.kt:98)
	at aws.smithy.kotlin.runtime.http.operation.SdkHttpOperationKt.roundTrip(SdkHttpOperation.kt:81)
	at aws.sdk.kotlin.services.dynamodb.DefaultDynamoDbClient$query$$inlined$withRootTraceSpan$1.invokeSuspend(CoroutineContextUtils.kt:228)
	at aws.sdk.kotlin.services.dynamodb.DefaultDynamoDbClient$query$$inlined$withRootTraceSpan$1.invoke(CoroutineContextUtils.kt)
	at aws.sdk.kotlin.services.dynamodb.DefaultDynamoDbClient$query$$inlined$withRootTraceSpan$1.invoke(CoroutineContextUtils.kt)
	at kotlinx.coroutines.intrinsics.UndispatchedKt.startUndispatchedOrReturn(Undispatched.kt:89)
	at kotlinx.coroutines.BuildersKt__Builders_commonKt.withContext(Builders.common.kt:169)
	at kotlinx.coroutines.BuildersKt.withContext(Unknown Source)
	at aws.sdk.kotlin.services.dynamodb.DefaultDynamoDbClient.query(DefaultDynamoDbClient.kt:3429)

Steps to Reproduce

• Start a local DynamoDB (com.amazonaws:DynamoDBLocal:1.21.0)
• Start an application creating a DynamoDB local client.
• Try to interact with it (In my case a query)
• Crash

Possible Solution

Downgrade for now to 0.19.4-beta

Context

No response

AWS Kotlin SDK version used

0.20.1-beta

Platform (JVM/JS/Native)

JVM

Operating System and version

MacOS 13.1

[lauzadis]

We recently started throwing this exception when a user tries to call getCredentials() after the credentials provider is already closed. In your workflow, are you making requests using the DynamoDB client after .close() has already been called on it?

[jurmous]

@lauzadis I discovered I made an assumption about the KTOR lifecycle which was wrong and it worked with the AWS clients before 0.20.0-beta. Now I changed to work better with the lifecycle and to close the clients after the ktor server ends, it all works well with the new versions.

Thanks for the response, it triggered me to go deeper with a debug run.

[github-actions]

⚠️COMMENT VISIBILITY WARNING⚠️

Comments on closed issues are hard for our team to see.
If you need more assistance, please either tag a team member or open a new issue that references this one.
If you wish to keep having a conversation with other community members under this issue feel free to do so.