-
Notifications
You must be signed in to change notification settings - Fork 863
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Updated credential providers to provide Async version for generating credentials. #3681
base: v4-development
Are you sure you want to change the base?
Updated credential providers to provide Async version for generating credentials. #3681
Conversation
997906f
to
75f984d
Compare
c4612e8
to
e90cbf2
Compare
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
I took an initial pass but as discussed offline there's a lot of code that had to be duplicated and I'm not sure it's the best approach.
For the question in the description, we removed the credential retriever pipeline handler and the signers now retrieve credentials themselves. The async method in the signer just invokes the sync version:
aws-sdk-net/sdk/src/Core/Amazon.Runtime/Internal/Auth/AbstractAWSSigner.cs
Lines 115 to 130 in 9d4fea3
public virtual System.Threading.Tasks.Task SignAsync( | |
IRequest request, | |
IClientConfig clientConfig, | |
RequestMetrics metrics, | |
BaseIdentity identity, | |
CancellationToken token = default) | |
{ | |
Sign(request, clientConfig, metrics, identity); | |
#if NETSTANDARD | |
return Task.CompletedTask; | |
#else | |
return Task.FromResult(0); | |
#endif | |
} | |
#endif | |
"Updated credential providers to provide Async version for generating credentials." | ||
], | ||
"type": "patch", | ||
"updateMinimum": false |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
It doesn't matter here for V4 since we ship all packages, but this should be true
(it's rare that a change to Core won't require all services to be updated too).
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Fixed.
@@ -35,11 +35,9 @@ public abstract class AWSCredentials : BaseIdentity | |||
/// </summary> | |||
protected virtual void Validate() { } | |||
|
|||
#if AWS_ASYNC_API |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Why are we removing AWS_ASYNC_API
? (By the way I think this constant is redundant since every target supports async but it's in a lot of other places too)
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
I removed it based on discussion with @normj since it is now redundant. And based on discussion, the current scope of removal was limited to credential providers.
|
||
[Fact] | ||
[Trait(TestBase.CategoryAttribute, "Credentials")] | ||
public async Task TestAssumeRoleAWSCredentialsAsyncGet() |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
FYI This test doesn't run in our build system (it skips the RequiresIAMUser
category that's applied to the parent class).
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Thanks for letting me know. Do we have reasoning for excluding the test RequiresIAMUser
category?
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
If you're updating this file I'm wondering if it makes sense to also fix #3613
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Discussed this with @norm and looked at the backlog item. The issue needs investigation per backlog item on why preemptExpiryTime
parameter is not used, and if we should be using it. It's the pending item and we should discuss about it.
@dscpinheiro Thanks for the review. I understand the signer will call Async version based on stack trace I shared earlier: AWSSDK.Core.dll!Amazon.Runtime.RefreshingAWSCredentials.GetCredentialsAsync() Line 183 C#
AWSSDK.Core.dll!Amazon.Runtime.Internal.Signer.SignRequestAsync(Amazon.Runtime.IRequestContext requestContext) Line 158 C#
AWSSDK.Core.dll!Amazon.Runtime.Internal.Signer.PreInvokeAsync(Amazon.Runtime.IExecutionContext executionContext) Line 82 C#
AWSSDK.Core.dll!Amazon.Runtime.Internal.Signer.InvokeAsync<Amazon.EC2.Model.DescribeTagsResponse>(Amazon.Runtime.IExecutionContext executionContext) Line 57 C#
AWSSDK.Core.dll!Amazon.Runtime.Internal.PipelineHandler.InvokeAsync<Amazon.EC2.Model.DescribeTagsResponse>(Amazon.Runtime.IExecutionContext executionContext) Line 82 C#
AWSSDK.Core.dll!Amazon.Runtime.Internal.EndpointDiscoveryHandler.InvokeAsync<Amazon.EC2.Model.DescribeTagsResponse>(Amazon.Runtime.IExecutionContext executionContext) Line 76 C#
AWSSDK.Core.dll!Amazon.Runtime.Internal.PipelineHandler.InvokeAsync<Amazon.EC2.Model.DescribeTagsResponse>(Amazon.Runtime.IExecutionContext executionContext) Line 82 C#
AWSSDK.Core.dll!Amazon.Runtime.Internal.ChecksumHandler.InvokeAsync<Amazon.EC2.Model.DescribeTagsResponse>(Amazon.Runtime.IExecutionContext executionContext) Line 52 C#
AWSSDK.Core.dll!Amazon.Runtime.Internal.PipelineHandler.InvokeAsync<Amazon.EC2.Model.DescribeTagsResponse>(Amazon.Runtime.IExecutionContext executionContext) Line 82 C# I'm referring to sync call in below stack trace after AWSSDK.Core.dll!Amazon.Runtime.AssumeRoleAWSCredentials.GenerateNewCredentials() Line 105 C#
AWSSDK.Core.dll!Amazon.Runtime.RefreshingAWSCredentials.GetCredentials.__GenerateCredentialsAndUpdateState|11_0() Line 166 C#
AWSSDK.Core.dll!Amazon.Runtime.RefreshingAWSCredentials.GetCredentials() Line 150 C#
AWSSDK.Core.dll!Amazon.Runtime.Internal.EndpointDiscoveryHandler.ProcessEndpointDiscovery(Amazon.Runtime.IRequestContext requestContext, bool evictCacheKey, System.Uri evictUri) Line 136 C#
AWSSDK.Core.dll!Amazon.Runtime.Internal.EndpointDiscoveryHandler.DiscoverEndpoints(Amazon.Runtime.IRequestContext requestContext, bool evictCacheKey) Line 112 C#
AWSSDK.Core.dll!Amazon.Runtime.Internal.EndpointDiscoveryHandler.PreInvoke(Amazon.Runtime.IExecutionContext executionContext) Line 101 C#
AWSSDK.Core.dll!Amazon.Runtime.Internal.EndpointDiscoveryHandler.InvokeAsync<Amazon.EC2.Model.DescribeTagsResponse>(Amazon.Runtime.IExecutionContext executionContext) Line 72 C#
AWSSDK.Core.dll!Amazon.Runtime.Internal.PipelineHandler.InvokeAsync<Amazon.EC2.Model.DescribeTagsResponse>(Amazon.Runtime.IExecutionContext executionContext) Line 82 C#
AWSSDK.Core.dll!Amazon.Runtime.Internal.ChecksumHandler.InvokeAsync<Amazon.EC2.Model.DescribeTagsResponse>(Amazon.Runtime.IExecutionContext executionContext) Line 52 C#
AWSSDK.Core.dll!Amazon.Runtime.Internal.PipelineHandler.InvokeAsync<Amazon.EC2.Model.DescribeTagsResponse>(Amazon.Runtime.IExecutionContext executionContext) Line 82 C#
AWSSDK.Core.dll!Amazon.Runtime.Internal.RetryHandler.InvokeAsync<Amazon.EC2.Model.DescribeTagsResponse>(Amazon.Runtime.IExecutionContext executionContext) Line 135 C#
AWSSDK.Core.dll!Amazon.Runtime.Internal.PipelineHandler.InvokeAsync<Amazon.EC2.Model.DescribeTagsResponse>(Amazon.Runtime.IExecutionContext executionContext) Line 82 C#
AWSSDK.Core.dll!Amazon.Runtime.Internal.CompressionHandler.InvokeAsync<Amazon.EC2.Model.DescribeTagsResponse>(Amazon.Runtime.IExecutionContext executionContext) Line 56 C#
AWSSDK.Core.dll!Amazon.Runtime.Internal.PipelineHandler.InvokeAsync<Amazon.EC2.Model.DescribeTagsResponse>(Amazon.Runtime.IExecutionContext executionContext) Line 82 C#
AWSSDK.Core.dll!Amazon.Runtime.Internal.CallbackHandler.InvokeAsync<Amazon.EC2.Model.DescribeTagsResponse>(Amazon.Runtime.IExecutionContext executionContext) Line 61 C#
AWSSDK.Core.dll!Amazon.Runtime.Internal.PipelineHandler.InvokeAsync<Amazon.EC2.Model.DescribeTagsResponse>(Amazon.Runtime.IExecutionContext executionContext) Line 82 C#
AWSSDK.EC2.dll!Amazon.EC2.Internal.AmazonEC2PostMarshallHandler.InvokeAsync<Amazon.EC2.Model.DescribeTagsResponse>(Amazon.Runtime.IExecutionContext executionContext) Line 57 C#
|
e90cbf2
to
06a4959
Compare
…or getting credentials.
06a4959
to
e5a45f5
Compare
@dscpinheiro Thanks for the feedback. I had discussed this with @normj and fixed it in commit e5a45f5. Please review. |
Description
Updated credential providers to provide Async version for generating credentials.
NOTE:
Async methods don't have parameter for
CancellationToken
since theGetCredentialsAsync()
doesn't have it.Motivation and Context
Issue #3626
Existing customer PR #3627
Testing
TestAssumeRoleAWSCredentialsAsyncGet
andTestEC2InstanceMetadataSecureCredentialsSuccessAsync
.DRY_RUN-e95a1656-57b3-44db-88d5-d14060e1e1db
completed successfully.References for SAML Federation:
This requires reserving Elastic IP address, create public hosted zone and registering domain name.
Federated SAML credentials cannot be tested via shared credentials file. I was able to step through the code and hack way around. The problem is that EndpointDiscoveryHandler.ProcessEndpointDiscovery() uses non-async version to get credentials.
Testing Customer's scenario:
TestEC2ReadOnlyRole
withAmazonEC2ReadOnlyAccess
permission policy and below trust relationship with the account:~/.aws/config
file, setup below profile:default
profile are configured in~/.aws/credentials
.AWS_PROFILE
environment variable totestassumerole
.While the
async
path forICoreAmazonSTS.CredentialsFromAssumeRoleAuthenticationAsync()
is triggered inV4
branch for this PR, but below are my observations which highlight thatsync
path is triggered at some point. Comparison between V3 and V4 stack trace is also mentioned.@normj / @dscpinheiro Prior to implementation of SRA, SDK used CredentialsRetriever (refer stack trace in customer's issue #3626). It emitted InvokeAsync() versions. Is there an equivalent in SRA world? Now
EndpointDiscoveryHandler
always uses sync version at EndpointDiscoveryHandler.ProcessEndpointDiscovery(). We have removedCredentialsRetriever
in V4 in commit. Please suggest if async invocation for retrieving credentials is no longer possible in V4!V3 Call stack:
While debugging it worked fine.
V4 Call stack:
It ultimately calls
async
version while invoking Signer to sign the request:But looks like at some point(s) we make sync calls. And it might fail at the previous sync stack trace while assuming role, on platforms like Android which don't support sync calls (per customer's issue).
Screenshots (if appropriate)
Types of changes
Checklist
License