AWS: Support assumed role credentials for REST SigV4 signing#16687
Open
Martozar wants to merge 1 commit into
Open
AWS: Support assumed role credentials for REST SigV4 signing#16687Martozar wants to merge 1 commit into
Martozar wants to merge 1 commit into
Conversation
When client.assume-role.arn is set, AWS clients created by AssumeRoleAwsClientFactory assume the configured role, but the REST catalog SigV4 signer kept signing requests with the base credentials. Wrap the base credentials in an auto-refreshing StsAssumeRoleCredentialsProvider so that REST catalog requests and data access are performed with the same assumed role. This enables credential-less SigV4 REST catalog authentication (e.g. instance profile or IRSA base credentials combined with a cross-account role). The returned provider closes the underlying STS client and base credentials provider when the auth session is closed. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Motivation
When
client.assume-role.arnis set, AWS clients created byAssumeRoleAwsClientFactory(S3, Glue, DynamoDB, KMS) assume the configured role, but the REST catalog SigV4 signer keeps signing requests with the base credentials. Catalog requests and data access are therefore performed as two different identities, which breaks credential-less setups (instance profile / IRSA base credentials + a cross-account role) against SigV4-signed REST catalogs.Changes
AwsProperties.restCredentialsProvider()wraps the base credentials in an auto-refreshingStsAssumeRoleCredentialsProviderwhenclient.assume-role.arnis set, reusing the existingclient.assume-role.*properties (session name, external-id, timeout, tags) consistently withAssumeRoleAwsClientFactory.SdkAutoCloseableand closes the underlying STS client and therefore performed as two different identities, which breaks credential-less setups (instance profile / IRSA base credentials + a cross-account role) against SigV4-signed REST catalogs.Changes
AwsProperties.restCredentialsProvider()wraps the base credentials in an auto-refreshingStsAssumeRoleCredentialsProviderwhenclient.assume-role.arnis set, auto-refreshingStsAssumeRoleCredentialsProviderwhenclient.assume-role.arnis set, reusing the existingclient.assume-role.*properties (session name, external-id, timeout, tags) consistently withAssumeRoleAwsClientFactory.SdkAutoCloseableand closes the underlying STS client and base provider;RESTSigV4AuthSessionalready closes closeable providers since #.Behavior change
Users who set
client.assume-role.arnfor data access while SigV4-signing with base credentials will now sign catalog requests with the assumed role. This is the intended consistency fix, but it changes the signing identity for that configuration.Alternatives considered
client.credentials-providerwith a custom provider class: requires every user to ship a wrapper class, since the SDK'sStsAssumeRoleCredentialsProviderhas no staticcreate()/create(Map)factory.Testing
New unit tests in
TestAwsPropertiesandTestRESTSigV4AuthSession;./gradlew :iceberg-aws:checkpasses.AI disclosure
Drafted with AI assistance (Claude Code) based on a patch we run in production against StarRocks-bundled Iceberg; design, tests and verification reviewed by the author. Reviewer attention welcome on the STS client lifecycle/closing in
AssumeRoleRestCredentialsProvider.