CASSANALYTICS-155: Add IAM credential support for S3 storage transport#212
Merged
Merged
Conversation
rustyrazorblade
force-pushed
the
iam_credentials
branch
2 times, most recently
from
6152c5c to
fb1d1f5
Compare
yifan-c
reviewed
May 28, 2026
Contributor
yifan-c
left a comment
yifan-c
left a comment
There was a problem hiding this comment.
Please update changes.txt.
It is sad that we have duplicate the code between Sidecar and Analytics
| } | ||
|
|
||
| @Test | ||
| void testHashcodeAndEquals() |
Contributor
There was a problem hiding this comment.
this test is also removed.
| transitionalStateStart.countDown(); | ||
| TestUninterruptibles.awaitUninterruptiblyOrThrow(transitionalStateStart, 4, TimeUnit.MINUTES); | ||
| TestUninterruptibles.awaitUninterruptiblyOrThrow(transitionalStateEnd, 2, TimeUnit.MINUTES); | ||
| Uninterruptibles.awaitUninterruptibly(transitionalStateEnd, 2, TimeUnit.MINUTES); |
Contributor
There was a problem hiding this comment.
Not sure why they are changed; seemingly unrelated change
Contributor
Author
There was a problem hiding this comment.
Several tests routinely fail due to a race condition here. I figured a green CI would be better than not, but happy to remove it if you want.
| protected void completeTransitionsAndValidateWrites(CountDownLatch transitionalStateEnd, Stream<Arguments> testInputs) | ||
| { | ||
| for (int i = 0; i < leavingNodesPerDc(); i++) | ||
| for (int i = 0; i < leavingNodes.size(); i++) |
Contributor
There was a problem hiding this comment.
Unrelated change; can we revert it?
rustyrazorblade
force-pushed
the
iam_credentials
branch
from
e6a1bc5 to
67090fe
Compare
yifan-c
approved these changes
May 29, 2026
Adds IAM-based authentication as an alternative to static credentials for S3 storage transport. Also fixes test report collection in CI: corrects Gradle output paths for both GitHub Actions (adds missing artifact uploads with if: always()) and CircleCI (fixes SRC_REPORT_DIR and store_test_results paths so real JUnit XML is collected instead of synthesized fallbacks). Patch by Jon Haddad; reviewed by Yifan Cai for CASSANALYTICS-155
rustyrazorblade
force-pushed
the
iam_credentials
branch
2 times, most recently
from
79bd150 to
87c553c
Compare
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
2 participants
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
CredentialTypeenum (STATIC/IAM) to control how Spark executors and Sidecar authenticate to S3STATIC(default): existing behavior — STS access key/secret/session token passed viaStorageCredentialsIAM: no credentials passed; both executor and Sidecar use the AWS SDK default provider chain (EC2 instance profile, EKS IRSA, ECS task role)StorageAuthinterface withStaticStorageAuthandIamStorageAuthimplementationsStorageCredentialPair.iamPair(writeRegion, readRegion)factory for IAM-only pairsRestoreJobSecretsupdated to carry the credential type and omit secrets when IAM is usedstorage_credential_typeoption and S3 bucket topologyTest plan
BuildRestoreJobSecretsTest,RestoreJobSecretsTest,StorageCredentialsTest,StorageCredentialPairTest, andCreateRestoreJobRequestPayloadTestcover bothSTATICandIAMpathsStorageAccessConfigurationTestupdated to reflect new auth modelCloudStorageStreamSessionTestupdated for credential type propagationJIRA
https://issues.apache.org/jira/browse/CASSANALYTICS-155