Skip to content

[filesystem] Support AssumeRole STS for RustFS#2989

Open
fresh-borzoni wants to merge 2 commits intoapache:mainfrom
fresh-borzoni:fix-s3-sts-assume-role
Open

[filesystem] Support AssumeRole STS for RustFS#2989
fresh-borzoni wants to merge 2 commits intoapache:mainfrom
fresh-borzoni:fix-s3-sts-assume-role

Conversation

@fresh-borzoni
Copy link
Copy Markdown
Contributor

@fresh-borzoni fresh-borzoni commented Apr 3, 2026

Summary

closes #2661

Fix verified end-to-end against RustFS following #2569
for tiering setup you may consult this: lakekeeper/lakekeeper#1688 in order to make it work with iceberg and lakekeeper

Before fix:

  SecurityTokenException: Failed to get file access security token: Region is not set.
  -> NoAwsCredentialsException: Dynamic session credentials for Fluss: No AWS Credentials
  ->  Flink job FAILED

After fix:

  S3DelegationTokenProvider - Obtaining session credentials via AssumeRole with access key: rustfsadmin, role: arn:xxx:xxx:xxx:xxxx
  S3DelegationTokenProvider - Session credentials obtained successfully with access key: I4MVXCHH4OV01YLLA9MC
  S3DelegationTokenReceiver - Session credentials updated successfully
  -> Flink job RUNNING

Config required for RustFS users:

  s3.region: us-east-1
  s3.assumed.role.arn: arn:xxx:xxx:xxx:xxxx
  s3.assumed.role.sts.endpoint: http://rustfs:9000

Backward compatibility:

  • Without s3.assumed.role.arn, falls back to GetSessionToken (existing AWS behavior)
  • Zero breaking changes for existing users

@fresh-borzoni
Copy link
Copy Markdown
Contributor Author

@leekeiabstraction @luoyuxia @wuchong
PTAL 🙏

Copy link
Copy Markdown
Contributor

Copilot AI left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Pull request overview

This PR extends Fluss’s S3 delegation-token acquisition to support RustFS/MinIO-style STS by allowing AssumeRole (instead of only GetSessionToken) and by supporting a configurable STS endpoint.

Changes:

  • Add support for obtaining temporary credentials via AssumeRole when fs.s3a.assumed.role.arn is configured.
  • Add support for using a custom STS endpoint via fs.s3a.assumed.role.sts.endpoint.
  • Refactor STS client construction into a helper method.

💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.

@fresh-borzoni
Copy link
Copy Markdown
Contributor Author

Addressed Copilot comments
@luoyuxia PTAL

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

Support obtain sts token from rustfs

2 participants