[Hotfix 25.9] feat(s3): support credential-scoped endpoint and OSS storage provider#2020
Merged
benflexcompute merged 1 commit intoApr 24, 2026
Conversation
…#2019) Co-authored-by: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
angranl-flex
approved these changes
Apr 24, 2026
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Hotfix port of #2019 (merged to main as 45677bd) onto
release-candidate/25.9.Summary
endpointandstorageProviderfields to_UserCredential, letting the STS response redirect the S3 client to a non-AWS backend.storageProvider == OSS, disable AWS integrity-check headers and switch to virtual-hosted addressing; preserves existing path-style behavior for other S3-compatible stores (s3proxy, MinIO) viaEnv.current.s3_endpoint_url.user_credential.endpointto the boto3 client.Env.current.s3_endpoint_urlremains the highest-priority override (manual user setting).Cherry-pick
Test plan
storageProvider, noendpoint) still builds client unchanged.storageProvider=OSSpath: virtual-hosted addressing + checksums disabled.Env.current.s3_endpoint_urlset, nostorageProvider): path-style + checksums disabled.user_credential.endpointset: client uses thatendpoint_urlunless env override is also set.🤖 Generated with Claude Code
Note
Medium Risk
Touches S3 client construction (endpoint selection, addressing style, checksum behavior), which can impact all upload/download flows if provider detection or endpoint precedence is wrong.
Overview
Adds
endpointandstorageProviderfields to the STS-derived_UserCredentialso backend-issued credentials can direct the client to a specific S3-compatible endpoint.Updates
_S3STSToken.get_client()to apply the credential-scopedendpoint_url(while keepingEnv.current.s3_endpoint_urlas the highest-priority override) and to special-casestorageProvider==OSSby disabling checksum validation and using virtual-hosted addressing; other non-AWS endpoints continue using path-style addressing.Reviewed by Cursor Bugbot for commit bcf46e1. Bugbot is set up for automated code reviews on this repo. Configure here.