Skip to content

[Hotfix 25.9] feat(s3): support credential-scoped endpoint and OSS storage provider#2020

Merged
benflexcompute merged 1 commit into
release-candidate/25.9from
ben/hotfix-25.9-oss-credential-endpoint
Apr 24, 2026
Merged

[Hotfix 25.9] feat(s3): support credential-scoped endpoint and OSS storage provider#2020
benflexcompute merged 1 commit into
release-candidate/25.9from
ben/hotfix-25.9-oss-credential-endpoint

Conversation

@benflexcompute
Copy link
Copy Markdown
Collaborator

@benflexcompute benflexcompute commented Apr 24, 2026

Hotfix port of #2019 (merged to main as 45677bd) onto release-candidate/25.9.

Summary

  • Add endpoint and storageProvider fields to _UserCredential, letting the STS response redirect the S3 client to a non-AWS backend.
  • When storageProvider == OSS, disable AWS integrity-check headers and switch to virtual-hosted addressing; preserves existing path-style behavior for other S3-compatible stores (s3proxy, MinIO) via Env.current.s3_endpoint_url.
  • Apply user_credential.endpoint to the boto3 client. Env.current.s3_endpoint_url remains the highest-priority override (manual user setting).

Cherry-pick

  • Source: main @ 45677bd
  • Conflicts: none

Test plan

  • Default AWS flow (no storageProvider, no endpoint) still builds client unchanged.
  • storageProvider=OSS path: virtual-hosted addressing + checksums disabled.
  • S3-compatible dev path (Env.current.s3_endpoint_url set, no storageProvider): path-style + checksums disabled.
  • user_credential.endpoint set: client uses that endpoint_url unless env override is also set.

🤖 Generated with Claude Code


Note

Medium Risk
Touches S3 client construction (endpoint selection, addressing style, checksum behavior), which can impact all upload/download flows if provider detection or endpoint precedence is wrong.

Overview
Adds endpoint and storageProvider fields to the STS-derived _UserCredential so backend-issued credentials can direct the client to a specific S3-compatible endpoint.

Updates _S3STSToken.get_client() to apply the credential-scoped endpoint_url (while keeping Env.current.s3_endpoint_url as the highest-priority override) and to special-case storageProvider==OSS by disabling checksum validation and using virtual-hosted addressing; other non-AWS endpoints continue using path-style addressing.

Reviewed by Cursor Bugbot for commit bcf46e1. Bugbot is set up for automated code reviews on this repo. Configure here.

…#2019)

Co-authored-by: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
@benflexcompute benflexcompute enabled auto-merge (squash) April 24, 2026 20:31
@benflexcompute benflexcompute merged commit d7760da into release-candidate/25.9 Apr 24, 2026
18 checks passed
@benflexcompute benflexcompute deleted the ben/hotfix-25.9-oss-credential-endpoint branch April 24, 2026 20:31
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants