Skip to content

[CELEBORN-1899] Fix configuration bug in shuffle s3#3146

Closed
shouwangyw wants to merge 2 commits intoapache:mainfrom
shouwangyw:bugfix/resolve_bugs_3145
Closed

[CELEBORN-1899] Fix configuration bug in shuffle s3#3146
shouwangyw wants to merge 2 commits intoapache:mainfrom
shouwangyw:bugfix/resolve_bugs_3145

Conversation

@shouwangyw
Copy link
Contributor

What changes were proposed in this pull request?

close issues-3145

Why are the changes needed?

  1. Support s3 schema.
  2. Fixed the problem that diskFileInfo judged the S3 type incorrectly.

Does this PR introduce any user-facing change?

No

How was this patch tested?

Deployment integration testing has been completed in the local environment.

@FMX
Copy link
Contributor

FMX commented Mar 11, 2025

@shouwangyw Hi, you can run ./build/sbt test in your local environment to ensure that the UTs are correct.

@SteNicholas
Copy link
Member

@shouwangyw, please create new ticket for this pull request.

@FMX FMX changed the title [CELEBORN-3145] Fix configuration bug in shuffle s3 [CELEBORN-1899] Fix configuration bug in shuffle s3 Mar 12, 2025
Copy link
Contributor

@FMX FMX left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM. Thanks.

@FMX
Copy link
Contributor

FMX commented Mar 12, 2025

@shouwangyw, please create new ticket for this pull request.

I've fixed the incorrect issue problem here.

@FMX FMX closed this in 464a384 Mar 12, 2025
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

[BUG] Errors When Configuring celeborn.storage.s3.dir with s3:// and s3a:// Schemes

3 participants