[fs] Upgrade filesystem Hadoop version to 3.4.2 #6290
Merged
Conversation
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
PatrickRen
force-pushed
the
support-azure-workload-identity
branch
3 times, most recently
from
6222f28 to
80de5e5
Compare
…ntity authentication
PatrickRen
force-pushed
the
support-azure-workload-identity
branch
from
fa12bef to
78b405b
Compare
PatrickRen
force-pushed
the
support-azure-workload-identity
branch
from
2558688 to
630a298
Compare
PatrickRen
changed the title
JingsongLi
approved these changes
Sep 30, 2025
Contributor
JingsongLi
left a comment
JingsongLi
left a comment
There was a problem hiding this comment.
Thanks @PatrickRen , looks good to me!
XiaoHongbo-Hope
pushed a commit
to XiaoHongbo-Hope/paimon
that referenced
this pull request
Oct 10, 2025
This reverts commit 7962e22.
JingsongLi
added a commit
to JingsongLi/paimon
that referenced
this pull request
Oct 10, 2025
This reverts commit 7962e22.
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
2 participants
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
Purpose
Linked issue: close #6284
This pull request updates the Azure Blob Storage integration and improves object store detection in the codebase. The most important changes are grouped below:
Dependency updates:
paimon-filesystems/pom.xml, which adds supports for Azure Workload Identity authentication in Hadoop's Azure Filesystem.Azure Blob Storage integration:
requiredOptionsmethod from theAzureLoaderclass. The original implementation checks if account name and key are set, but this will block usages of other auth mechanisms such as OAuth.Object store detection improvements:
FlinkFileIOto recognize theabfsscheme as an object store, ensuring proper handling of Azure Blob Storage.Tests
Unfortunately this pull request doesn't contain any test coverage, as it requires Azure resources for testing, and Azure simulator Azurite doesn't support ABFS.
However I tested it with my own Azure account with OAuth authentication.
API and Format
No.
Documentation
No.