[refactor](fe) Phase 0: Prerequisite decoupling for filesystem SPI#61862
Merged
morningman merged 3 commits intoapache:branch-fs-spifrom Mar 30, 2026
Merged
[refactor](fe) Phase 0: Prerequisite decoupling for filesystem SPI#61862morningman merged 3 commits intoapache:branch-fs-spifrom
morningman merged 3 commits intoapache:branch-fs-spifrom
Conversation
### What problem does this PR solve? Issue Number: N/A Problem Summary: Before splitting filesystem implementations into independent Maven modules (Phase 3), several compile-time couplings must be eliminated. This commit completes all Phase 0 prerequisite decoupling tasks: - P0.1: Introduce FsStorageType enum in fe-foundation (zero-dep module) to replace StorageBackend.StorageType (Thrift-generated) in PersistentFileSystem. Add FsStorageTypeAdapter for bidirectional Thrift conversion. Update all subclasses and callers (Repository, BackupJob, RestoreJob, CloudRestoreJob). - P0.2: Add IOException-based default bridge methods to ObjStorage interface (checkObjectExists, getObjectChecked, putObjectChecked, deleteObjectChecked, deleteObjectsChecked, copyObjectChecked, listObjectsChecked). Add ObjStorageStatusAdapter for Status→IOException conversion. Zero changes to existing implementations. - P0.3: Decouple SwitchingFileSystem from ExternalMetaCacheMgr via new FileSystemLookup functional interface. FileSystemProviderImpl passes a lambda. - P0.4: Extract MultipartUploadCapable interface from ObjFileSystem, removing the forced abstract method. S3FileSystem and AzureFileSystem implement it. HMSTransaction now uses instanceof check instead of ObjFileSystem cast. - P0.5: Introduce FileSystemDescriptor POJO for Repository metadata serialization, replacing direct PersistentFileSystem subclass serialization. Migrate GsonUtils to string-based Class.forName() reflection for legacy format backward compat, removing 7 compile-time imports of concrete filesystem classes. - P0.6: Add FileSystemSpiProvider interface skeleton in fs/spi/ as the future ServiceLoader contract for Phase 3 module split. ### Release note None ### Check List (For Author) - Test: No need to test (pure refactor; all changes are backward compatible; three successful FE builds verified during development) - Behavior changed: No - Does this need documentation: No Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
Maven build cache was caching checkstyle:check results and emitting 'Skipping plugin execution (cached)' even when sources had changed. Two fixes: 1. Add check/checkstyle/ to global cache input so that changes to checkstyle rules (checkstyle.xml, suppressions.xml, etc.) correctly invalidate all module caches. 2. Mark the checkstyle:check execution (id: validate) as runAlways in executionControl so it is never skipped regardless of cache state. Checkstyle is a quality gate and must always execute. Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
- ObjFileSystem: remove unused Map import (was used by the now-removed abstract completeMultipartUpload method) - GsonUtils: fix CustomImportOrder violations - LogManager/Logger imports were inserted before com.google.* imports; move them after all com.* imports in correct lexicographical order Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
morningman
requested review from
CalvinKirs,
dataroaring,
gavinchou and
w41ter
as code owners
Contributor
|
Thank you for your contribution to Apache Doris. Please clearly describe your PR:
|
Contributor
Author
|
run buildall |
Open
26 tasks
Contributor
Author
|
Run test in #61859 |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
2 participants
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
This PR completes Phase 0 of the FE filesystem SPI refactoring — removing compile-time couplings that would otherwise prevent splitting filesystem implementations into independent Maven modules in later phases.
Changes
P0.1 — FsStorageType enum migration
FsStorageTypeenum infe-foundation(zero-dependency module) to replace Thrift-generatedStorageBackend.StorageTypeinPersistentFileSystemFsStorageTypeAdapterinfe-corefor bidirectional Thrift↔FsStorageType conversionRepository,BackupJob,RestoreJob,CloudRestoreJobP0.2 — ObjStorage IOException bridge
IOException-baseddefaultbridge methods toObjStorageinterfaceObjStorageStatusAdapterforStatus→IOExceptionconversionP0.3 — SwitchingFileSystem decoupling
FileSystemLookupfunctional interfaceSwitchingFileSystemfromExternalMetaCacheMgrP0.4 — MultipartUploadCapable interface
MultipartUploadCapableinterface fromObjFileSystemS3FileSystemandAzureFileSystemimplement it;HMSTransactionusesinstanceofcheckP0.5 — GsonUtils compile-time decoupling
FileSystemDescriptorPOJO forRepositorymetadata serializationGsonUtilsremoves 7 compile-time concrete class imports, usesClass.forName()reflectionP0.6 — FileSystemSpiProvider skeleton
FileSystemSpiProviderinterface infs/spi/Build
checkstyle:checkTesting
Closes part of #61860