[fix](iceberg) fix doris can't select from iceberg on bos (#65170)#65171
Open
WeiPrince wants to merge 1 commit into
Open
[fix](iceberg) fix doris can't select from iceberg on bos (#65170)#65171WeiPrince wants to merge 1 commit into
WeiPrince wants to merge 1 commit into
Conversation
WeiPrince
requested review from
CalvinKirs,
gavinchou,
liaoxin01 and
morningman
as code owners
Contributor
|
Thank you for your contribution to Apache Doris. Please clearly describe your PR:
|
yiguolei
approved these changes
Jul 7, 2026
Contributor
|
/review |
github-actions
Bot
added
the
approved
Contributor
|
PR approved by at least one committer and no changes requested. |
Contributor
|
PR approved by anyone and no changes requested. |
Contributor
|
run buildall |
![github-actions[bot]](https://avatars.githubusercontent.com/in/15368?s=60&v=4){kind=small}
Contributor
There was a problem hiding this comment.
Automated review conclusion: request changes.
Critical checkpoint conclusions:
- Goal and tests: the PR maps
bos://to S3/FILE_S3so the normal Iceberg file path can be scanned through the existing S3-compatible handling. The same end-to-end BOS Iceberg catalog workflow is not covered by a focused test. - Scope: the code change is small and clear, but it misses a parallel catalog-creation connectivity path.
- Concurrency and lifecycle: no new concurrency, locks, lifecycle ownership, or static initialization risk found in the changed code.
- Configuration and compatibility: no new config is added. Existing
test_connection=truebehavior remains incompatible withbos://Iceberg warehouses, which is the blocking finding. - Parallel paths: normal
LocationPathscan/sink routing and S3-compatible normalization were checked; catalog connectivity validation still uses separate scheme checks. - Testing: existing generic mapper and S3 normalization tests partially cover the mapping/normalization helpers, but there is no
bos://Iceberg warehousetest_connection=truecoverage. - Persistence, transactions, writes, observability, and performance: not applicable or no additional concern found for this one-line mapper change.
Subagent conclusions:
optimizer-rewrite: no optimizer/rewrite, join, aggregate, or Nereids semantic-equivalence finding.tests-session-config:TEST-1was accepted asMAIN-1and submitted as the inline comment.- Convergence round 1 ended with both live subagents reporting
NO_NEW_VALUABLE_FINDINGSfor the final ledger/comment set.
User focus: no additional user-provided focus points were supplied.
Validation notes: static review only in this runner. .worktree_initialized, thirdparty/installed, and thirdparty/installed/bin/protoc are missing, so I did not run FE build/unit tests locally. I verified the live PR head remains 524196b0bfa15b4e4c489fd2303b9d2f74bf3731, the GitHub PR file list still contains only SchemaTypeMapper.java, and there were no existing inline review comments to deduplicate against.
| HTTP("http", StorageProperties.Type.HTTP, FileSystemType.HTTP, TFileType.FILE_HTTP), | ||
| HTTPS("https", StorageProperties.Type.HTTP, FileSystemType.HTTP, TFileType.FILE_HTTP); | ||
| HTTPS("https", StorageProperties.Type.HTTP, FileSystemType.HTTP, TFileType.FILE_HTTP), | ||
| BOS("bos", StorageProperties.Type.S3, FileSystemType.S3, TFileType.FILE_S3); |
Contributor
There was a problem hiding this comment.
This makes the normal LocationPath/Iceberg scan path treat bos:// as S3, but the test_connection=true catalog-creation path still has its own scheme checks. ExternalCatalog.checkWhenCreating() runs CatalogConnectivityTestCoordinator; for Iceberg Glue, getTestLocation() rejects any warehouse that does not match ^(s3|s3a|s3n)://, so a bos://... Glue warehouse still fails before users reach the fixed scan path. For HMS/REST, the base tester can return the configured bos:// warehouse, but findMatchingObjectStorage() only matches s3:///s3a://, so the S3-compatible BOS storage test is skipped. Please route this path through the same SchemaTypeMapper/LocationPath normalization or add bos to the connectivity validation/matching logic, and cover a bos:// Iceberg warehouse with test_connection=true.
Contributor
TPC-H: Total hot run time: 29765 ms |
Contributor
TPC-DS: Total hot run time: 179902 ms |
Contributor
ClickBench: Total hot run time: 25.18 s |
Contributor
FE UT Coverage ReportIncrement line coverage |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
3 participants
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
What problem does this PR solve?
Issue Number: close #65170
Related PR: #xxx
Problem Summary:
Release note
None
Check List (For Author)
Test
Behavior changed:
Does this need documentation?
Check List (For Reviewer who merge this PR)