[fix](fe) Report Iceberg scan metrics in batch mode - #66521
Conversation
### What problem does this PR solve?
Issue Number: None
Related PR: None
Problem Summary: Iceberg eager scans attach an SDK metrics reporter and drain the resulting scan profile immediately after planning. Streaming batch scans skipped both steps, so their query profiles showed no Iceberg scan metrics even though the same files and manifests were planned. Attach the existing reporter to streaming scans, drain it after the split source closes using the query-scoped execution summary captured before asynchronous dispatch, and serialize profile-group creation across concurrent streaming scans.
### Release note
Iceberg batch-mode queries now report the same scan metrics as non-batch queries.
### Check List (For Author)
- Test: Unit Test
- IcebergScanPlanProviderTest
- PluginDrivenScanNodeScanProfileTest
- Behavior changed: Yes. Iceberg batch-mode query profiles now include scan metrics.
- Does this need documentation: No
|
Thank you for your contribution to Apache Doris. Please clearly describe your PR:
|
|
run buildall |
FE Regression Coverage ReportIncrement line coverage |
|
/review |
|
Codex automated review failed and did not complete. Error: All Codex review accounts are usage-limited; earliest retry is 2026-08-08T03:32:00Z. Please trigger /review again after that time. |
FE Regression Coverage ReportIncrement line coverage |
3 similar comments
FE Regression Coverage ReportIncrement line coverage |
FE Regression Coverage ReportIncrement line coverage |
FE Regression Coverage ReportIncrement line coverage |
What problem does this PR solve?
Issue Number: None
Related PR: None
Problem Summary:
Iceberg eager scans attach an SDK metrics reporter and drain the resulting scan profile immediately after planning. Streaming batch scans skipped both steps, so their query profiles showed no Iceberg scan metrics even though the same files and manifests were planned.
This change attaches the existing reporter to streaming scans and drains it after the split source closes. Because streaming split generation runs asynchronously, the query execution summary is captured before dispatch. Profile-group creation is also serialized so concurrent streaming scan nodes cannot replace one another's metric groups.
Release note
Iceberg batch-mode queries now report the same scan metrics as non-batch queries.
Check List (For Author)
Validation performed:
IcebergScanPlanProviderTest: 129 tests passed.
PluginDrivenScanNodeScanProfileTest: 4 tests passed.
Connector SPI test suite: 140 tests passed.
Full FE build:
./build.sh --fesucceeded.Behavior changed:
Does this need documentation?
Check List (For Reviewer who merge this PR)