[fix](scanner) Make Scanner close() method thread-safe using atomic operations #57436
Conversation
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Contributor
|
Thank you for your contribution to Apache Doris. Please clearly describe your PR:
|
Contributor
Author
|
run buildall |
BE UT Coverage ReportIncrement line coverage Increment coverage report
|
Contributor
BE Regression && UT Coverage ReportIncrement line coverage Increment coverage report
|
github-actions
bot
added
the
approved
Contributor
|
PR approved by at least one committer and no changes requested. |
Contributor
|
PR approved by anyone and no changes requested. |
Contributor
|
run performance |
There was a problem hiding this comment.
Pull Request Overview
This PR makes the _is_closed flag in the Scanner class thread-safe by converting it from a plain bool to std::atomic<bool>, and updates all close() methods in scanner implementations to use atomic compare-and-swap operations instead of simple boolean checks.
- Changed
_is_closedfrombooltostd::atomic<bool>in theScannerbase class - Updated all
close()implementations to usecompare_exchange_strongfor thread-safe double-close prevention - Fixed a typo in a comment from "dctor" to "destructor"
Reviewed Changes
Copilot reviewed 8 out of 8 changed files in this pull request and generated 2 comments.
Show a summary per file
| File | Description |
|---|---|
| be/src/vec/exec/scan/scanner.h | Changed _is_closed to std::atomic<bool> and added <atomic> include |
| be/src/vec/exec/scan/scanner.cpp | Updated base Scanner::close() to use atomic compare-exchange |
| be/src/vec/exec/scan/scan_node.h | Fixed typo in comment (dctor → destructor) |
| be/src/vec/exec/scan/olap_scanner.cpp | Updated OlapScanner::close() to use atomic compare-exchange |
| be/src/vec/exec/scan/meta_scanner.cpp | Updated MetaScanner::close() to use atomic compare-exchange |
| be/src/vec/exec/scan/jdbc_scanner.cpp | Updated JdbcScanner::close() to use atomic compare-exchange |
| be/src/vec/exec/scan/file_scanner.cpp | Updated FileScanner::close() to use atomic compare-exchange |
| be/src/vec/exec/scan/es_scanner.cpp | Updated EsScanner::close() to use atomic compare-exchange |
Comments suppressed due to low confidence (3)
be/src/vec/exec/scan/meta_scanner.cpp:531
- Double close can still occur: after checking
_is_closedand before callingScanner::close(state), the parent class'sclose()is called at line 531 which will perform its own compare-exchange on_is_closed. Since this method already set_is_closedto true at line 525, the parent's compare-exchange will always fail (expected=false won't match _is_closed=true), making line 531 effectively a no-op. However, if_reader->close()fails and returns an error,_is_closedwill remain true butScanner::close()will never be called, potentially leaking resources. The parent'sclose()should be called unconditionally or the logic should be restructured.
bool expected = false;
if (!_is_closed.compare_exchange_strong(expected, true)) {
return Status::OK();
}
if (_reader) {
RETURN_IF_ERROR(_reader->close());
}
RETURN_IF_ERROR(Scanner::close(state));
be/src/vec/exec/scan/file_scanner.cpp:1726
- Double close protection is flawed: after setting
_is_closedto true at line 1718, the parent classScanner::close(state)is called at line 1726, which performs its own compare-exchange. Since_is_closedis already true, the parent's compare-exchange will always fail, making line 1726 a no-op. Additionally, if_cur_reader->close()fails at line 1723,_is_closedremains true but the parent close is never executed, potentially leaking resources. The parent's close should be called unconditionally.
bool expected = false;
if (!_is_closed.compare_exchange_strong(expected, true)) {
return Status::OK();
}
if (_cur_reader) {
RETURN_IF_ERROR(_cur_reader->close());
}
RETURN_IF_ERROR(Scanner::close(state));
be/src/vec/exec/scan/es_scanner.cpp:203
- Double close protection is flawed: after setting
_is_closedto true at line 195, the parent classScanner::close(state)is called at line 203, which performs its own compare-exchange. Since_is_closedis already true, the parent's compare-exchange will always fail, making line 203 a no-op. Additionally, if_es_reader->close()fails at line 200,_is_closedremains true but the parent close is never executed, potentially leaking resources. The parent's close should be called unconditionally.
bool expected = false;
if (!_is_closed.compare_exchange_strong(expected, true)) {
return Status::OK();
}
if (_es_reader != nullptr) {
RETURN_IF_ERROR(_es_reader->close());
}
RETURN_IF_ERROR(Scanner::close(state));
💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.
ClickBench: Total hot run time: 27.81 s |
Contributor
Author
|
run buildall |
github-actions
bot
removed
the
approved
TPC-DS: Total hot run time: 190348 ms |
ClickBench: Total hot run time: 28.19 s |
Contributor
BE UT Coverage ReportIncrement line coverage Increment coverage report
|
Contributor
Author
|
run external |
morningman
approved these changes
Oct 30, 2025
github-actions
bot
added
the
approved
Contributor
|
PR approved by at least one committer and no changes requested. |
Contributor
BE Regression && UT Coverage ReportIncrement line coverage Increment coverage report
|
CalvinKirs
approved these changes
Oct 31, 2025
github-actions bot
pushed a commit
that referenced
this pull request
Oct 31, 2025
…perations (#57436) ### What problem does this PR solve? close: #56328 Problem Summary: The current close() method in Scanner class and its derived classes has thread safety issues in multi-threaded environments. Multiple threads may call close() simultaneously, leading to: 1. Race conditions: Multiple threads may check _is_closed status and execute close operations simultaneously 2. Double close: May cause resources to be released or cleaned up multiple times 3. Potential memory leaks or crash risks
suxiaogang223
added a commit
to suxiaogang223/doris
that referenced
this pull request
Nov 3, 2025
…perations (apache#57436) close: apache#56328 Problem Summary: The current close() method in Scanner class and its derived classes has thread safety issues in multi-threaded environments. Multiple threads may call close() simultaneously, leading to: 1. Race conditions: Multiple threads may check _is_closed status and execute close operations simultaneously 2. Double close: May cause resources to be released or cleaned up multiple times 3. Potential memory leaks or crash risks
suxiaogang223
added a commit
to suxiaogang223/doris
that referenced
this pull request
Jan 6, 2026
…perations (apache#57436) ### What problem does this PR solve? close: apache#56328 Problem Summary: The current close() method in Scanner class and its derived classes has thread safety issues in multi-threaded environments. Multiple threads may call close() simultaneously, leading to: 1. Race conditions: Multiple threads may check _is_closed status and execute close operations simultaneously 2. Double close: May cause resources to be released or cleaned up multiple times 3. Potential memory leaks or crash risks
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
6 participants
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
What problem does this PR solve?
close: #56328
Problem Summary:
The current close() method in Scanner class and its derived classes has thread safety issues in multi-threaded environments. Multiple threads may call close() simultaneously, leading to:
Release note
None
Check List (For Author)
Test
Behavior changed:
Does this need documentation?
Check List (For Reviewer who merge this PR)