[fix](be) fix BE core due to explode_bitmap size exceeding INT_MAX#66034
Open
Baymine wants to merge 1 commit into
Open
[fix](be) fix BE core due to explode_bitmap size exceeding INT_MAX#66034Baymine wants to merge 1 commit into
Baymine wants to merge 1 commit into
Conversation
Contributor
|
Thank you for your contribution to Apache Doris. Please clearly describe your PR:
|
3 tasks
## Proposed changes Issue Number: close apache#66033 Problem Summary: `VExplodeBitmapTableFunction::get_value` computed the number of rows to emit with `max_step = std::min(max_step, (int)(_cur_size - _cur_offset));`. Both `_cur_size` (the bitmap cardinality) and `_cur_offset` are `int64_t`. When a bitmap holds more than `INT_MAX` (~2.1 billion) elements, the subtraction `_cur_size - _cur_offset` exceeds the range of `int`, so the C-style `(int)` cast overflows and yields a negative `max_step`. That negative value is later used in `target->resize(origin_size + max_step)`, which underflows the size computation and crashes the BE (core dump). The fix performs the `std::min` in `int64_t` space so the comparison is done without truncation, then casts the result — which is now provably within `[0, max_step]` and therefore in `int` range — back to `int`. A `DCHECK_GE(max_step, 0)` guard documents and asserts the post-condition in debug builds, matching the glog `DCHECK` idiom already used by the sibling table_function sources. ### Release note Fix BE crash when exploding a bitmap whose cardinality exceeds INT_MAX. ### Check List (For Author) - Test: Unit Test — added be/test/exprs/function/table_function_test.cpp:vexplode_bitmap_cardinality_exceeds_int_max, which builds a bitmap with cardinality > INT_MAX (via a Roaring range, ~KB) and asserts get_value returns a positive batch; this reproduces the pre-fix negative-overflow crash. - Behavior changed: No (crash avoidance only; correct results are unchanged). - Does this need documentation: No
Baymine
force-pushed
the
fix/explode-bitmap-int-max
branch
from
July 25, 2026 04:52
e6a1861 to
d3a872e
Compare
Contributor
Author
|
/review |
Contributor
Author
|
run buildall |
Contributor
TPC-H: Total hot run time: 29358 ms |
Contributor
TPC-DS: Total hot run time: 176812 ms |
Contributor
ClickBench: Total hot run time: 25.1 s |
Contributor
BE Regression && UT Coverage ReportIncrement line coverage Increment coverage report
|
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
2 participants
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
What problem does this PR solve?
Issue Number: close #66033
Problem Summary:
VExplodeBitmapTableFunction::get_valuecomputed the number of rows to emit withmax_step = std::min(max_step, (int)(_cur_size - _cur_offset));. Both_cur_size(the bitmap cardinality) and_cur_offsetareint64_t. When a bitmap holds more thanINT_MAX(~2.1 billion) elements, the subtraction_cur_size - _cur_offsetexceeds the range ofint, so the C-style(int)cast overflows and yields a negativemax_step. That negative value is later used intarget->resize(origin_size + max_step), which underflows the size computation and crashes the BE (core dump).The fix performs the
std::mininint64_tspace so the comparison is done without truncation, then casts the result — now provably within[0, max_step]and therefore inintrange — back toint. ADCHECK_GE(max_step, 0)guard documents and asserts the post-condition, matching the glogDCHECKidiom already used by the sibling table_function sources.Release note
Fix BE crash when exploding a bitmap whose cardinality exceeds INT_MAX.
Check List (For Author)
Test
be/test/exprs/function/table_function_test.cpp:vexplode_bitmap_cardinality_exceeds_int_max. It cheaply builds a bitmap whose cardinality exceeds INT_MAX from a Roaring range (a few KB — the overflow occurs on the firstget_valuecall, before any element is materialized, so no large allocation is needed) and assertsget_valuereturns a positive batch. On the pre-fix code the(int)cast overflows to a negativemax_stepand crashes; the test reproduces this and passes on the fix (verified under ASAN).Behavior changed:
Does this need documentation?