branch-3.0:[fix](core) normalize DeleteSubPredicatePB operator on read path #62052 by felixwluo · Pull Request #61961 · apache/doris

felixwluo · 2026-03-31T12:23:24Z

Problem Summary:

pick: #62052

compaction core

#0  0x0000000000000000 in ?? ()
#1  0x00005561c333fa74 in doris::parse_to_predicate (column=..., index=6, condition=..., arena=0x7ff95ae1b710, opposite=true)
    at /root/be/src/olap/predicate_creator.h:329
#2  0x00005561c333f351 in doris::DeleteHandler::_parse_column_pred<doris::DeleteSubPredicatePB> (this=this@entry=0x7ff89bc0bb08, 
    complete_schema=..., delete_pred_related_schema=..., sub_pred_list=..., delete_conditions=delete_conditions@entry=0x7ff89bc0b330)
    at /root/be/src/olap/delete_handler.cpp:375
#3  0x00005561c333c57f in doris::DeleteHandler::init (this=0x7ff89bc0bb08, tablet_schema=..., delete_preds=..., version=15)
    at /root/be/src/olap/delete_handler.cpp:410
#4  0x00005561c3c504d7 in doris::TabletReader::_init_delete_condition (this=this@entry=0x7ff89bc0b848, read_params=...)
    at /root/be/src/olap/tablet_reader.cpp:637
#5  0x00005561c3c4e1fb in doris::TabletReader::_init_params (this=this@entry=0x7ff89bc0b848, read_params=...)
    at /root/be/src/olap/tablet_reader.cpp:297
#6  0x00005561c3c4dda6 in doris::TabletReader::init (this=0x7ff89bc0b848, read_params=...) at /root/be/src/olap/tablet_reader.cpp:126
#7  0x00005561cd62a27a in doris::vectorized::VerticalBlockReader::init (this=0x7ff89bc0b848, read_params=..., sample_info=0x7ff89bc0c5e0)
    at /root/be/src/vec/olap/vertical_block_reader.cpp:224
#8  0x00005561c3ac6f0f in doris::Merger::vertical_compact_one_group (tablet=..., 
    reader_type=reader_type@entry=doris::ReaderType::READER_BASE_COMPACTION, tablet_schema=..., is_key=true, column_group=..., 
    row_source_buf=0x7ff89bc0c590, src_rowset_readers=..., dst_rowset_writer=0x7ff928685500, max_rows_per_segment=460833, 
    stats_output=0x7ff89bc0cb70, key_group_cluster_key_idxes=..., batch_size=992, sample_info=0x7ff89bc0c5e0) at /root/be/src/olap/merger.cpp:252
#9  0x00005561c3ac8e02 in doris::Merger::vertical_merge_rowsets (tablet=..., reader_type=<optimized out>, tablet_schema=..., 
    src_rowset_readers=..., dst_rowset_writer=0x7ff928685500, max_rows_per_segment=460833, merge_way_num=1, stats_output=0x7ff89bc0cb70)
    at /root/be/src/olap/merger.cpp:427
#10 0x00005561c3aacade in doris::Compaction::do_compaction_impl (this=this@entry=0x7ff92e8a9290, permits=permits@entry=3)
    at /root/be/src/olap/compaction.cpp:392
#11 0x00005561c3aabcac in doris::Compaction::do_compaction (this=0x7ff92e8a9290, permits=3) at /root/be/src/olap/compaction.cpp:140
#12 0x00005561c3c07d92 in doris::BaseCompaction::execute_compact_impl (this=0x7ff92e8a9290) at /root/be/src/olap/base_compaction.cpp:79
#13 0x00005561c3aab8d3 in doris::Compaction::execute_compact (this=0x6) at /root/be/src/olap/compaction.cpp:122
#14 0x00005561c3bddf1b in doris::Tablet::execute_compaction(doris::Compaction&)::$_0::operator()() const (this=<optimized out>)
    at /root/be/src/olap/tablet.cpp:2093
#15 doris::Tablet::execute_compaction (this=0x7ff8e54f5610, compaction=...) at /root/be/src/olap/tablet.cpp:2093
#16 0x00005561c3a7e336 in doris::StorageEngine::_submit_compaction_task(std::shared_ptr<doris::Tablet>, doris::CompactionType, bool)::$_0::operator()() const (this=0x7ff92bc351c0) at /root/be/src/olap/olap_server.cpp:1068
#17 std::__invoke_impl<void, doris::StorageEngine::_submit_compaction_task(std::shared_ptr<doris::Tablet>, doris::CompactionType, bool)::$_0&>(std::__invoke_other, doris::StorageEngine::_submit_compaction_task(std::shared_ptr<doris::Tablet>, doris::CompactionType, bool)::$_0&) (__f=...)
    at /var/local/ldb-toolchain/bin/../lib/gcc/x86_64-linux-gnu/11/../../../../include/c++/11/bits/invoke.h:61
#18 std::__invoke_r<void, doris::StorageEngine::_submit_compaction_task(std::shared_ptr<doris::Tablet>, doris::CompactionType, bool)::$_0&>(doris::StorageEngine::_submit_compaction_task(std::shared_ptr<doris::Tablet>, doris::CompactionType, bool)::$_0&) (__fn=...)
    at /var/local/ldb-toolchain/bin/../lib/gcc/x86_64-linux-gnu/11/../../../../include/c++/11/bits/invoke.h:111
#19 std::_Function_handler<void (), doris::StorageEngine::_submit_compaction_task(std::shared_ptr<doris::Tablet>, doris::CompactionType, bool)::$_0>::_M_invoke(std::_Any_data const&) (__functor=...)
    at /var/local/ldb-toolchain/bin/../lib/gcc/x86_64-linux-gnu/11/../../../../include/c++/11/bits/std_function.h:291
#20 0x00005561c4109fb8 in doris::ThreadPool::dispatch_thread (this=0x7ff92b66da00) at /root/be/src/util/threadpool.cpp:544
#21 0x00005561c40ff341 in std::function<void ()>::operator()() const (this=0x7ff9e3e65e10)
    at /var/local/ldb-toolchain/bin/../lib/gcc/x86_64-linux-gnu/11/../../../../include/c++/11/bits/std_function.h:560
#22 doris::Thread::supervise_thread (arg=0x7ff9dd49b860) at /root/be/src/util/thread.cpp:498
#23 0x00007ffa0a7a2215 in start_thread () from /lib64/libc.so.6
#24 0x00007ffa0a824bdc in clone3 () from /lib64/libc.so.6

query core

#0  0x0000000000000000 in ?? ()
#1  0x000055c0b109da74 in doris::parse_to_predicate (column=..., index=6, condition=..., arena=0x7f8357530270, opposite=true)
    at /root/be/src/olap/predicate_creator.h:329
#2  0x000055c0b109d351 in doris::DeleteHandler::_parse_column_pred<doris::DeleteSubPredicatePB> (this=this@entry=0x7f837ab178c0, 
    complete_schema=..., delete_pred_related_schema=..., sub_pred_list=..., delete_conditions=delete_conditions@entry=0x7f7f53dc5b00)
    at /root/be/src/olap/delete_handler.cpp:375
#3  0x000055c0b109a57f in doris::DeleteHandler::init (this=0x7f837ab178c0, tablet_schema=..., delete_preds=..., version=16)
    at /root/be/src/olap/delete_handler.cpp:410
#4  0x000055c0b19ae4d7 in doris::TabletReader::_init_delete_condition (this=this@entry=0x7f837ab17600, read_params=...)
    at /root/be/src/olap/tablet_reader.cpp:637
#5  0x000055c0b19ac1fb in doris::TabletReader::_init_params (this=this@entry=0x7f837ab17600, read_params=...)
    at /root/be/src/olap/tablet_reader.cpp:297
#6  0x000055c0b19abda6 in doris::TabletReader::init (this=0x7f837ab17600, read_params=...) at /root/be/src/olap/tablet_reader.cpp:126
#7  0x000055c0bb36da9f in doris::vectorized::BlockReader::init (this=0x6, read_params=...) at /root/be/src/vec/olap/block_reader.cpp:201
#8  0x000055c0b61ddffc in doris::vectorized::NewOlapScanner::open (this=0x7f7fac09a810, state=<optimized out>)
    at /root/be/src/vec/exec/scan/new_olap_scanner.cpp:228
#9  0x000055c0b61eb5e0 in doris::vectorized::ScannerScheduler::_scanner_scan (ctx=..., scan_task=...)
    at /root/be/src/vec/exec/scan/scanner_scheduler.cpp:249
#10 0x000055c0b61ec2cd in doris::vectorized::ScannerScheduler::submit(std::shared_ptr<doris::vectorized::ScannerContext>, std::shared_ptr<doris::vectorized::ScanTask>)::$_1::operator()() const::{lambda()#1}::operator()() const::{lambda()#1}::operator()() const (this=<optimized out>)
    at /root/be/src/vec/exec/scan/scanner_scheduler.cpp:170
#11 doris::vectorized::ScannerScheduler::submit(std::shared_ptr<doris::vectorized::ScannerContext>, std::shared_ptr<doris::vectorized::ScanTask>)::$_1::operator()() const::{lambda()#1}::operator()() const (this=0x7f802ccdef20) at /root/be/src/vec/exec/scan/scanner_scheduler.cpp:169
#12 std::__invoke_impl<void, doris::vectorized::ScannerScheduler::submit(std::shared_ptr<doris::vectorized::ScannerContext>, std::shared_ptr<doris::vectorized::ScanTask>)::$_1::operator()() const::{lambda()#1}&>(std::__invoke_other, doris::vectorized::ScannerScheduler::submit(std::shared_ptr<doris::vectorized::ScannerContext>, std::shared_ptr<doris::vectorized::ScanTask>)::$_1::operator()() const::{lambda()#1}&) (__f=...)
    at /var/local/ldb-toolchain/bin/../lib/gcc/x86_64-linux-gnu/11/../../../../include/c++/11/bits/invoke.h:61
#13 std::__invoke_r<void, doris::vectorized::ScannerScheduler::submit(std::shared_ptr<doris::vectorized::ScannerContext>, std::shared_ptr<doris::vectorized::ScanTask>)::$_1::operator()() const::{lambda()#1}&>(doris::vectorized::ScannerScheduler::submit(std::shared_ptr<doris::vectorized::ScannerContext>, std::shared_ptr<doris::vectorized::ScanTask>)::$_1::operator()() const::{lambda()#1}&) (__fn=...)
    at /var/local/ldb-toolchain/bin/../lib/gcc/x86_64-linux-gnu/11/../../../../include/c++/11/bits/invoke.h:111
#14 std::_Function_handler<void (), doris::vectorized::ScannerScheduler::submit(std::shared_ptr<doris::vectorized::ScannerContext>, std::shared_ptr<doris::vectorized::ScanTask>)::$_1::operator()() const::{lambda()#1}>::_M_invoke(std::_Any_data const&) (__functor=...)
    at /var/local/ldb-toolchain/bin/../lib/gcc/x86_64-linux-gnu/11/../../../../include/c++/11/bits/std_function.h:291
#15 0x000055c0b1e67fb8 in doris::ThreadPool::dispatch_thread (this=0x7f8368f68200) at /root/be/src/util/threadpool.cpp:544
#16 0x000055c0b1e5d341 in std::function<void ()>::operator()() const (this=0x7f837ab06f90)
    at /var/local/ldb-toolchain/bin/../lib/gcc/x86_64-linux-gnu/11/../../../../include/c++/11/bits/std_function.h:560
#17 doris::Thread::supervise_thread (arg=0x7f8247529280) at /root/be/src/util/thread.cpp:498
#18 0x00007f837d5a2215 in start_thread () from /lib64/libc.so.6
#19 0x00007f837d624bdc in clone3 () from /lib64/libc.so.6

Summary

This PR fixes a BE coredump caused by legacy DeleteSubPredicatePB.op encoding during delete predicate parsing
The issue affects both query scan and compaction paths and is rooted in inconsistent operator encoding in historical rowset metadata

Root Cause

DeleteSubPredicatePB read path did not normalize op before predicate creation
Historical metadata may contain legacy operators such as

<
*=

But predicate creation logic expects normalized internal operators

<<
=

When condition_op is not recognized, parse_to_predicate() leaves creator function pointer null and then calls it, causing coredump

Temporary Contingency Plan

set skip_delete_predicate = true;
disable compaction

Reproduction Steps

set delete_without_partition = true
DELETE FROM ${tableName} WHERE k1 <=> 1
Because the metadata contains the <=> operator, but BE does not implement the corresponding logic

Release note

None

Check List (For Author)

Test
- Regression test
- Unit Test
- Manual test (add detailed scripts or steps below)
- No need to test or manual test. Explain why:
  - This is a refactor/code format and no logic has been changed.
  - Previous test can cover this change.
  - No code files have been changed.
  - Other reason
Behavior changed:
- No.
- Yes.
Does this need documentation?
- No.
- Yes.

Check List (For Reviewer who merge this PR)

Confirm the release note
Confirm test cases
Confirm document
Add branch pick label

hello-stephen · 2026-03-31T12:23:31Z

Thank you for your contribution to Apache Doris.
Don't know what should be done next? See How to process your PR.

Please clearly describe your PR:

What problem was fixed (it's best to include specific error reporting information). How it was fixed.
Which behaviors were modified. What was the previous behavior, what is it now, why was it modified, and what possible impacts might there be.
What features were added. Why was this function added?
Which code was refactored and why was this part of the code refactored?
Which functions were optimized and what is the difference before and after the optimization?

felixwluo · 2026-03-31T12:24:48Z

run buildall

doris-robot · 2026-03-31T14:26:57Z

BE UT Coverage Report

Increment line coverage 100.00% (1/1) 🎉

Increment coverage report
Complete coverage report

Category	Coverage
Function Coverage	42.09% (11243/26712)
Line Coverage	32.62% (96245/295084)
Region Coverage	30.56% (55276/180897)
Branch Coverage	26.88% (27359/101786)

hello-stephen · 2026-03-31T15:09:22Z

BE Regression && UT Coverage Report

Increment line coverage 100.00% (1/1) 🎉

Increment coverage report
Complete coverage report

Category	Coverage
Function Coverage	75.11% (19718/26253)
Line Coverage	68.35% (201165/294305)
Region Coverage	66.44% (120371/181167)
Branch Coverage	59.72% (61021/102182)

felixwluo · 2026-03-31T16:31:36Z

run p0

felixwluo · 2026-03-31T18:48:39Z

run p0

hello-stephen · 2026-03-31T18:50:12Z

BE Regression && UT Coverage Report

Increment line coverage 100.00% (1/1) 🎉

Increment coverage report
Complete coverage report

Category	Coverage
Function Coverage	75.12% (19721/26253)
Line Coverage	68.36% (201201/294305)
Region Coverage	66.45% (120394/181167)
Branch Coverage	59.73% (61033/102182)

hello-stephen · 2026-03-31T20:58:45Z

BE Regression && UT Coverage Report

Increment line coverage 100.00% (1/1) 🎉

Increment coverage report
Complete coverage report

Category	Coverage
Function Coverage	75.12% (19721/26253)
Line Coverage	68.37% (201204/294305)
Region Coverage	66.46% (120405/181167)
Branch Coverage	59.74% (61043/102182)

felixwluo · 2026-04-01T03:36:53Z

run external

hello-stephen · 2026-04-01T05:37:13Z

BE Regression && UT Coverage Report

Increment line coverage 100.00% (1/1) 🎉

Increment coverage report
Complete coverage report

Category	Coverage
Function Coverage	75.12% (19721/26253)
Line Coverage	68.36% (201198/294305)
Region Coverage	66.46% (120404/181167)
Branch Coverage	59.73% (61037/102182)

felixwluo · 2026-04-01T06:20:48Z

run p0

Hastyshell

LGTM

github-actions · 2026-04-01T07:19:50Z

PR approved by anyone and no changes requested.

hello-stephen · 2026-04-01T08:29:58Z

BE Regression && UT Coverage Report

Increment line coverage 100.00% (1/1) 🎉

Increment coverage report
Complete coverage report

Category	Coverage
Function Coverage	75.12% (19720/26253)
Line Coverage	68.36% (201189/294305)
Region Coverage	66.45% (120392/181167)
Branch Coverage	59.74% (61040/102182)

felixwluo · 2026-04-01T17:30:15Z

run buildall

felixwluo · 2026-04-01T17:52:52Z

run buildall

hello-stephen · 2026-04-01T19:09:13Z

BE UT Coverage Report

Increment line coverage 75.00% (3/4) 🎉

Increment coverage report
Complete coverage report

Category	Coverage
Function Coverage	42.10% (11245/26712)
Line Coverage	32.62% (96254/295086)
Region Coverage	30.56% (55276/180903)
Branch Coverage	26.88% (27357/101790)

hello-stephen · 2026-04-01T20:56:18Z

BE Regression && UT Coverage Report

Increment line coverage 100.00% (4/4) 🎉

Increment coverage report
Complete coverage report

Category	Coverage
Function Coverage	75.10% (19715/26253)
Line Coverage	68.34% (201130/294307)
Region Coverage	66.46% (120401/181173)
Branch Coverage	59.74% (61049/102186)

github-actions · 2026-04-02T04:17:36Z

PR approved by at least one committer and no changes requested.

Hastyshell

LGTM

lide-reed

LGTM

[fix](core) normalize DeleteSubPredicatePB operator on read path

8051b78

felixwluo requested a review from lide-reed as a code owner March 31, 2026 12:23

add beut

128b4e1

Hastyshell approved these changes Apr 1, 2026

View reviewed changes

github-actions bot added the reviewed label Apr 1, 2026

save

f84e01e

felixwluo added 3 commits April 2, 2026 01:32

delete beut

3cd80a7

save

b0496c7

save

c2f4105

cambyzju approved these changes Apr 2, 2026

View reviewed changes

github-actions bot added the approved Indicates a PR has been approved by one committer. label Apr 2, 2026

Hastyshell approved these changes Apr 2, 2026

View reviewed changes

felixwluo changed the title ~~[fix](core) normalize DeleteSubPredicatePB operator on read path~~ branch-3.0:[fix](core) normalize DeleteSubPredicatePB operator on read path #62052 Apr 2, 2026

lide-reed approved these changes Apr 3, 2026

View reviewed changes

lide-reed merged commit 0d3907c into apache:branch-3.0 Apr 3, 2026
30 of 31 checks passed

Conversation

felixwluo commented Mar 31, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

compaction core

query core

Summary

Root Cause

Temporary Contingency Plan

Reproduction Steps

Release note

Check List (For Author)

Check List (For Reviewer who merge this PR)

Uh oh!

hello-stephen commented Mar 31, 2026

Uh oh!

felixwluo commented Mar 31, 2026

Uh oh!

doris-robot commented Mar 31, 2026

BE UT Coverage Report

Uh oh!

hello-stephen commented Mar 31, 2026

BE Regression && UT Coverage Report

Uh oh!

felixwluo commented Mar 31, 2026

Uh oh!

felixwluo commented Mar 31, 2026

Uh oh!

hello-stephen commented Mar 31, 2026

BE Regression && UT Coverage Report

Uh oh!

hello-stephen commented Mar 31, 2026

BE Regression && UT Coverage Report

Uh oh!

felixwluo commented Apr 1, 2026

Uh oh!

hello-stephen commented Apr 1, 2026

BE Regression && UT Coverage Report

Uh oh!

felixwluo commented Apr 1, 2026

Uh oh!

Hastyshell left a comment

Choose a reason for hiding this comment

Uh oh!

github-actions bot commented Apr 1, 2026

Uh oh!

hello-stephen commented Apr 1, 2026

BE Regression && UT Coverage Report

Uh oh!

felixwluo commented Apr 1, 2026

Uh oh!

felixwluo commented Apr 1, 2026

Uh oh!

hello-stephen commented Apr 1, 2026

BE UT Coverage Report

Uh oh!

hello-stephen commented Apr 1, 2026

BE Regression && UT Coverage Report

Uh oh!

github-actions bot commented Apr 2, 2026

Uh oh!

Hastyshell left a comment

Choose a reason for hiding this comment

Uh oh!

lide-reed left a comment

Choose a reason for hiding this comment

Uh oh!

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

6 participants

felixwluo commented Mar 31, 2026 •

edited

Loading