[fix](jsonb) Avoid crashing caused by invalid path #50978

mrhhsg · 2025-05-16T09:07:45Z

What problem does this PR solve?

*** SIGABRT unknown detail explain (@0x1d45) received by PID 7493 (TID 9664 OR 0x7f1a05f7d640) from PID 7493; stack trace: ***
doris_be: /root/doris/be/src/util/jsonb_document.h:273: char doris::Stream::peek() const: Assertion `!exhausted()' failed.
 0# doris::signal::(anonymous namespace)::FailureSignalHandler(int, siginfo_t*, void*) at /root/doris/be/src/common/signal_handler.h:421
 1# 0x00007F23B6C2F520 in /lib/x86_64-linux-gnu/libc.so.6
 2# pthread_kill at ./nptl/pthread_kill.c:89
 3# raise at ../sysdeps/posix/raise.c:27
 4# abort at ./stdlib/abort.c:81
 5# _nl_load_domain at ./intl/loadmsgcat.c:1177
 6# 0x00007F23B6C26E96 in /lib/x86_64-linux-gnu/libc.so.6
 7# doris::Stream::peek() const at /root/doris/be/src/util/jsonb_document.h:273
 8# doris::JsonbPath::seek(char const*, unsigned long) at /root/doris/be/src/util/jsonb_document.h:1427
 9# doris::vectorized::JsonbExtractImpl::vector_vector(doris::FunctionContext*, doris::vectorized::PODArray, 16ul, 15ul> const&, doris::vectorized::PODArray, 16ul, 15ul> const&, doris::vectorized::PODArray, 16ul, 15ul> const&, doris::vectorized::PODArray, 16ul, 15ul> const&, doris::vectorized::PODArray, 16ul, 15ul>&, doris::vectorized::PODArray, 16ul, 15ul>&, bool&) in /mnt/hdd01/ci/doris-deploy-master-local/be/lib/doris_be
10# doris::vectorized::FunctionJsonbExtract::execute_impl(doris::FunctionContext*, doris::vectorized::Block&, std::vector > const&, unsigned int, unsigned long) const at /root/doris/be/src/vec/functions/function_jsonb.cpp:419
11# doris::vectorized::DefaultExecutable::execute_impl(doris::FunctionContext*, doris::vectorized::Block&, std::vector > const&, unsigned int, unsigned long) const at /root/doris/be/src/vec/functions/function.h:454
12# doris::vectorized::PreparedFunctionImpl::_execute_skipped_constant_deal(doris::FunctionContext*, doris::vectorized::Block&, std::vector > const&, unsigned int, unsigned long, bool) const in /mnt/hdd01/ci/doris-deploy-master-local/be/lib/doris_be
13# doris::vectorized::PreparedFunctionImpl::execute_without_low_cardinality_columns(doris::FunctionContext*, doris::vectorized::Block&, std::vector > const&, unsigned int, unsigned long, bool) const at /root/doris/be/src/vec/functions/function.cpp:244
14# doris::vectorized::PreparedFunctionImpl::default_implementation_for_nulls(doris::FunctionContext*, doris::vectorized::Block&, std::vector > const&, unsigned int, unsigned long, bool, bool*) const at /root/doris/be/src/vec/functions/function.cpp:216
15# doris::vectorized::PreparedFunctionImpl::_execute_skipped_constant_deal(doris::FunctionContext*, doris::vectorized::Block&, std::vector > const&, unsigned int, unsigned long, bool) const at /root/doris/be/src/vec/functions/function.cpp:110
16# doris::vectorized::PreparedFunctionImpl::execute_without_low_cardinality_columns(doris::FunctionContext*, doris::vectorized::Block&, std::vector > const&, unsigned int, unsigned long, bool) const at /root/doris/be/src/vec/functions/function.cpp:244
17# doris::vectorized::PreparedFunctionImpl::execute(doris::FunctionContext*, doris::vectorized::Block&, std::vector > const&, unsigned int, unsigned long, bool) const at /root/doris/be/src/vec/functions/function.cpp:250
18# doris::vectorized::IFunctionBase::execute(doris::FunctionContext*, doris::vectorized::Block&, std::vector > const&, unsigned int, unsigned long, bool) const at /root/doris/be/src/vec/functions/function.h:196
19# doris::vectorized::VectorizedFnCall::_do_execute(doris::vectorized::VExprContext*, doris::vectorized::Block*, int*, std::vector >&) at /root/doris/be/src/vec/exprs/vectorized_fn_call.cpp:203
20# doris::vectorized::VectorizedFnCall::execute(doris::vectorized::VExprContext*, doris::vectorized::Block*, int*) at /root/doris/be/src/vec/exprs/vectorized_fn_call.cpp:237
21# doris::vectorized::VectorizedFnCall::_do_execute(doris::vectorized::VExprContext*, doris::vectorized::Block*, int*, std::vector >&) in /mnt/hdd01/ci/doris-deploy-master-local/be/lib/doris_be
22# doris::vectorized::VectorizedFnCall::execute(doris::vectorized::VExprContext*, doris::vectorized::Block*, int*) at /root/doris/be/src/vec/exprs/vectorized_fn_call.cpp:237
23# doris::vectorized::VExprContext::execute(doris::vectorized::Block*, int*) in /mnt/hdd01/ci/doris-deploy-master-local/be/lib/doris_be
24# doris::vectorized::VExprContext::execute_conjuncts(std::vector, std::allocator > > const&, std::vector, 16ul, 15ul>*, std::allocator, 16ul, 15ul>*> > const*, bool, doris::vectorized::Block*, doris::vectorized::PODArray, 16ul, 15ul>*, bool*) in /mnt/hdd01/ci/doris-deploy-master-local/be/lib/doris_be
25# doris::vectorized::VExprContext::execute_conjuncts_and_filter_block(std::vector, std::allocator > > const&, doris::vectorized::Block*, std::vector >&, int, doris::vectorized::PODArray, 16ul, 15ul>&) at /root/doris/be/src/vec/exprs/vexpr_context.cpp:377
26# doris::segment_v2::SegmentIterator::_execute_common_expr(unsigned short*, unsigned short&, doris::vectorized::Block*) in /mnt/hdd01/ci/doris-deploy-master-local/be/lib/doris_be
27# doris::segment_v2::SegmentIterator::_next_batch_internal(doris::vectorized::Block*) at /root/doris/be/src/olap/rowset/segment_v2/segment_iterator.cpp:2228
28# doris::segment_v2::SegmentIterator::next_batch(doris::vectorized::Block*) at /root/doris/be/src/olap/rowset/segment_v2/segment_iterator.cpp:1920
29# doris::segment_v2::LazyInitSegmentIterator::next_batch(doris::vectorized::Block*) at /root/doris/be/src/olap/rowset/segment_v2/lazy_init_segment_iterator.h:49
30# doris::BetaRowsetReader::next_block(doris::vectorized::Block*) at /root/doris/be/src/olap/rowset/beta_rowset_reader.cpp:346
31# doris::vectorized::VCollectIterator::Level0Iterator::_refresh() at /root/doris/be/src/vec/olap/vcollect_iterator.h:256
32# doris::vectorized::VCollectIterator::Level0Iterator::refresh_current_row() in /mnt/hdd01/ci/doris-deploy-master-local/be/lib/doris_be
33# doris::vectorized::VCollectIterator::Level0Iterator::ensure_first_row_ref() at /root/doris/be/src/vec/olap/vcollect_iterator.cpp:483
34# doris::vectorized::VCollectIterator::Level1Iterator::ensure_first_row_ref() in /mnt/hdd01/ci/doris-deploy-master-local/be/lib/doris_be
35# doris::vectorized::VCollectIterator::build_heap(std::vector, std::allocator > >&) in /mnt/hdd01/ci/doris-deploy-master-local/be/lib/doris_be
36# doris::vectorized::BlockReader::_init_collect_iter(doris::TabletReader::ReaderParams const&) in /mnt/hdd01/ci/doris-deploy-master-local/be/lib/doris_be
37# doris::vectorized::BlockReader::init(doris::TabletReader::ReaderParams const&) at /root/doris/be/src/vec/olap/block_reader.cpp:222
38# doris::vectorized::OlapScanner::open(doris::RuntimeState*) at /root/doris/be/src/vec/exec/scan/olap_scanner.cpp:237
39# doris::vectorized::ScannerScheduler::_scanner_scan(std::shared_ptr, std::shared_ptr) in /mnt/hdd01/ci/doris-deploy-master-local/be/lib/doris_be
40# std::_Function_handler, std::shared_ptr)::$_1::operator()() const::{lambda()#1}>::_M_invoke(std::_Any_data const&) at /var/local/ldb-toolchain/bin/../lib/gcc/x86_64-linux-gnu/11/../../../../include/c++/11/bits/std_function.h:291
41# doris::ThreadPool::dispatch_thread() in /mnt/hdd01/ci/doris-deploy-master-local/be/lib/doris_be
42# doris::Thread::supervise_thread(void*) at /root/doris/be/src/util/thread.cpp:496
43# start_thread at ./nptl/pthread_create.c:442
44# 0x00007F23B6D13850 at ../sysdeps/unix/sysv/linux/x86_64/clone3.S:83

Related PR: #xxx

Problem Summary:

Release note

None

Check List (For Author)

Test
- Regression test
- Unit Test
- Manual test (add detailed scripts or steps below)
- No need to test or manual test. Explain why:
  - This is a refactor/code format and no logic has been changed.
  - Previous test can cover this change.
  - No code files have been changed.
  - Other reason
Behavior changed:
- No.
- Yes.
Does this need documentation?
- No.
- Yes.

Check List (For Reviewer who merge this PR)

Confirm the release note
Confirm test cases
Confirm document
Add branch pick label

Thearas · 2025-05-16T09:07:51Z

Thank you for your contribution to Apache Doris.
Don't know what should be done next? See How to process your PR.

Please clearly describe your PR:

What problem was fixed (it's best to include specific error reporting information). How it was fixed.
Which behaviors were modified. What was the previous behavior, what is it now, why was it modified, and what possible impacts might there be.
What features were added. Why was this function added?
Which code was refactored and why was this part of the code refactored?
Which functions were optimized and what is the difference before and after the optimization?

mrhhsg · 2025-05-16T09:08:00Z

run buildall

doris-robot · 2025-05-16T10:45:32Z

TPC-H: Total hot run time: 34003 ms

machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/tpch-tools
Tpch sf100 test result on commit 348582c5b97b45dd9b189fa23df9507c8d2e6355, data reload: false

------ Round 1 ----------------------------------
q1	27784	5080	5120	5080
q2	2088	274	181	181
q3	10588	1266	687	687
q4	10258	1036	530	530
q5	7945	2700	2347	2347
q6	178	161	135	135
q7	922	744	605	605
q8	9303	1281	1060	1060
q9	6850	5068	5097	5068
q10	6827	2327	1878	1878
q11	462	290	291	290
q12	342	355	215	215
q13	17782	3672	3127	3127
q14	232	252	215	215
q15	524	491	480	480
q16	430	434	379	379
q17	614	836	410	410
q18	7721	7119	7171	7119
q19	1301	976	566	566
q20	344	351	230	230
q21	3870	3164	2404	2404
q22	1073	1013	997	997
Total cold run time: 117438 ms
Total hot run time: 34003 ms

----- Round 2, with runtime_filter_mode=off -----
q1	5119	5078	5023	5023
q2	237	328	231	231
q3	2167	2665	2321	2321
q4	1337	1803	1453	1453
q5	4510	4382	4409	4382
q6	212	176	131	131
q7	2012	1949	1772	1772
q8	2582	2530	2540	2530
q9	7265	7186	7143	7143
q10	3004	3200	2745	2745
q11	577	509	510	509
q12	676	771	637	637
q13	3554	3912	3398	3398
q14	278	294	270	270
q15	507	485	475	475
q16	445	500	443	443
q17	1178	1580	1370	1370
q18	7948	7642	7563	7563
q19	787	841	1030	841
q20	1953	1959	1847	1847
q21	4960	4461	4415	4415
q22	1106	1081	1016	1016
Total cold run time: 52414 ms
Total hot run time: 50515 ms

doris-robot · 2025-05-16T10:57:28Z

TPC-DS: Total hot run time: 193409 ms

machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/tpcds-tools
TPC-DS sf100 test result on commit 348582c5b97b45dd9b189fa23df9507c8d2e6355, data reload: false

query1	1409	1101	1042	1042
query2	6423	1920	1909	1909
query3	11177	4804	4887	4804
query4	25575	23802	23327	23327
query5	4413	626	453	453
query6	300	210	207	207
query7	3993	497	294	294
query8	303	278	246	246
query9	8495	2656	2662	2656
query10	461	325	280	280
query11	15418	15346	14792	14792
query12	157	117	105	105
query13	1555	531	412	412
query14	8909	6308	6192	6192
query15	210	193	170	170
query16	7187	632	497	497
query17	1199	769	630	630
query18	1982	424	334	334
query19	205	210	177	177
query20	130	127	122	122
query21	208	127	109	109
query22	4544	4641	4262	4262
query23	34692	33933	33767	33767
query24	8522	2453	2445	2445
query25	547	491	402	402
query26	1230	275	159	159
query27	2772	525	348	348
query28	4804	2179	2147	2147
query29	741	609	436	436
query30	281	219	186	186
query31	901	891	798	798
query32	71	69	65	65
query33	524	374	314	314
query34	829	866	536	536
query35	829	828	745	745
query36	956	993	897	897
query37	107	98	78	78
query38	4258	4306	4216	4216
query39	1571	1469	1473	1469
query40	211	127	117	117
query41	56	55	52	52
query42	132	112	117	112
query43	509	551	506	506
query44	1356	847	849	847
query45	185	176	167	167
query46	850	1023	667	667
query47	1836	1852	1840	1840
query48	403	429	340	340
query49	787	528	441	441
query50	667	710	413	413
query51	4229	4291	4339	4291
query52	116	108	103	103
query53	230	265	196	196
query54	575	588	525	525
query55	85	85	84	84
query56	323	309	302	302
query57	1177	1203	1149	1149
query58	279	268	278	268
query59	2794	2814	2694	2694
query60	341	320	301	301
query61	127	128	122	122
query62	754	712	653	653
query63	230	188	183	183
query64	4049	1002	666	666
query65	4370	4268	4231	4231
query66	1018	399	302	302
query67	15954	15542	15461	15461
query68	8768	903	519	519
query69	499	317	276	276
query70	1248	1159	1106	1106
query71	482	330	303	303
query72	5630	4751	4851	4751
query73	731	632	350	350
query74	9289	9248	8832	8832
query75	3954	3195	2668	2668
query76	3787	1208	751	751
query77	780	403	285	285
query78	10203	10135	9406	9406
query79	2287	837	595	595
query80	652	516	439	439
query81	485	262	220	220
query82	450	121	92	92
query83	258	256	235	235
query84	247	110	82	82
query85	790	428	305	305
query86	375	301	271	271
query87	4428	4492	4331	4331
query88	3606	2320	2287	2287
query89	415	323	287	287
query90	1842	212	215	212
query91	140	146	113	113
query92	72	62	57	57
query93	1891	980	578	578
query94	664	381	311	311
query95	375	294	284	284
query96	493	571	291	291
query97	2700	2770	2641	2641
query98	237	208	205	205
query99	1433	1405	1265	1265
Total cold run time: 280742 ms
Total hot run time: 193409 ms

doris-robot · 2025-05-16T11:02:40Z

ClickBench: Total hot run time: 29.08 s

machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/clickbench-tools
ClickBench test result on commit 348582c5b97b45dd9b189fa23df9507c8d2e6355, data reload: false

query1	0.04	0.04	0.03
query2	0.12	0.11	0.10
query3	0.25	0.19	0.19
query4	1.59	0.20	0.20
query5	0.42	0.41	0.43
query6	1.17	0.65	0.65
query7	0.02	0.02	0.02
query8	0.04	0.04	0.04
query9	0.58	0.53	0.51
query10	0.57	0.58	0.56
query11	0.16	0.11	0.11
query12	0.15	0.12	0.12
query13	0.61	0.60	0.60
query14	0.78	0.79	0.80
query15	0.86	0.84	0.86
query16	0.39	0.37	0.40
query17	1.01	1.01	1.08
query18	0.23	0.21	0.21
query19	1.93	1.79	1.90
query20	0.02	0.01	0.01
query21	15.41	0.88	0.56
query22	0.76	1.15	0.66
query23	14.93	1.36	0.62
query24	6.75	2.21	0.72
query25	0.46	0.16	0.08
query26	0.64	0.18	0.14
query27	0.05	0.06	0.05
query28	9.21	0.87	0.46
query29	12.84	4.01	3.37
query30	0.25	0.09	0.07
query31	2.82	0.58	0.38
query32	3.25	0.55	0.47
query33	3.05	3.13	3.06
query34	15.76	5.10	4.54
query35	4.54	4.54	4.53
query36	0.66	0.50	0.48
query37	0.09	0.07	0.06
query38	0.06	0.04	0.04
query39	0.03	0.02	0.03
query40	0.18	0.14	0.14
query41	0.08	0.03	0.02
query42	0.04	0.02	0.03
query43	0.03	0.03	0.04
Total cold run time: 102.83 s
Total hot run time: 29.08 s

doris-robot · 2025-05-16T12:23:16Z

BE UT Coverage Report

Increment line coverage 16.67% (1/6) 🎉

Increment coverage report
Complete coverage report

Category	Coverage
Function Coverage	55.87% (14885/26641)
Line Coverage	44.65% (131898/295378)
Region Coverage	43.76% (66406/151744)
Branch Coverage	38.36% (34027/88698)

hello-stephen · 2025-05-16T15:10:05Z

BE Regression && UT Coverage Report

Increment line coverage 50.00% (3/6) 🎉

Increment coverage report
Complete coverage report

Category	Coverage
Function Coverage	79.34% (20810/26228)
Line Coverage	72.62% (214500/295389)
Region Coverage	70.78% (126205/178300)
Branch Coverage	64.58% (65436/101330)

hello-stephen · 2025-05-17T01:10:22Z

BE Regression && UT Coverage Report

Increment line coverage 50.00% (3/6) 🎉

Increment coverage report
Complete coverage report

Category	Coverage
Function Coverage	79.34% (20810/26228)
Line Coverage	72.62% (214500/295389)
Region Coverage	70.78% (126205/178300)
Branch Coverage	64.58% (65436/101330)

eldenmoon

LGTM

github-actions · 2025-05-17T10:30:48Z

PR approved by at least one committer and no changes requested.

github-actions · 2025-05-17T10:30:50Z

PR approved by anyone and no changes requested.

csun5285

LGTM

pick #50978

[fix](jsonb) Avoid crashing caused by invalid path

348582c

eldenmoon approved these changes May 17, 2025

View reviewed changes

eldenmoon added dev/2.1.x dev/3.0.x labels May 17, 2025

github-actions bot added the approved Indicates a PR has been approved by one committer. label May 17, 2025

github-actions bot added the reviewed label May 17, 2025

csun5285 approved these changes May 17, 2025

View reviewed changes

yiguolei added the usercase Important user case type label label May 17, 2025

yiguolei merged commit 1af10e8 into apache:master May 17, 2025
27 of 30 checks passed

github-actions bot added dev/3.0.x-conflict dev/2.1.x-conflict labels May 17, 2025

mrhhsg added a commit to mrhhsg/doris that referenced this pull request May 18, 2025

[fix](jsonb) Avoid crashing caused by invalid path (apache#50978)

1c1f38a

mrhhsg mentioned this pull request May 18, 2025

[fix](jsonb) Avoid crashing caused by invalid path (#50978) #50994

Merged

16 tasks

mrhhsg deleted the fix_jsonb branch May 18, 2025 06:35

dataroaring pushed a commit that referenced this pull request May 18, 2025

[fix](jsonb) Avoid crashing caused by invalid path (#50978) (#50994)

e06e252

pick #50978

dataroaring added dev/3.0.6-merged and removed dev/3.0.x dev/3.0.x-conflict labels May 18, 2025

yiguolei removed dev/2.1.x dev/2.1.x-conflict labels May 18, 2025

koarz pushed a commit to koarz/doris that referenced this pull request Jun 4, 2025

[fix](jsonb) Avoid crashing caused by invalid path (apache#50978)

c525c54

gavinchou mentioned this pull request Jun 11, 2025

Release Note 3.0.6 #51585

Open

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Uh oh!

[fix](jsonb) Avoid crashing caused by invalid path #50978

[fix](jsonb) Avoid crashing caused by invalid path #50978

Uh oh!

mrhhsg commented May 16, 2025

Uh oh!

Thearas commented May 16, 2025

Uh oh!

mrhhsg commented May 16, 2025

Uh oh!

doris-robot commented May 16, 2025

Uh oh!

doris-robot commented May 16, 2025

Uh oh!

doris-robot commented May 16, 2025

Uh oh!

doris-robot commented May 16, 2025

Uh oh!

hello-stephen commented May 16, 2025

Uh oh!

hello-stephen commented May 17, 2025

Uh oh!

eldenmoon left a comment

Uh oh!

github-actions bot commented May 17, 2025

Uh oh!

github-actions bot commented May 17, 2025

Uh oh!

csun5285 left a comment

Uh oh!

Uh oh!

Uh oh!

[fix](jsonb) Avoid crashing caused by invalid path #50978

[fix](jsonb) Avoid crashing caused by invalid path #50978

Uh oh!

Conversation

mrhhsg commented May 16, 2025

What problem does this PR solve?

Release note

Check List (For Author)

Check List (For Reviewer who merge this PR)

Uh oh!

Thearas commented May 16, 2025

Uh oh!

mrhhsg commented May 16, 2025

Uh oh!

doris-robot commented May 16, 2025

Uh oh!

doris-robot commented May 16, 2025

Uh oh!

doris-robot commented May 16, 2025

Uh oh!

doris-robot commented May 16, 2025

BE UT Coverage Report

Uh oh!

hello-stephen commented May 16, 2025

BE Regression && UT Coverage Report

Uh oh!

hello-stephen commented May 17, 2025

BE Regression && UT Coverage Report

Uh oh!

eldenmoon left a comment

Choose a reason for hiding this comment

Uh oh!

github-actions bot commented May 17, 2025

Uh oh!

github-actions bot commented May 17, 2025

Uh oh!

csun5285 left a comment

Choose a reason for hiding this comment

Uh oh!

Uh oh!

Uh oh!