Skip to content

[Fix](JsonPath) return null when meet unknown escape sequence, example '$.name\\k' #50859

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom
Merged
merged 1 commit into from
May 14, 2025
Merged

[Fix](JsonPath) return null when meet unknown escape sequence, example '$.name\\k' #50859

merged 1 commit into from
May 14, 2025
+28 −8

Conversation

eldenmoon
Copy link
Member

@eldenmoon eldenmoon commented May 13, 2025 •
edited
Loading

What problem does this PR solve?

Issue Number: close #xxx

Related PR: #xxx

Problem Summary:

Release note

None

Check List (For Author)

  • Test

    • Regression test
    • Unit Test
    • Manual test (add detailed scripts or steps below)
    • No need to test or manual test. Explain why:
      • This is a refactor/code format and no logic has been changed.
      • Previous test can cover this change.
      • No code files have been changed.
      • Other reason

  • Behavior changed:

    • No.
    • Yes.

  • Does this need documentation?

    • No.
    • Yes.

Check List (For Reviewer who merge this PR)

  • Confirm the release note
  • Confirm test cases
  • Confirm document
  • Add branch pick label

@Thearas
Copy link
Contributor

Thearas commented May 13, 2025

Thank you for your contribution to Apache Doris.
Don't know what should be done next? See How to process your PR.

Please clearly describe your PR:

  1. What problem was fixed (it's best to include specific error reporting information). How it was fixed.
  2. Which behaviors were modified. What was the previous behavior, what is it now, why was it modified, and what possible impacts might there be.
  3. What features were added. Why was this function added?
  4. Which code was refactored and why was this part of the code refactored?
  5. Which functions were optimized and what is the difference before and after the optimization?

@eldenmoon eldenmoon changed the title [Fix](JsonPath) return null when meet unknown escape sequence, exampl… [Fix](JsonPath) return null when meet unknown escape sequence, example '$.name\\k' May 13, 2025
@eldenmoon
Copy link
Member Author

run buildall

csun5285
csun5285 approved these changes May 13, 2025
View reviewed changes
Copy link
Contributor

@csun5285 csun5285 left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM

@github-actions GitHub Actions
Copy link
Contributor

PR approved by anyone and no changes requested.

@doris-robot
Copy link

TPC-H: Total hot run time: 33764 ms 
machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/tpch-tools
Tpch sf100 test result on commit ebe183f5ee24d6e2394411cca6371003b697268c, data reload: false

------ Round 1 ----------------------------------
q1	25859	5028	5004	5004
q2	2057	276	199	199
q3	10379	1289	671	671
q4	10240	989	532	532
q5	7501	2316	2377	2316
q6	178	164	130	130
q7	900	758	618	618
q8	9475	1263	1093	1093
q9	6908	5183	5110	5110
q10	6874	2307	1903	1903
q11	504	285	259	259
q12	343	350	218	218
q13	17797	3651	3120	3120
q14	222	238	207	207
q15	529	485	478	478
q16	429	429	372	372
q17	578	841	352	352
q18	7670	7024	7145	7024
q19	1321	935	550	550
q20	340	328	216	216
q21	3858	3293	2413	2413
q22	1046	981	979	979
Total cold run time: 115008 ms
Total hot run time: 33764 ms

----- Round 2, with runtime_filter_mode=off -----
q1	5069	5035	5057	5035
q2	242	321	234	234
q3	2161	2597	2288	2288
q4	1386	1757	1388	1388
q5	4324	4389	4366	4366
q6	217	178	123	123
q7	2027	1912	1787	1787
q8	2588	2625	2530	2530
q9	7237	7138	6950	6950
q10	3004	3187	2733	2733
q11	582	514	489	489
q12	669	764	634	634
q13	3519	3889	3281	3281
q14	298	314	277	277
q15	537	496	484	484
q16	443	482	446	446
q17	1114	1519	1391	1391
q18	7706	7509	7468	7468
q19	784	766	1009	766
q20	1996	2016	1819	1819
q21	5078	4737	4702	4702
q22	1093	1050	1048	1048
Total cold run time: 52074 ms
Total hot run time: 50239 ms

@doris-robot
Copy link

TPC-DS: Total hot run time: 192959 ms 
machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/tpcds-tools
TPC-DS sf100 test result on commit ebe183f5ee24d6e2394411cca6371003b697268c, data reload: false

query1	1396	1084	1070	1070
query2	6340	1849	1840	1840
query3	10992	4605	4479	4479
query4	55046	24758	23057	23057
query5	4982	558	464	464
query6	344	223	214	214
query7	4872	526	297	297
query8	317	254	234	234
query9	5503	2652	2673	2652
query10	447	329	274	274
query11	14950	14967	14881	14881
query12	166	110	111	110
query13	1067	539	426	426
query14	10258	6283	6208	6208
query15	197	204	186	186
query16	7067	644	500	500
query17	1144	726	553	553
query18	1560	410	322	322
query19	212	193	175	175
query20	132	132	130	130
query21	208	127	116	116
query22	4310	4419	4367	4367
query23	34269	33547	33572	33547
query24	6952	2418	2425	2418
query25	464	482	479	479
query26	689	274	156	156
query27	2243	507	359	359
query28	3147	2165	2139	2139
query29	576	565	431	431
query30	266	219	198	198
query31	872	885	750	750
query32	82	69	62	62
query33	440	353	313	313
query34	781	860	555	555
query35	789	850	754	754
query36	955	1050	911	911
query37	118	104	79	79
query38	4264	4280	4265	4265
query39	1511	1419	1477	1419
query40	212	116	103	103
query41	63	54	54	54
query42	132	113	118	113
query43	502	507	485	485
query44	1329	844	830	830
query45	178	178	174	174
query46	865	1049	641	641
query47	1859	1918	1814	1814
query48	392	444	326	326
query49	699	529	437	437
query50	670	704	410	410
query51	4213	4266	4253	4253
query52	115	106	112	106
query53	240	259	188	188
query54	633	611	531	531
query55	94	87	86	86
query56	343	316	326	316
query57	1156	1194	1141	1141
query58	266	272	260	260
query59	2726	2787	2689	2689
query60	359	346	314	314
query61	137	122	145	122
query62	729	748	687	687
query63	227	191	185	185
query64	1707	1065	685	685
query65	4333	4185	4220	4185
query66	704	394	309	309
query67	16116	15681	15484	15484
query68	6222	888	527	527
query69	538	316	274	274
query70	1172	1154	1077	1077
query71	466	344	302	302
query72	6187	4869	4684	4684
query73	1320	632	362	362
query74	9377	9002	8663	8663
query75	3234	3264	2735	2735
query76	3846	1255	748	748
query77	516	375	287	287
query78	10096	10218	9404	9404
query79	2828	818	585	585
query80	984	513	448	448
query81	532	269	216	216
query82	693	127	93	93
query83	304	249	227	227
query84	299	107	86	86
query85	792	359	313	313
query86	452	323	284	284
query87	4505	4486	4292	4292
query88	3379	2302	2284	2284
query89	391	315	293	293
query90	1640	210	213	210
query91	147	141	111	111
query92	75	59	57	57
query93	2576	928	575	575
query94	778	396	350	350
query95	365	286	285	285
query96	491	576	280	280
query97	3165	3218	3107	3107
query98	238	205	203	203
query99	1442	1411	1279	1279
Total cold run time: 299977 ms
Total hot run time: 192959 ms

@doris-robot
Copy link

ClickBench: Total hot run time: 28.95 s 
machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/clickbench-tools
ClickBench test result on commit ebe183f5ee24d6e2394411cca6371003b697268c, data reload: false

query1	0.04	0.04	0.03
query2	0.12	0.10	0.12
query3	0.26	0.19	0.19
query4	1.59	0.20	0.20
query5	0.61	0.59	0.58
query6	1.19	0.72	0.73
query7	0.02	0.02	0.01
query8	0.04	0.04	0.04
query9	0.58	0.53	0.51
query10	0.56	0.58	0.56
query11	0.15	0.10	0.11
query12	0.14	0.12	0.12
query13	0.61	0.60	0.61
query14	0.78	0.80	0.83
query15	0.89	0.86	0.88
query16	0.39	0.39	0.39
query17	1.02	1.05	1.06
query18	0.23	0.22	0.21
query19	1.99	1.83	1.89
query20	0.01	0.00	0.01
query21	15.39	0.88	0.53
query22	0.74	1.26	0.76
query23	14.76	1.37	0.62
query24	6.98	1.80	0.36
query25	0.42	0.19	0.06
query26	0.71	0.16	0.13
query27	0.06	0.05	0.05
query28	9.36	0.88	0.46
query29	12.53	3.95	3.32
query30	0.25	0.09	0.06
query31	2.82	0.57	0.38
query32	3.23	0.55	0.47
query33	3.05	3.14	3.10
query34	15.61	5.14	4.46
query35	4.50	4.48	4.53
query36	0.69	0.50	0.49
query37	0.09	0.06	0.06
query38	0.06	0.04	0.04
query39	0.03	0.02	0.03
query40	0.17	0.13	0.13
query41	0.08	0.03	0.02
query42	0.03	0.02	0.02
query43	0.03	0.03	0.03
Total cold run time: 102.81 s
Total hot run time: 28.95 s

amorynan
amorynan approved these changes May 13, 2025
View reviewed changes
Copy link
Contributor

@amorynan amorynan left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM

@hello-stephen
Copy link
Contributor

BE UT Coverage Report

Increment line coverage 75.00% (6/8) 🎉

Increment coverage report
Complete coverage report

Category Coverage
Function Coverage 55.79% (14894/26698)
Line Coverage 44.59% (131795/295581)
Region Coverage 43.64% (66273/151850)
Branch Coverage 38.26% (33962/88758)

@hello-stephen
Copy link
Contributor

BE Regression && UT Coverage Report

Increment line coverage 75.00% (6/8) 🎉

Increment coverage report
Complete coverage report

Category Coverage
Function Coverage 79.37% (20847/26264)
Line Coverage 72.61% (214564/295517)
Region Coverage 70.75% (126188/178355)
Branch Coverage 64.49% (65373/101368)

@github-actions github-actions bot added the approved Indicates a PR has been approved by one committer. label May 14, 2025
@github-actions GitHub Actions
Copy link
Contributor

PR approved by at least one committer and no changes requested.

@eldenmoon eldenmoon merged commit 522e7ce into apache:master May 14, 2025
31 of 33 checks passed
github-actions bot pushed a commit that referenced this pull request May 14, 2025
@eldenmoon
[Fix](JsonPath) return null when meet unknown escape sequence, exampl…
9acdaf5 
…e '$.name\\k' (#50859)
@github-actions github-actions bot mentioned this pull request May 14, 2025
Merged
@eldenmoon eldenmoon deleted the fix-jsonpath-escape branch May 14, 2025 02:19
@eldenmoon eldenmoon added p0_b usercase Important user case type label labels May 14, 2025
dataroaring pushed a commit that referenced this pull request May 14, 2025
@github-actions @eldenmoon
branch-3.0: [Fix](JsonPath) return null when meet unknown escape sequ…
889d35c 
…ence, example '$.name\\k' #50859 (#50870)

Cherry-picked from #50859

Co-authored-by: lihangyu <lihangyu@selectdb.com>
eldenmoon added a commit to eldenmoon/incubator-doris that referenced this pull request May 15, 2025
@eldenmoon
[Fix](JsonPath) return null when meet unknown escape sequence, exampl…
bfd1f5e 
…e '$.name\\k' (apache#50859)
@eldenmoon eldenmoon mentioned this pull request May 15, 2025
Merged
eldenmoon added a commit to eldenmoon/incubator-doris that referenced this pull request May 15, 2025
@eldenmoon
[Fix](JsonPath) return null when meet unknown escape sequence, exampl…
11df0b3 
…e '$.name\\k' (apache#50859)
yiguolei pushed a commit that referenced this pull request May 17, 2025
@eldenmoon
[Fix](JsonPath) return null when meet unknown escape sequence, exampl…
5611a39 
…e '$.name\\k' (#50930)

cherry-pick from #50859
koarz pushed a commit to koarz/doris that referenced this pull request Jun 4, 2025
@eldenmoon @koarz
[Fix](JsonPath) return null when meet unknown escape sequence, exampl…
47e97a3 
…e '$.name\\k' (apache#50859)
@gavinchou gavinchou mentioned this pull request Jun 11, 2025
Open
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Reviewers

@englefly englefly englefly approved these changes

@amorynan amorynan amorynan approved these changes

@csun5285 csun5285 csun5285 approved these changes

Assignees
No one assigned
Labels
approved Indicates a PR has been approved by one committer. dev/2.1.11-merged dev/3.0.6-merged p0_b reviewed usercase Important user case type label
Projects
None yet
Milestone
No milestone
Development

Successfully merging this pull request may close these issues.
9 participants
@eldenmoon @Thearas @doris-robot @hello-stephen @englefly @amorynan @csun5285 @yiguolei @dataroaring