[enhancement](inverted index) ensure consistent ordering of properties for inverted index show #51467

airborne12 · 2025-06-04T03:50:47Z

What problem does this PR solve?

Issue Number: close #xxx

Related PR: #xxx

Problem Summary:
This PR enhances the inverted index implementation to always output its properties in a consistent, alphabetical order and adds tests to verify this behavior.

Release note

None

Check List (For Author)

Test
- Regression test
- Unit Test
- Manual test (add detailed scripts or steps below)
- No need to test or manual test. Explain why:
  - This is a refactor/code format and no logic has been changed.
  - Previous test can cover this change.
  - No code files have been changed.
  - Other reason
Behavior changed:
- No.
- Yes.
Does this need documentation?
- No.
- Yes.

Check List (For Reviewer who merge this PR)

Confirm the release note
Confirm test cases
Confirm document
Add branch pick label

…s for inverted index show

airborne12 · 2025-06-04T03:51:00Z

run buildall

Thearas · 2025-06-04T03:51:06Z

Thank you for your contribution to Apache Doris.
Don't know what should be done next? See How to process your PR.

Please clearly describe your PR:

What problem was fixed (it's best to include specific error reporting information). How it was fixed.
Which behaviors were modified. What was the previous behavior, what is it now, why was it modified, and what possible impacts might there be.
What features were added. Why was this function added?
Which code was refactored and why was this part of the code refactored?
Which functions were optimized and what is the difference before and after the optimization?

Copilot

Pull Request Overview

This PR enhances the inverted index implementation to always output its properties in a consistent, alphabetical order and adds tests to verify this behavior.

Refactored Index.getPropertiesString() to use a sorted map for deterministic ordering.
Added IndexPropertiesOrderTest to validate consistent property string generation.
Added ShowCreateTableStmtTest.testIndexPropertiesOrder to verify ordering in SHOW CREATE TABLE.

Reviewed Changes

Copilot reviewed 3 out of 3 changed files in this pull request and generated no comments.

File	Description
fe/fe-core/src/main/java/org/apache/doris/catalog/Index.java	Switched to `TreeMap` in `getPropertiesString()` for sorting.
fe/fe-core/src/test/java/org/apache/doris/catalog/IndexPropertiesOrderTest.java	New unit test ensuring index properties string is consistent.
fe/fe-core/src/test/java/org/apache/doris/analysis/ShowCreateTableStmtTest.java	New test verifying property order in `SHOW CREATE TABLE` output.

Comments suppressed due to low confidence (3)

fe/fe-core/src/test/java/org/apache/doris/analysis/ShowCreateTableStmtTest.java:102

Using assertTrue(...contains(...)) on a long, concatenated string can make tests brittle. Consider asserting the exact expected substring with assertEquals or using a regex/matcher to validate the property order more robustly.

Assertions.assertTrue(showSql1.contains("INDEX index_name (`name`) USING INVERTED " +

fe/fe-core/src/test/java/org/apache/doris/catalog/IndexPropertiesOrderTest.java:58

The test confirms each property is present but does not strictly enforce their ordering. Consider adding assertions that check the index positions (e.g., props1.indexOf("lower_case") < props1.indexOf("parser")) to ensure the alphabetical order is correctly applied.

Assertions.assertTrue(props1.contains("lower_case"), "Properties should contain lower_case");

fe/fe-core/src/main/java/org/apache/doris/catalog/Index.java:149

[nitpick] Consider importing java.util.TreeMap at the top of the file instead of using the fully qualified name inline for cleaner and more readable code.

return "(" + new PrintableMap(new java.util.TreeMap<>(properties), "=", true, false, ",").toString() + ")";

…s for inverted index show

airborne12 · 2025-06-04T04:03:47Z

run buildall

…s for inverted index show

airborne12 · 2025-06-04T04:07:26Z

run buildall

doris-robot · 2025-06-04T04:27:06Z

TPC-H: Total hot run time: 33658 ms

machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/tpch-tools
Tpch sf100 test result on commit 015e3fc1b1bbc03fd320ec9fadfd789f5c8abd48, data reload: false

------ Round 1 ----------------------------------
q1	26280	5041	5041	5041
q2	1966	277	173	173
q3	10282	1218	675	675
q4	10229	979	550	550
q5	7550	2374	2312	2312
q6	199	166	132	132
q7	910	739	602	602
q8	9295	1227	1138	1138
q9	6932	5123	5054	5054
q10	6867	2285	1899	1899
q11	484	279	270	270
q12	342	352	222	222
q13	17779	3677	3044	3044
q14	228	245	208	208
q15	572	476	484	476
q16	427	423	372	372
q17	584	856	366	366
q18	7731	7259	7005	7005
q19	1778	954	552	552
q20	337	325	217	217
q21	3673	3177	2370	2370
q22	1080	987	980	980
Total cold run time: 115525 ms
Total hot run time: 33658 ms

----- Round 2, with runtime_filter_mode=off -----
q1	5238	5094	5112	5094
q2	243	310	223	223
q3	2132	2649	2310	2310
q4	1322	1771	1339	1339
q5	4543	4428	4389	4389
q6	222	174	126	126
q7	1973	1894	1754	1754
q8	2598	2575	2516	2516
q9	7203	7025	7174	7025
q10	2993	3221	2771	2771
q11	597	512	492	492
q12	681	757	598	598
q13	3478	3846	3328	3328
q14	285	295	268	268
q15	504	481	463	463
q16	450	479	443	443
q17	1160	1512	1406	1406
q18	7677	7333	7409	7333
q19	778	811	936	811
q20	2002	1996	1900	1900
q21	4749	4403	4148	4148
q22	1054	1030	985	985
Total cold run time: 51882 ms
Total hot run time: 49722 ms

doris-robot · 2025-06-04T04:38:38Z

TPC-DS: Total hot run time: 185423 ms

machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/tpcds-tools
TPC-DS sf100 test result on commit 015e3fc1b1bbc03fd320ec9fadfd789f5c8abd48, data reload: false

query1	997	473	475	473
query2	6566	1870	1823	1823
query3	6756	228	224	224
query4	26038	23714	23335	23335
query5	4361	628	459	459
query6	315	209	203	203
query7	4623	488	298	298
query8	278	225	227	225
query9	8625	2592	2615	2592
query10	474	351	269	269
query11	15758	15263	14734	14734
query12	160	108	106	106
query13	1662	512	393	393
query14	8776	6289	6162	6162
query15	200	196	181	181
query16	7145	626	472	472
query17	1190	704	542	542
query18	1965	395	305	305
query19	193	186	168	168
query20	118	125	121	121
query21	214	126	107	107
query22	4106	4154	4028	4028
query23	34065	33031	33163	33031
query24	8415	2418	2335	2335
query25	573	468	390	390
query26	1226	265	150	150
query27	2761	495	342	342
query28	4335	2112	2082	2082
query29	754	557	439	439
query30	280	219	188	188
query31	930	833	741	741
query32	70	62	61	61
query33	560	356	306	306
query34	776	838	524	524
query35	779	813	739	739
query36	919	979	881	881
query37	113	107	81	81
query38	4049	4156	4022	4022
query39	1513	1425	1591	1425
query40	213	121	110	110
query41	71	69	72	69
query42	133	131	119	119
query43	504	510	464	464
query44	1281	832	825	825
query45	188	179	177	177
query46	835	1029	639	639
query47	1783	1776	1694	1694
query48	395	431	319	319
query49	760	488	418	418
query50	656	673	436	436
query51	4055	4164	4180	4164
query52	111	109	101	101
query53	227	252	181	181
query54	573	573	501	501
query55	87	85	86	85
query56	339	300	303	300
query57	1105	1167	1072	1072
query58	262	256	249	249
query59	2622	2790	2481	2481
query60	327	328	298	298
query61	129	126	147	126
query62	804	742	678	678
query63	217	186	183	183
query64	4386	1044	672	672
query65	4207	4116	4164	4116
query66	1151	407	312	312
query67	15701	15348	15473	15348
query68	7896	863	523	523
query69	475	304	264	264
query70	1182	1151	1103	1103
query71	419	315	301	301
query72	5591	4732	4826	4732
query73	676	642	361	361
query74	8863	8957	8923	8923
query75	3635	3203	2673	2673
query76	3361	1191	763	763
query77	770	376	293	293
query78	9963	10046	9351	9351
query79	2663	816	583	583
query80	628	539	435	435
query81	498	258	222	222
query82	192	126	100	100
query83	253	244	249	244
query84	255	112	101	101
query85	767	368	312	312
query86	383	313	294	294
query87	4380	4528	4328	4328
query88	4045	2246	2283	2246
query89	389	312	283	283
query90	1951	211	203	203
query91	132	144	109	109
query92	83	61	57	57
query93	2368	954	576	576
query94	657	413	306	306
query95	392	293	284	284
query96	490	557	283	283
query97	2689	2747	2609	2609
query98	240	212	207	207
query99	1344	1522	1260	1260
Total cold run time: 273390 ms
Total hot run time: 185423 ms

doris-robot · 2025-06-04T04:43:48Z

ClickBench: Total hot run time: 29.3 s

machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/clickbench-tools
ClickBench test result on commit 015e3fc1b1bbc03fd320ec9fadfd789f5c8abd48, data reload: false

query1	0.03	0.03	0.03
query2	0.12	0.10	0.11
query3	0.25	0.19	0.19
query4	1.59	0.19	0.20
query5	0.45	0.45	0.44
query6	1.15	0.66	0.66
query7	0.02	0.02	0.01
query8	0.05	0.04	0.04
query9	0.59	0.52	0.51
query10	0.57	0.58	0.56
query11	0.16	0.11	0.11
query12	0.14	0.11	0.12
query13	0.62	0.62	0.59
query14	0.79	0.80	0.80
query15	0.88	0.86	0.86
query16	0.37	0.37	0.38
query17	1.01	1.02	1.03
query18	0.22	0.22	0.21
query19	1.86	1.81	1.85
query20	0.01	0.01	0.02
query21	15.42	0.89	0.54
query22	0.75	1.22	0.96
query23	14.69	1.36	0.63
query24	7.66	1.03	0.74
query25	0.51	0.24	0.07
query26	0.59	0.16	0.15
query27	0.06	0.05	0.05
query28	9.65	0.90	0.44
query29	12.53	4.14	3.39
query30	0.25	0.10	0.07
query31	2.82	0.58	0.39
query32	3.23	0.53	0.46
query33	3.03	3.04	3.10
query34	15.68	5.14	4.47
query35	4.46	4.48	4.48
query36	0.68	0.51	0.48
query37	0.09	0.06	0.06
query38	0.05	0.04	0.04
query39	0.03	0.03	0.02
query40	0.16	0.14	0.13
query41	0.08	0.02	0.02
query42	0.03	0.02	0.02
query43	0.03	0.04	0.03
Total cold run time: 103.36 s
Total hot run time: 29.3 s

…s for inverted index show

airborne12 · 2025-06-04T05:42:27Z

run buildall

doris-robot · 2025-06-04T06:02:26Z

TPC-H: Total hot run time: 33843 ms

machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/tpch-tools
Tpch sf100 test result on commit 8018d073aa1ef98b0d54b4953ef966b31c13ad44, data reload: false

------ Round 1 ----------------------------------
q1	26505	5054	4999	4999
q2	1965	283	188	188
q3	10271	1216	714	714
q4	10226	998	514	514
q5	7572	2339	2385	2339
q6	175	165	131	131
q7	902	757	629	629
q8	9305	1290	1017	1017
q9	6845	5188	5114	5114
q10	6837	2356	1902	1902
q11	481	287	284	284
q12	336	360	213	213
q13	17776	3718	3082	3082
q14	222	238	216	216
q15	554	479	483	479
q16	448	429	374	374
q17	582	853	375	375
q18	7656	7194	7166	7166
q19	1216	943	557	557
q20	342	330	226	226
q21	3698	3129	2343	2343
q22	1051	1025	981	981
Total cold run time: 114965 ms
Total hot run time: 33843 ms

----- Round 2, with runtime_filter_mode=off -----
q1	5069	5039	5078	5039
q2	242	317	218	218
q3	2196	2689	2354	2354
q4	1373	1793	1300	1300
q5	4525	4441	4390	4390
q6	211	170	124	124
q7	1982	1929	1749	1749
q8	2588	2491	2561	2491
q9	7280	7137	7079	7079
q10	2986	3170	2780	2780
q11	611	514	492	492
q12	708	751	619	619
q13	3508	3896	3200	3200
q14	288	291	276	276
q15	531	477	476	476
q16	454	485	440	440
q17	1161	1525	1426	1426
q18	7700	7613	7512	7512
q19	798	801	832	801
q20	1979	2047	1899	1899
q21	4896	4246	4202	4202
q22	1073	1064	1003	1003
Total cold run time: 52159 ms
Total hot run time: 49870 ms

doris-robot · 2025-06-04T06:13:57Z

TPC-DS: Total hot run time: 186107 ms

machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/tpcds-tools
TPC-DS sf100 test result on commit 8018d073aa1ef98b0d54b4953ef966b31c13ad44, data reload: false

query1	998	477	496	477
query2	6562	1866	1833	1833
query3	6751	224	224	224
query4	26416	23694	23754	23694
query5	4336	623	475	475
query6	311	199	210	199
query7	4617	475	275	275
query8	247	226	220	220
query9	8596	2581	2592	2581
query10	498	331	285	285
query11	15791	15050	14811	14811
query12	172	110	110	110
query13	1657	519	406	406
query14	8719	6113	6206	6113
query15	196	198	178	178
query16	7129	619	433	433
query17	1172	705	553	553
query18	1969	382	287	287
query19	184	177	155	155
query20	118	114	123	114
query21	219	123	103	103
query22	4092	4121	4014	4014
query23	33919	33124	33221	33124
query24	8444	2355	2374	2355
query25	537	456	397	397
query26	1236	269	143	143
query27	2770	499	331	331
query28	4362	2100	2090	2090
query29	769	544	419	419
query30	281	217	190	190
query31	938	867	747	747
query32	73	68	62	62
query33	560	359	310	310
query34	796	833	520	520
query35	772	792	733	733
query36	963	981	872	872
query37	111	103	81	81
query38	4134	4141	4046	4046
query39	1493	1412	1419	1412
query40	217	125	115	115
query41	71	66	67	66
query42	126	111	109	109
query43	505	519	496	496
query44	1299	832	813	813
query45	177	178	174	174
query46	833	1031	625	625
query47	1757	1795	1739	1739
query48	392	424	314	314
query49	760	523	415	415
query50	637	676	409	409
query51	4161	4190	4059	4059
query52	108	103	98	98
query53	225	254	186	186
query54	582	573	520	520
query55	90	88	87	87
query56	357	302	288	288
query57	1131	1150	1104	1104
query58	262	266	260	260
query59	2580	2734	2675	2675
query60	321	333	298	298
query61	146	123	174	123
query62	802	725	656	656
query63	218	187	197	187
query64	4382	1031	659	659
query65	4278	4159	4166	4159
query66	1155	405	333	333
query67	15888	15627	15462	15462
query68	8648	876	511	511
query69	469	306	268	268
query70	1229	1150	1110	1110
query71	465	330	322	322
query72	5541	4766	4693	4693
query73	718	596	345	345
query74	9216	9058	8817	8817
query75	3999	3195	2768	2768
query76	3618	1208	740	740
query77	788	392	280	280
query78	10072	10243	9292	9292
query79	2261	823	576	576
query80	598	519	433	433
query81	487	263	218	218
query82	432	129	104	104
query83	285	250	243	243
query84	302	115	93	93
query85	787	350	304	304
query86	336	331	279	279
query87	4448	4484	4392	4392
query88	3393	2274	2320	2274
query89	388	312	285	285
query90	1923	205	208	205
query91	149	145	111	111
query92	79	59	59	59
query93	1372	972	569	569
query94	662	418	305	305
query95	373	288	277	277
query96	493	561	281	281
query97	2771	2771	2651	2651
query98	240	216	207	207
query99	1427	1416	1294	1294
Total cold run time: 274337 ms
Total hot run time: 186107 ms

doris-robot · 2025-06-04T06:19:11Z

ClickBench: Total hot run time: 29.21 s

machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/clickbench-tools
ClickBench test result on commit 8018d073aa1ef98b0d54b4953ef966b31c13ad44, data reload: false

query1	0.04	0.04	0.03
query2	0.13	0.10	0.11
query3	0.27	0.19	0.20
query4	1.60	0.20	0.11
query5	0.45	0.42	0.45
query6	1.16	0.65	0.66
query7	0.02	0.02	0.02
query8	0.04	0.04	0.04
query9	0.59	0.52	0.51
query10	0.57	0.59	0.58
query11	0.16	0.11	0.10
query12	0.15	0.12	0.12
query13	0.61	0.61	0.61
query14	0.78	0.82	0.82
query15	0.89	0.84	0.86
query16	0.41	0.38	0.39
query17	1.02	1.07	1.03
query18	0.23	0.21	0.21
query19	1.88	1.80	1.92
query20	0.02	0.01	0.02
query21	15.40	0.89	0.55
query22	0.77	1.22	0.79
query23	14.82	1.41	0.64
query24	7.26	0.84	0.79
query25	0.50	0.20	0.13
query26	0.59	0.17	0.15
query27	0.05	0.05	0.05
query28	9.66	0.96	0.46
query29	12.53	4.03	3.37
query30	0.26	0.10	0.07
query31	2.82	0.60	0.38
query32	3.23	0.55	0.47
query33	3.06	3.01	3.06
query34	15.60	5.11	4.47
query35	4.53	4.56	4.50
query36	0.67	0.50	0.48
query37	0.09	0.07	0.06
query38	0.06	0.04	0.04
query39	0.03	0.03	0.03
query40	0.16	0.13	0.13
query41	0.07	0.03	0.02
query42	0.03	0.02	0.02
query43	0.04	0.03	0.03
Total cold run time: 103.25 s
Total hot run time: 29.21 s

eldenmoon

LGTM

github-actions · 2025-06-05T02:24:13Z

PR approved by at least one committer and no changes requested.

github-actions · 2025-06-05T02:24:15Z

PR approved by anyone and no changes requested.

…s for inverted index show (#51467) Problem Summary: This PR enhances the inverted index implementation to always output its properties in a consistent, alphabetical order and adds tests to verify this behavior.

…of properties for inverted index show #51467 (#51586) Cherry-picked from #51467 Co-authored-by: airborne12 <jiangkai@selectdb.com>

[enhancement](inverted index) ensure consistent ordering of propertie…

0814568

…s for inverted index show

airborne12 requested a review from Copilot June 4, 2025 03:50

Copilot AI reviewed Jun 4, 2025

View reviewed changes

[enhancement](inverted index) ensure consistent ordering of propertie…

560da3f

…s for inverted index show

[enhancement](inverted index) ensure consistent ordering of propertie…

015e3fc

…s for inverted index show

[enhancement](inverted index) ensure consistent ordering of propertie…

8018d07

…s for inverted index show

eldenmoon added the kind/behavior-changed label Jun 5, 2025

eldenmoon approved these changes Jun 5, 2025

View reviewed changes

github-actions bot added the approved Indicates a PR has been approved by one committer. label Jun 5, 2025

github-actions bot added the reviewed label Jun 5, 2025

hello-stephen approved these changes Jun 5, 2025

View reviewed changes

hello-stephen merged commit 47ffca9 into apache:master Jun 5, 2025
26 of 27 checks passed

airborne12 deleted the fix-properties branch June 5, 2025 06:49

shuke987 added the dev/3.0.x label Jun 9, 2025

github-actions bot mentioned this pull request Jun 9, 2025

branch-3.0: [enhancement](inverted index) ensure consistent ordering of properties for inverted index show #51467 #51586

Merged

dataroaring pushed a commit that referenced this pull request Jun 10, 2025

branch-3.0: [enhancement](inverted index) ensure consistent ordering …

da61e82

…of properties for inverted index show #51467 (#51586) Cherry-picked from #51467 Co-authored-by: airborne12 <jiangkai@selectdb.com>

dataroaring added dev/3.0.6-merged dev/3.0.7-merged and removed dev/3.0.x dev/3.0.6-merged labels Jun 10, 2025

morrySnow added the dev/3.1.0-merged label Jun 17, 2025

gavinchou mentioned this pull request Aug 15, 2025

Release Note 3.0.7 #54868

Open

[enhancement](inverted index) ensure consistent ordering of properties for inverted index show #51467

[enhancement](inverted index) ensure consistent ordering of properties for inverted index show #51467

Uh oh!

Conversation

airborne12 commented Jun 4, 2025 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

What problem does this PR solve?

Release note

Check List (For Author)

Check List (For Reviewer who merge this PR)

Uh oh!

airborne12 commented Jun 4, 2025

Uh oh!

Thearas commented Jun 4, 2025

Uh oh!

Copilot AI left a comment

Choose a reason for hiding this comment

Pull Request Overview

Reviewed Changes

Uh oh!

airborne12 commented Jun 4, 2025

Uh oh!

airborne12 commented Jun 4, 2025

Uh oh!

doris-robot commented Jun 4, 2025

Uh oh!

doris-robot commented Jun 4, 2025

Uh oh!

doris-robot commented Jun 4, 2025

Uh oh!

airborne12 commented Jun 4, 2025

Uh oh!

doris-robot commented Jun 4, 2025

Uh oh!

doris-robot commented Jun 4, 2025

Uh oh!

doris-robot commented Jun 4, 2025

Uh oh!

eldenmoon left a comment

Choose a reason for hiding this comment

Uh oh!

github-actions bot commented Jun 5, 2025

Uh oh!

github-actions bot commented Jun 5, 2025

Uh oh!

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

8 participants

airborne12 commented Jun 4, 2025 •

edited

Loading