[improvement](load) reduce MemTable per-row memory overhead

[liaoxin01]

What problem does this PR solve?

Problem Summary:

MemTable keeps one RowInBlock per loaded row in _row_in_blocks, held as a
shared_ptr, so there is one make_shared per row. That costs 80 bytes per
row, measured: 16 for the pointer in the vector, plus a 64 byte heap chunk
holding the 16 byte control block and the 40 byte struct. A 7.05M row memtable
spends 564 MB on row bookkeeping alone, resident for its whole life.

Two of the struct's five fields do not need to be there:

field	why it can go
size_t* _agg_state_offset	held _offsets_of_aggregate_states.data(), the same pointer for every row
bool _has_init_agg	said exactly what _agg_mem being non-null already says

That leaves 24 bytes, small enough to keep in the vector by value and drop the
per-row allocation with it. The remaining fields are unchanged, and so is
everything the memtable does with them.

7.05M rows	before	after
_row_in_blocks	564 MB	169 MB
building the array	401 ms	111 ms

Both are measured rather than derived: sizeof(RowInBlock) plus the resident
size of 5M rows for the first, a build loop of 7.05M rows for the second.

What the shared_ptr was quietly providing

Rows in _row_in_blocks and the copies _aggregate() works on used to be the
same object. They are not any more, and two places depended on that:

• prev_row now points into temp_row_in_blocks. Aggregation mutates the
  group representative, and the copy _finalize_one_row() later reads is the
  one in temp_row_in_blocks, so that is the one it has to point at.
• _aggregate() adopts temp_row_in_blocks unconditionally rather than
  only on the non-final path, so the entries left behind cannot still name an
  aggregate state that _finalize_one_row() has released. Without this, a
  memtable that aggregates across a shrink_memtable_by_agg() round and then
  again in to_block() releases those states twice. That is not hypothetical:
  instrumenting ~MemTable showed it destroying two states that
  _finalize_one_row<true>() had already destroyed.

Related

#66588 replaces the memtable sort with the vectorized ColumnSorter and is
stacked on this one. Its scratch grows with the key width, so it wants the rows
themselves to be cheap first; the ordering and the resulting peaks per key type
are spelled out there. Neither change needs the other to be correct.

Release note

None

Check List (For Author)

• Test
  • Unit Test
  • Regression test
  • Manual test (add detailed scripts or steps below)
  • No need to test or manual test. Explain why:
    • This is a refactor/code format and no logic has been changed.
    • Previous test can cover this change.
    • No code files have been changed.
    • Other reason

be/test/load/memtable/memtable_sort_test.cpp only covered class Tie. It
still does, and now also drives a MemTable through insert()/to_block(),
covering multi-column key ordering, nullable key ordering (NULL first), the
DUP_KEYS tie-break direction, independence from how rows are split across
insert() calls, UNIQUE_KEYS last-writer-wins, AGG_KEYS aggregation, and
aggregate state surviving shrink_memtable_by_agg() rounds. The AGG_KEYS
schema carries a BITMAP BITMAP_UNION column so those last cases run over an
aggregate state that owns heap memory rather than a trivially destroyed one.

Two gaps worth stating plainly:

• Those cases reach the double-release path but do not fail on it in a
  release build, where a double free is not reliably fatal. ASAN is what turns
  it into a failure.

• The flexible-partial-update paths are the largest part of this diff and have
  no unit coverage, before or after. I checked them line by line against the
  previous semantics, but that is a reading, not a test, and P0 is what has to
  cover them.

• Behavior changed:

  • No.
  • Yes.

• Does this need documentation?

  • No.

[hello-stephen]

Thank you for your contribution to Apache Doris.
Don't know what should be done next? See How to process your PR.

Please clearly describe your PR:

1. What problem was fixed (it's best to include specific error reporting information). How it was fixed.
2. Which behaviors were modified. What was the previous behavior, what is it now, why was it modified, and what possible impacts might there be.
3. What features were added. Why was this function added?
4. Which code was refactored and why was this part of the code refactored?
5. Which functions were optimized and what is the difference before and after the optimization?

[liaoxin01]

run buildall

[Copilot]

Pull request overview

This PR optimizes BE load-path memtable sorting by replacing MemTable’s per-comparison virtual/indirect multi-key sort with the vectorized ColumnSorter used by the query engine, and updates unit tests to validate the new ordering and aggregation behavior.

Changes:

• Implement _sort_permutation_by_key_columns() and use ColumnSorter-based permutation sorting in _sort() and _sort_by_cluster_keys().
• Remove now-unused Tie and _sort_one_column() helpers.
• Rewrite memtable sort unit tests to exercise real MemTable::insert()/to_block() sorting and key semantics.

Reviewed changes

Copilot reviewed 3 out of 3 changed files in this pull request and generated 3 comments.

File	Description
be/src/load/memtable/memtable.cpp	Replaces memtable sorting with ColumnSorter permutation sorting and updates cluster-key sort accordingly.
be/src/load/memtable/memtable.h	Removes unused Tie and declares the new permutation-sorting helper.
be/test/load/memtable/memtable_sort_test.cpp	Reworks tests to validate multi-key ordering, NULL handling, batching independence, and UNIQUE/AGG behaviors via real MemTable flow.

Suppressed comments (2)

be/test/load/memtable/memtable_sort_test.cpp:150

• If mem_table.insert() fails, the test should stop immediately; otherwise later steps may operate on a partially-initialized MemTable and produce confusing follow-up failures. Prefer ASSERT_TRUE here.

            Status st = mem_table.insert(&block, payload);
            EXPECT_TRUE(st.ok()) << st;

be/test/load/memtable/memtable_sort_test.cpp:156

• If to_block() fails (or doesn't set out), the helper should abort; using EXPECT_TRUE can return a null out and crash the caller. Use ASSERT_TRUE and assert out is non-null.

        std::unique_ptr<Block> out;
        Status st = mem_table.to_block(&out);
        EXPECT_TRUE(st.ok()) << st;
        return out;

---

💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.

[hello-stephen]

BE UT Coverage Report

Increment line coverage 94.83% (55/58) 🎉

Increment coverage report
Complete coverage report

Category	Coverage
Function Coverage	59.76% (26061/43607)
Line Coverage	44.09% (264383/599593)
Region Coverage	39.86% (210655/528421)
Branch Coverage	41.27% (96435/233670)

[hello-stephen]

BE Regression && UT Coverage Report

Increment line coverage 94.83% (55/58) 🎉

Increment coverage report
Complete coverage report

Category	Coverage
Function Coverage	75.93% (32285/42522)
Line Coverage	60.75% (361782/595540)
Region Coverage	57.29% (303706/530158)
Branch Coverage	58.66% (137096/233694)

[liaoxin01]

run buildall

[liaoxin01]

run buildall

[hello-stephen]

BE UT Coverage Report

Increment line coverage 96.72% (59/61) 🎉

Increment coverage report
Complete coverage report

Category	Coverage
Function Coverage	59.76% (26061/43607)
Line Coverage	44.09% (264386/599599)
Region Coverage	39.87% (210694/528427)
Branch Coverage	41.27% (96433/233674)

[hello-stephen]

TPC-H: Total hot run time: 28783 ms

machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/tpch-tools
Tpch sf100 test result on commit 5ccb379a0da5627bc04b0b384589083ec1db449b, data reload: false

------ Round 1 ----------------------------------
============================================
q1	17887	3883	3873	3873
q2	1989	317	193	193
q3	10357	1385	819	819
q4	4682	472	344	344
q5	7565	820	563	563
q6	179	168	136	136
q7	728	803	605	605
q8	9911	1655	1550	1550
q9	5839	4040	4003	4003
q10	6799	1640	1371	1371
q11	503	353	320	320
q12	714	579	454	454
q13	18092	3227	2731	2731
q14	266	259	244	244
q15	q16	738	717	660	660
q17	1035	1008	928	928
q18	6585	5605	5520	5520
q19	1356	1237	1124	1124
q20	778	677	581	581
q21	5842	2547	2466	2466
q22	415	347	298	298
Total cold run time: 102260 ms
Total hot run time: 28783 ms

----- Round 2, with runtime_filter_mode=off -----
============================================
q1	4273	4165	4185	4165
q2	274	313	205	205
q3	4537	4863	4356	4356
q4	2177	2231	1418	1418
q5	4213	4127	4073	4073
q6	230	182	125	125
q7	1691	1580	1423	1423
q8	2807	2107	2086	2086
q9	7182	7250	7112	7112
q10	4271	4287	3865	3865
q11	549	414	378	378
q12	712	716	501	501
q13	3224	3654	2853	2853
q14	289	320	275	275
q15	q16	706	742	628	628
q17	1308	1300	1304	1300
q18	12160	11039	11847	11039
q19	1153	1130	1154	1130
q20	2213	2221	1959	1959
q21	5574	4871	4752	4752
q22	514	477	410	410
Total cold run time: 60057 ms
Total hot run time: 54053 ms

[hello-stephen]

TPC-DS: Total hot run time: 166522 ms

machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/tpcds-tools
TPC-DS sf100 test result on commit 5ccb379a0da5627bc04b0b384589083ec1db449b, data reload: false

query5	4306	583	452	452
query6	461	221	209	209
query7	4869	604	350	350
query8	313	161	149	149
query9	8745	4027	4026	4026
query10	481	355	299	299
query11	5770	2206	2007	2007
query12	151	99	94	94
query13	1239	621	440	440
query14	6028	4263	3945	3945
query14_1	3786	3780	3742	3742
query15	199	193	176	176
query16	1024	469	447	447
query17	934	675	535	535
query18	2430	462	339	339
query19	214	191	147	147
query20	104	101	99	99
query21	239	155	132	132
query22	12903	13025	12848	12848
query23	15740	15052	14759	14759
query23_1	14931	14632	14780	14632
query24	7698	1733	1216	1216
query24_1	1253	1244	1288	1244
query25	569	438	386	386
query26	1340	378	224	224
query27	2549	604	385	385
query28	4563	2057	2043	2043
query29	1122	628	509	509
query30	351	265	224	224
query31	1210	1112	1091	1091
query32	114	63	60	60
query33	534	320	288	288
query34	1220	1179	640	640
query35	742	743	648	648
query36	771	772	741	741
query37	160	109	94	94
query38	1829	1755	1688	1688
query39	840	806	786	786
query39_1	770	790	774	774
query40	251	165	153	153
query41	65	64	60	60
query42	93	92	92	92
query43	316	322	281	281
query44	1452	772	765	765
query45	185	176	168	168
query46	1053	1227	735	735
query47	1559	1519	1393	1393
query48	406	386	270	270
query49	587	404	293	293
query50	1074	422	336	336
query51	10446	10330	10634	10330
query52	91	87	75	75
query53	261	266	206	206
query54	279	242	221	221
query55	79	73	66	66
query56	298	309	278	278
query57	1034	1002	946	946
query58	285	253	267	253
query59	1549	1658	1406	1406
query60	332	271	250	250
query61	159	146	153	146
query62	389	317	267	267
query63	232	210	199	199
query64	2858	1023	851	851
query65	3902	3809	3811	3809
query66	1836	473	365	365
query67	27975	28101	27912	27912
query68	3076	1587	1015	1015
query69	393	309	259	259
query70	900	816	756	756
query71	377	346	317	317
query72	2981	2729	2488	2488
query73	871	769	419	419
query74	4615	4496	4285	4285
query75	2388	2345	1997	1997
query76	2308	1150	819	819
query77	346	385	284	284
query78	11029	10968	10558	10558
query79	1404	1161	792	792
query80	688	578	481	481
query81	453	330	286	286
query82	622	176	142	142
query83	409	377	295	295
query84	326	173	134	134
query85	937	608	521	521
query86	315	240	224	224
query87	1978	1959	1826	1826
query88	3773	2840	2860	2840
query89	400	321	293	293
query90	1955	206	202	202
query91	220	195	162	162
query92	71	63	54	54
query93	1581	1523	955	955
query94	554	365	325	325
query95	804	594	475	475
query96	1097	854	373	373
query97	2452	2443	2392	2392
query98	189	189	183	183
query99	738	724	607	607
Total cold run time: 251805 ms
Total hot run time: 166522 ms

[hello-stephen]

ClickBench: Total hot run time: 23.82 s

machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/clickbench-tools
ClickBench test result on commit 5ccb379a0da5627bc04b0b384589083ec1db449b, data reload: false

query1	0.00	0.00	0.01
query2	0.09	0.05	0.05
query3	0.25	0.14	0.13
query4	1.61	0.14	0.15
query5	0.23	0.21	0.21
query6	1.15	0.83	0.84
query7	0.03	0.01	0.01
query8	0.06	0.04	0.04
query9	0.37	0.32	0.30
query10	0.59	0.54	0.53
query11	0.20	0.14	0.13
query12	0.17	0.13	0.14
query13	0.46	0.45	0.46
query14	1.00	0.98	1.01
query15	0.60	0.58	0.58
query16	0.31	0.31	0.32
query17	1.14	1.07	1.09
query18	0.21	0.19	0.21
query19	2.06	1.97	1.94
query20	0.02	0.01	0.01
query21	15.41	0.23	0.15
query22	4.70	0.05	0.06
query23	16.13	0.31	0.12
query24	2.96	0.42	0.32
query25	0.12	0.06	0.08
query26	0.74	0.20	0.15
query27	0.04	0.04	0.04
query28	3.52	0.87	0.37
query29	12.51	3.99	3.20
query30	0.27	0.15	0.16
query31	2.77	0.57	0.32
query32	3.22	0.58	0.50
query33	3.09	3.17	3.22
query34	15.50	3.96	3.26
query35	3.23	3.20	3.19
query36	0.56	0.44	0.40
query37	0.08	0.06	0.07
query38	0.05	0.04	0.03
query39	0.04	0.03	0.03
query40	0.17	0.15	0.14
query41	0.07	0.03	0.03
query42	0.04	0.02	0.02
query43	0.04	0.04	0.03
Total cold run time: 95.81 s
Total hot run time: 23.82 s

[hello-stephen]

BE Regression && UT Coverage Report

Increment line coverage 96.72% (59/61) 🎉

Increment coverage report
Complete coverage report

Category	Coverage
Function Coverage	75.95% (32294/42522)
Line Coverage	60.75% (361789/595546)
Region Coverage	57.40% (304322/530164)
Branch Coverage	58.84% (137513/233698)

[sollhui]

This change may significantly increase the peak memory usage of MemTable sorting, especially for narrow tables with many rows.
The old implementation sorted _row_in_blocks in place and its main row-count-dependent temporary allocation was Tie, which uses roughly 1 byte per row.
The new implementation introduces several additional O(N) allocations:

• perm: N * sizeof(size_t), about 8 bytes/row;

• EqualFlags: about 1 byte/row;

• ColumnSorter’s inline permutation: approximately 8 bytes/row for INT32, 16 bytes/row for INT64, 24 bytes/row for strings, and potentially 32 bytes/row for Decimal128 due to alignment;

• sorted_rows: another N * sizeof(shared_ptr), normally 16 bytes/row, while the original _row_in_blocks storage is still alive.

For the 7.05M-row benchmark mentioned in the PR, the old sort-specific temporary allocation was only about 7 MB. The new peak is approximately:

• at least 169 MB during the perm + sorted_rows phase;

• around 176 MB for an INT64 key during ColumnSorter;

• around 233 MB for a string key;

• potentially around 289 MB for a Decimal128 key.

The inline permutation is released between key columns, so these numbers do not multiply by the number of keys, but the peak is still much larger than before. Concurrent MemTable flushes could amplify this and potentially hit the load/process memory limit.

[liaoxin01]

Thanks, the arithmetic is right and I reproduced every term. Two things: one of them I removed, and I want to put the rest next to what the memtable already spends.

Removed: sorted_rows (16 B/row). That one was avoidable and is gone in the latest commit. The permutation is now applied to _row_in_blocks in place by following its cycles, using perm itself as the "already moved" marker since it is dead afterwards. That is also one move per row instead of two (gather into sorted_rows, then std::move back), so it is not a trade.

The rest is what it is. perm is 8 B/row because IColumn::Permutation is PaddedPODArray<size_t>, and the inline permutation is the whole point of the change -- the speedup comes from the key sitting next to the row id. EqualFlags is 1 B/row. Note the inline permutation is a std::vector local to ColumnSorter::_sort_by_inline_permutation, so it is released between key columns and the peak holds one of them, as you said.

Peak for the 7.05M-row shape, before and after this commit:

key type	before	after
INT32	169 MB	120 MB
INT64	176 MB	176 MB
string	233 MB	233 MB
Decimal128	289 MB	289 MB

So it only moves the needle where sorted_rows was the peak; for wider keys the inline permutation dominates and the number is unchanged.

Context for the remainder. _row_in_blocks holds one shared_ptr (16 B) plus one make_shared<RowInBlock> allocation per row -- sizeof(RowInBlock) is 40, and libstdc++ puts a 16 B control block in front of it, so 72 B/row. At 7.05M rows that is ~507 MB, resident for the whole life of the memtable rather than transient. Against that, a 120-289 MB transient during _sort() is real but it is not the term that decides whether a load fits.

It is also accounted: to_block() runs under SCOPED_SWITCH_THREAD_MEM_TRACKER_LIMITER(write_tracker) + SCOPED_CONSUME_MEM_TRACKER(memtable->mem_tracker()) in MemTableFlushExecutor, so these allocations land on the load tracker and MemTableMemoryLimiter sees them.

If the peak is still the blocker, the honest fix is not to shave the sort but to drop _row_in_blocks in favour of a plain IColumn::Permutation -- it exists mostly to carry _row_pos, and the agg-state fields are only used on the aggregating paths. That removes ~450 MB of the 507 MB above and makes the sort temporaries look small. I would rather do that as its own change than fold it in here. Happy to hold this PR until that one lands if you prefer that order.

[liaoxin01]

run buildall

[hello-stephen]

TPC-H: Total hot run time: 28683 ms

machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/tpch-tools
Tpch sf100 test result on commit 3568ae06792288230df01f08123fb0f06fb70ca1, data reload: false

------ Round 1 ----------------------------------
============================================
q1	17614	4000	3942	3942
q2	2009	320	200	200
q3	10274	1350	801	801
q4	4680	473	344	344
q5	7512	842	550	550
q6	187	171	140	140
q7	737	821	591	591
q8	9872	1753	1590	1590
q9	5802	4095	4094	4094
q10	6792	1641	1372	1372
q11	501	346	314	314
q12	761	574	455	455
q13	18076	3343	2718	2718
q14	262	257	235	235
q15	q16	737	721	654	654
q17	1001	944	995	944
q18	6553	5630	5548	5548
q19	1272	1310	1022	1022
q20	774	684	586	586
q21	5635	2473	2284	2284
q22	422	360	299	299
Total cold run time: 101473 ms
Total hot run time: 28683 ms

----- Round 2, with runtime_filter_mode=off -----
============================================
q1	4299	4212	4207	4207
q2	283	334	208	208
q3	4575	4927	4346	4346
q4	2172	2252	1436	1436
q5	4251	4188	4105	4105
q6	229	180	126	126
q7	1814	1621	1423	1423
q8	2796	2087	2095	2087
q9	7264	7364	7233	7233
q10	4243	4273	3930	3930
q11	547	395	360	360
q12	703	752	511	511
q13	3215	3646	2915	2915
q14	287	313	298	298
q15	q16	743	726	636	636
q17	1313	1324	1313	1313
q18	12176	11077	11904	11077
q19	1251	1161	1150	1150
q20	2230	2256	1945	1945
q21	5718	4880	4627	4627
q22	503	459	433	433
Total cold run time: 60612 ms
Total hot run time: 54366 ms

[hello-stephen]

TPC-DS: Total hot run time: 158329 ms

machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/tpcds-tools
TPC-DS sf100 test result on commit 3568ae06792288230df01f08123fb0f06fb70ca1, data reload: false

query5	4314	610	449	449
query6	460	214	200	200
query7	4837	600	357	357
query8	328	156	144	144
query9	8755	4033	4014	4014
query10	474	376	282	282
query11	5819	2185	2014	2014
query12	159	97	96	96
query13	1251	567	413	413
query14	6035	4264	3966	3966
query14_1	3785	3765	3799	3765
query15	200	189	175	175
query16	969	475	440	440
query17	909	711	545	545
query18	2423	468	343	343
query19	198	190	141	141
query20	103	101	100	100
query21	223	157	135	135
query22	12942	12905	12881	12881
query23	15669	14958	14531	14531
query23_1	14609	14626	14633	14626
query24	7451	1688	1208	1208
query24_1	1237	1217	1237	1217
query25	516	421	365	365
query26	1323	356	220	220
query27	2610	586	368	368
query28	4542	2030	2023	2023
query29	1059	588	472	472
query30	344	264	227	227
query31	1185	1124	1044	1044
query32	111	65	63	63
query33	530	324	250	250
query34	1205	1202	637	637
query35	739	749	650	650
query36	768	771	719	719
query37	156	108	94	94
query38	1827	1767	1658	1658
query39	820	824	808	808
query39_1	798	786	777	777
query40	257	171	146	146
query41	72	71	72	71
query42	100	98	95	95
query43	320	327	282	282
query44	1420	777	782	777
query45	186	181	180	180
query46	1086	1170	729	729
query47	1571	1515	1458	1458
query48	430	442	323	323
query49	600	423	305	305
query50	1092	434	328	328
query51	10647	10724	10721	10721
query52	94	96	77	77
query53	271	280	206	206
query54	293	244	252	244
query55	81	74	68	68
query56	324	343	320	320
query57	1001	1006	942	942
query58	316	267	271	267
query59	1514	1598	1401	1401
query60	320	287	267	267
query61	169	183	170	170
query62	400	324	277	277
query63	243	204	207	204
query64	3003	1156	814	814
query65	3832	3810	3814	3810
query66	1836	474	351	351
query67	19822	20086	20129	20086
query68	3137	1616	1036	1036
query69	404	297	268	268
query70	870	785	781	781
query71	352	328	312	312
query72	2964	2585	2234	2234
query73	835	777	441	441
query74	4601	4504	4297	4297
query75	2364	2331	1989	1989
query76	2334	1162	732	732
query77	332	364	271	271
query78	11110	10981	10636	10636
query79	1520	1140	780	780
query80	1261	529	478	478
query81	532	327	277	277
query82	646	171	137	137
query83	372	324	297	297
query84	325	162	141	141
query85	940	592	508	508
query86	403	229	227	227
query87	1997	1966	1839	1839
query88	3717	2796	2782	2782
query89	383	320	279	279
query90	1890	207	189	189
query91	203	191	167	167
query92	62	62	58	58
query93	1707	1594	1039	1039
query94	701	337	317	317
query95	790	613	469	469
query96	1087	814	345	345
query97	2438	2445	2336	2336
query98	197	188	179	179
query99	738	735	627	627
Total cold run time: 244407 ms
Total hot run time: 158329 ms

[hello-stephen]

ClickBench: Total hot run time: 23.68 s

machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/clickbench-tools
ClickBench test result on commit 3568ae06792288230df01f08123fb0f06fb70ca1, data reload: false

query1	0.01	0.00	0.00
query2	0.09	0.05	0.05
query3	0.25	0.13	0.14
query4	1.62	0.14	0.14
query5	0.24	0.21	0.21
query6	1.15	0.80	0.82
query7	0.04	0.01	0.01
query8	0.06	0.04	0.04
query9	0.37	0.34	0.32
query10	0.54	0.53	0.57
query11	0.19	0.13	0.14
query12	0.17	0.14	0.14
query13	0.46	0.47	0.47
query14	0.99	0.99	1.00
query15	0.61	0.58	0.58
query16	0.31	0.32	0.32
query17	1.09	1.12	1.06
query18	0.21	0.19	0.20
query19	2.01	1.88	1.91
query20	0.02	0.01	0.01
query21	15.41	0.21	0.14
query22	4.87	0.06	0.05
query23	16.09	0.32	0.13
query24	2.94	0.42	0.32
query25	0.11	0.04	0.04
query26	0.73	0.22	0.15
query27	0.04	0.03	0.04
query28	3.45	0.74	0.34
query29	12.46	3.97	3.14
query30	0.27	0.14	0.14
query31	2.76	0.56	0.31
query32	3.22	0.58	0.48
query33	3.19	3.20	3.19
query34	15.61	3.93	3.28
query35	3.22	3.20	3.24
query36	0.56	0.43	0.41
query37	0.09	0.07	0.06
query38	0.04	0.04	0.03
query39	0.04	0.02	0.02
query40	0.18	0.15	0.14
query41	0.08	0.03	0.03
query42	0.03	0.03	0.02
query43	0.04	0.04	0.03
Total cold run time: 95.86 s
Total hot run time: 23.68 s

[hello-stephen]

BE Regression && UT Coverage Report

Increment line coverage 97.26% (71/73) 🎉

Increment coverage report
Complete coverage report

Category	Coverage
Function Coverage	67.44% (28681/42531)
Line Coverage	52.11% (310393/595676)
Region Coverage	48.58% (257510/530103)
Branch Coverage	50.13% (117216/233834)

[liaoxin01]

run buildall

[hello-stephen]

TPC-H: Total hot run time: 29165 ms

machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/tpch-tools
Tpch sf100 test result on commit 6983af10e01f37dcda75c316b13f5d50019e766f, data reload: false

------ Round 1 ----------------------------------
============================================
q1	17629	3995	3953	3953
q2	2064	342	199	199
q3	10243	1383	793	793
q4	4684	480	332	332
q5	7495	817	558	558
q6	186	171	138	138
q7	753	825	603	603
q8	9338	1662	1589	1589
q9	5382	4106	4035	4035
q10	6734	1603	1372	1372
q11	520	356	328	328
q12	713	579	462	462
q13	18087	3340	2756	2756
q14	262	257	237	237
q15	q16	740	732	656	656
q17	1029	1023	1010	1010
q18	6520	5648	5604	5604
q19	1177	1285	1087	1087
q20	819	643	612	612
q21	5712	2577	2528	2528
q22	439	353	313	313
Total cold run time: 100526 ms
Total hot run time: 29165 ms

----- Round 2, with runtime_filter_mode=off -----
============================================
q1	4297	4199	4189	4189
q2	280	329	209	209
q3	4575	4953	4422	4422
q4	2194	2238	1385	1385
q5	4218	4148	4091	4091
q6	224	170	125	125
q7	1692	1619	1405	1405
q8	2764	2215	2090	2090
q9	7367	7445	7510	7445
q10	4362	4294	3882	3882
q11	546	408	365	365
q12	717	746	526	526
q13	3281	3518	3028	3028
q14	306	291	278	278
q15	q16	686	710	631	631
q17	1316	1305	1279	1279
q18	12229	11010	11777	11010
q19	1170	1213	1144	1144
q20	2210	2220	1938	1938
q21	5595	4865	4860	4860
q22	589	487	418	418
Total cold run time: 60618 ms
Total hot run time: 54720 ms

[liaoxin01]

Splitting this in two, since the memory question and the sort question are independent and the memory one deserves to be judged on its own.

This PR is now only the row diet: RowInBlock goes from a shared_ptr with a make_shared per row to a 24 byte value stored in the vector, which is 564 MB -> 169 MB for a 7.05M row memtable. It no longer touches the sort at all, so it is a straight reduction against master rather than something that has to be netted against a regression.

The ColumnSorter change moved to #66588, stacked on this one. Its inline permutation is what costs 8-32 bytes per row depending on key width, and it only comes out ahead of master once this PR has landed -- that ordering is now explicit in its description, along with the before/after peak for each key type.

Two things in here worth your eye, both consequences of dropping the shared_ptr rather than of the sort:

• _aggregate() now adopts temp_row_in_blocks unconditionally. Without that, a memtable that aggregates across a shrink_memtable_by_agg() round and then again in to_block() releases the same aggregate states twice. I confirmed it by instrumenting ~MemTable rather than reasoning about it -- it destroyed two states that _finalize_one_row<true>() had already released. The new shrink-round unit tests reach that path, but a release build does not reliably die on a double free, so ASAN is what would actually catch a regression there.
• The flexible-partial-update paths are the largest part of the diff and have no unit coverage, before or after. I went through them line by line against the previous semantics, but that is a reading, not a test.

[liaoxin01]

run buildall

[hello-stephen]

TPC-DS: Total hot run time: 158484 ms

machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/tpcds-tools
TPC-DS sf100 test result on commit 6983af10e01f37dcda75c316b13f5d50019e766f, data reload: false

query5	4309	595	450	450
query6	465	248	202	202
query7	4896	601	333	333
query8	320	162	142	142
query9	8771	4087	4068	4068
query10	478	338	301	301
query11	5814	2218	1992	1992
query12	153	97	95	95
query13	1264	599	414	414
query14	6039	4260	3995	3995
query14_1	3927	3824	3763	3763
query15	195	189	176	176
query16	975	470	424	424
query17	889	671	539	539
query18	2419	455	333	333
query19	201	181	148	148
query20	102	101	105	101
query21	231	159	133	133
query22	13011	13105	12931	12931
query23	15638	14987	14551	14551
query23_1	15434	15017	15190	15017
query24	7682	1725	1260	1260
query24_1	1250	1254	1226	1226
query25	564	444	386	386
query26	1324	365	229	229
query27	2566	566	376	376
query28	4598	2063	2016	2016
query29	1071	632	484	484
query30	338	261	219	219
query31	1169	1118	1047	1047
query32	106	60	58	58
query33	533	313	259	259
query34	1166	1174	639	639
query35	746	752	644	644
query36	768	761	715	715
query37	155	105	92	92
query38	1821	1771	1698	1698
query39	809	806	799	799
query39_1	775	813	781	781
query40	248	164	142	142
query41	67	67	69	67
query42	95	90	90	90
query43	320	326	274	274
query44	1428	789	780	780
query45	181	174	167	167
query46	1046	1195	735	735
query47	1543	1534	1391	1391
query48	432	429	318	318
query49	596	399	293	293
query50	1047	423	356	356
query51	10659	10622	10284	10284
query52	88	89	77	77
query53	278	281	197	197
query54	297	256	250	250
query55	122	72	71	71
query56	311	305	279	279
query57	1002	974	927	927
query58	302	265	258	258
query59	1537	1578	1374	1374
query60	317	275	247	247
query61	149	154	151	151
query62	390	322	267	267
query63	228	190	194	190
query64	2872	1037	863	863
query65	3879	3812	3805	3805
query66	1840	471	344	344
query67	20069	20188	19939	19939
query68	3033	1566	1014	1014
query69	405	347	266	266
query70	872	808	794	794
query71	386	318	307	307
query72	2972	2613	2342	2342
query73	856	755	455	455
query74	4601	4495	4281	4281
query75	2364	2352	2000	2000
query76	2297	1167	767	767
query77	334	371	279	279
query78	11148	11119	10511	10511
query79	1425	1069	774	774
query80	1278	570	461	461
query81	549	321	277	277
query82	689	174	138	138
query83	394	340	309	309
query84	317	158	131	131
query85	964	607	511	511
query86	413	228	230	228
query87	1977	1955	1828	1828
query88	3721	2834	2789	2789
query89	409	327	273	273
query90	1915	200	190	190
query91	221	199	169	169
query92	66	58	59	58
query93	1670	1615	1004	1004
query94	707	353	313	313
query95	834	501	495	495
query96	1098	802	360	360
query97	2445	2440	2385	2385
query98	190	188	181	181
query99	736	731	611	611
Total cold run time: 245823 ms
Total hot run time: 158484 ms

[hello-stephen]

ClickBench: Total hot run time: 23.8 s

machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/clickbench-tools
ClickBench test result on commit 6983af10e01f37dcda75c316b13f5d50019e766f, data reload: false

query1	0.01	0.00	0.00
query2	0.10	0.05	0.05
query3	0.26	0.13	0.14
query4	1.61	0.14	0.13
query5	0.23	0.22	0.21
query6	1.16	0.81	0.81
query7	0.04	0.01	0.00
query8	0.06	0.04	0.03
query9	0.36	0.30	0.31
query10	0.54	0.55	0.58
query11	0.18	0.13	0.13
query12	0.18	0.14	0.14
query13	0.47	0.46	0.47
query14	0.98	0.99	0.99
query15	0.62	0.57	0.59
query16	0.32	0.32	0.32
query17	1.09	1.07	1.14
query18	0.21	0.19	0.20
query19	2.07	1.95	1.87
query20	0.02	0.02	0.01
query21	15.44	0.22	0.15
query22	4.76	0.06	0.06
query23	16.22	0.30	0.12
query24	2.78	0.43	0.32
query25	0.13	0.04	0.04
query26	0.71	0.20	0.15
query27	0.04	0.03	0.03
query28	3.53	0.75	0.37
query29	12.52	4.01	3.22
query30	0.26	0.15	0.18
query31	2.77	0.55	0.31
query32	3.22	0.59	0.49
query33	3.14	3.25	3.20
query34	15.71	3.93	3.25
query35	3.24	3.21	3.19
query36	0.58	0.44	0.43
query37	0.09	0.06	0.06
query38	0.05	0.04	0.04
query39	0.04	0.03	0.03
query40	0.16	0.15	0.16
query41	0.10	0.03	0.03
query42	0.04	0.03	0.02
query43	0.04	0.04	0.03
Total cold run time: 96.08 s
Total hot run time: 23.8 s

[hello-stephen]

TPC-H: Total hot run time: 29158 ms

machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/tpch-tools
Tpch sf100 test result on commit f73e312172a4884fe8d9a6ea53b124420e2164fb, data reload: false

------ Round 1 ----------------------------------
============================================
q1	17651	3978	3973	3973
q2	2097	330	208	208
q3	10236	1399	814	814
q4	4702	478	338	338
q5	7492	846	564	564
q6	186	177	144	144
q7	740	823	606	606
q8	9337	1651	1644	1644
q9	5384	4132	4090	4090
q10	6780	1618	1361	1361
q11	517	354	316	316
q12	745	596	456	456
q13	18127	3320	2721	2721
q14	265	256	245	245
q15	q16	739	729	667	667
q17	1027	960	1021	960
q18	6622	5621	5537	5537
q19	1329	1270	1117	1117
q20	813	695	577	577
q21	5855	2601	2521	2521
q22	440	356	299	299
Total cold run time: 101084 ms
Total hot run time: 29158 ms

----- Round 2, with runtime_filter_mode=off -----
============================================
q1	4358	4408	4259	4259
q2	276	320	237	237
q3	4605	4891	4408	4408
q4	2162	2258	1383	1383
q5	4298	4205	4155	4155
q6	229	172	129	129
q7	1684	1602	1406	1406
q8	2250	1928	1908	1908
q9	6765	6759	6800	6759
q10	4244	4211	3843	3843
q11	525	393	355	355
q12	703	713	502	502
q13	3004	3413	2813	2813
q14	288	297	255	255
q15	q16	671	686	606	606
q17	1272	1238	1253	1238
q18	12097	10984	11848	10984
q19	1094	1078	1096	1078
q20	2213	2206	1920	1920
q21	5312	4527	4571	4527
q22	523	455	415	415
Total cold run time: 58573 ms
Total hot run time: 53180 ms

[hello-stephen]

TPC-DS: Total hot run time: 158117 ms

machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/tpcds-tools
TPC-DS sf100 test result on commit f73e312172a4884fe8d9a6ea53b124420e2164fb, data reload: false

query5	4308	592	451	451
query6	467	221	196	196
query7	4829	589	314	314
query8	326	162	145	145
query9	8783	4078	4071	4071
query10	481	362	301	301
query11	5790	2194	2011	2011
query12	157	100	95	95
query13	1308	609	433	433
query14	6047	4267	3982	3982
query14_1	3821	3807	3819	3807
query15	202	193	178	178
query16	990	462	483	462
query17	920	684	545	545
query18	2472	471	345	345
query19	200	186	151	151
query20	101	105	101	101
query21	236	161	136	136
query22	13074	12993	12836	12836
query23	15689	14994	14578	14578
query23_1	14721	14633	14629	14629
query24	7550	1728	1212	1212
query24_1	1261	1219	1224	1219
query25	555	459	386	386
query26	1308	367	213	213
query27	2591	606	367	367
query28	4582	2030	1988	1988
query29	1086	634	497	497
query30	353	263	217	217
query31	1176	1121	1057	1057
query32	111	65	63	63
query33	525	313	272	272
query34	1175	1198	670	670
query35	732	742	628	628
query36	789	764	714	714
query37	162	108	94	94
query38	1831	1755	1664	1664
query39	825	812	801	801
query39_1	770	787	784	784
query40	257	170	143	143
query41	66	64	64	64
query42	93	93	92	92
query43	323	319	275	275
query44	1450	772	778	772
query45	182	176	170	170
query46	1033	1198	712	712
query47	1531	1530	1440	1440
query48	392	400	300	300
query49	585	424	307	307
query50	1118	429	348	348
query51	10507	10765	10312	10312
query52	87	91	77	77
query53	252	276	201	201
query54	277	236	233	233
query55	79	72	67	67
query56	285	307	294	294
query57	1037	978	949	949
query58	297	254	249	249
query59	1517	1576	1430	1430
query60	310	290	247	247
query61	151	144	147	144
query62	393	317	266	266
query63	237	204	207	204
query64	2840	1023	846	846
query65	3880	3815	3788	3788
query66	1862	480	346	346
query67	20210	20086	19887	19887
query68	3422	1582	997	997
query69	404	315	266	266
query70	872	794	821	794
query71	385	349	317	317
query72	3257	2637	2410	2410
query73	808	791	435	435
query74	4641	4522	4304	4304
query75	2381	2353	2003	2003
query76	2406	1161	756	756
query77	356	381	292	292
query78	11017	11061	10530	10530
query79	1251	1112	770	770
query80	672	585	484	484
query81	453	322	289	289
query82	420	173	144	144
query83	414	336	307	307
query84	328	164	138	138
query85	983	612	536	536
query86	326	235	231	231
query87	2000	1984	1836	1836
query88	3754	2824	2792	2792
query89	386	322	281	281
query90	1905	192	199	192
query91	202	189	159	159
query92	60	59	54	54
query93	1654	1555	1036	1036
query94	532	356	323	323
query95	806	495	588	495
query96	1003	846	370	370
query97	2474	2429	2317	2317
query98	204	185	185	185
query99	745	730	619	619
Total cold run time: 244358 ms
Total hot run time: 158117 ms

[hello-stephen]

ClickBench: Total hot run time: 24.01 s

machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/clickbench-tools
ClickBench test result on commit f73e312172a4884fe8d9a6ea53b124420e2164fb, data reload: false

query1	0.01	0.00	0.00
query2	0.10	0.05	0.04
query3	0.25	0.14	0.14
query4	1.61	0.14	0.14
query5	0.23	0.22	0.22
query6	1.16	0.82	0.81
query7	0.04	0.01	0.00
query8	0.06	0.04	0.04
query9	0.39	0.33	0.35
query10	0.58	0.57	0.57
query11	0.19	0.13	0.14
query12	0.18	0.14	0.14
query13	0.47	0.46	0.46
query14	1.00	0.98	0.99
query15	0.60	0.58	0.58
query16	0.32	0.32	0.33
query17	1.11	1.11	1.15
query18	0.21	0.21	0.19
query19	2.02	1.94	1.95
query20	0.02	0.02	0.01
query21	15.41	0.24	0.14
query22	4.78	0.05	0.05
query23	16.14	0.30	0.13
query24	2.95	0.43	0.34
query25	0.10	0.05	0.05
query26	0.75	0.21	0.16
query27	0.05	0.04	0.03
query28	3.49	0.75	0.34
query29	12.48	4.00	3.21
query30	0.28	0.16	0.15
query31	2.77	0.56	0.32
query32	3.22	0.60	0.49
query33	3.27	3.17	3.31
query34	15.67	3.89	3.29
query35	3.17	3.19	3.23
query36	0.55	0.44	0.42
query37	0.10	0.06	0.06
query38	0.05	0.04	0.04
query39	0.04	0.03	0.03
query40	0.17	0.17	0.15
query41	0.09	0.04	0.04
query42	0.04	0.03	0.03
query43	0.04	0.03	0.04
Total cold run time: 96.16 s
Total hot run time: 24.01 s

[hello-stephen]

BE UT Coverage Report

Increment line coverage 55.32% (78/141) 🎉

Increment coverage report
Complete coverage report

Category	Coverage
Function Coverage	59.96% (26167/43644)
Line Coverage	44.30% (265802/600015)
Region Coverage	40.00% (211448/528606)
Branch Coverage	41.45% (96978/233954)

[hello-stephen]

BE Regression && UT Coverage Report

Increment line coverage 97.87% (138/141) 🎉

Increment coverage report
Complete coverage report

Category	Coverage
Function Coverage	76.07% (32367/42550)
Line Coverage	60.95% (363200/595927)
Region Coverage	57.57% (305293/530283)
Branch Coverage	59.02% (138062/233942)

[hello-stephen]

BE Regression && UT Coverage Report

Increment line coverage 97.87% (138/141) 🎉

Increment coverage report
Complete coverage report

Category	Coverage
Function Coverage	76.07% (32367/42550)
Line Coverage	60.95% (363199/595927)
Region Coverage	57.57% (305298/530283)
Branch Coverage	59.02% (138065/233942)