[improvement](segment) reduce memory usage when open segments (#46570)

[jacktengg]

Issue Number: close #xxx

BP #46570

Problem Summary:
When there are a lot of segments in one rowset, it will consume plenty of memory if open all the segments all at once. This PR open segments one by one and release the Segment object immediately if it's not need to be kept for later use, thus reduce memory footprints dramatically.

What problem does this PR solve?

Issue Number: close #xxx

Related PR: #xxx

Problem Summary:

Release note

None

Check List (For Author)

• Test

  • Regression test
  • Unit Test
  • Manual test (add detailed scripts or steps below)
  • No need to test or manual test. Explain why:
    • This is a refactor/code format and no logic has been changed.
    • Previous test can cover this change.
    • No code files have been changed.
    • Other reason

• Behavior changed:

  • No.
  • Yes.

• Does this need documentation?

  • No.
  • Yes.

Check List (For Reviewer who merge this PR)

• Confirm the release note
• Confirm test cases
• Confirm document
• Add branch pick label

[Thearas]

Thank you for your contribution to Apache Doris.
Don't know what should be done next? See How to process your PR.

Please clearly describe your PR:

1. What problem was fixed (it's best to include specific error reporting information). How it was fixed.
2. Which behaviors were modified. What was the previous behavior, what is it now, why was it modified, and what possible impacts might there be.
3. What features were added. Why was this function added?
4. Which code was refactored and why was this part of the code refactored?
5. Which functions were optimized and what is the difference before and after the optimization?

[jacktengg]

run buildall

[doris-robot]

TPC-H: Total hot run time: 41498 ms

machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/tpch-tools
Tpch sf100 test result on commit f52f8fd4ca16f892d954ac4bef9f8b06cfd4e556, data reload: false

------ Round 1 ----------------------------------
q1	17583	7794	7621	7621
q2	2064	177	167	167
q3	10615	1110	1154	1110
q4	10564	770	717	717
q5	7742	2876	2840	2840
q6	243	152	150	150
q7	980	615	610	610
q8	9347	1996	2077	1996
q9	6552	6418	6394	6394
q10	7049	2318	2317	2317
q11	461	263	253	253
q12	405	221	217	217
q13	17766	3041	3083	3041
q14	251	213	218	213
q15	589	527	538	527
q16	674	614	621	614
q17	1012	589	593	589
q18	7386	6791	6594	6594
q19	1392	1053	1079	1053
q20	497	209	202	202
q21	4274	3445	3278	3278
q22	1115	995	998	995
Total cold run time: 108561 ms
Total hot run time: 41498 ms

----- Round 2, with runtime_filter_mode=off -----
q1	7567	7605	7515	7515
q2	333	238	240	238
q3	3074	2990	2970	2970
q4	2139	1885	1826	1826
q5	5768	5804	5799	5799
q6	224	152	152	152
q7	2244	1806	1814	1806
q8	3446	3574	3478	3478
q9	8966	8909	8871	8871
q10	3624	3580	3605	3580
q11	609	528	520	520
q12	841	632	592	592
q13	9816	3225	3175	3175
q14	333	275	303	275
q15	599	526	526	526
q16	737	679	680	679
q17	1867	1659	1614	1614
q18	8322	7899	7533	7533
q19	1748	1519	1477	1477
q20	2089	1906	1887	1887
q21	5540	5472	5473	5472
q22	1128	1060	1066	1060
Total cold run time: 71014 ms
Total hot run time: 61045 ms

[doris-robot]

TPC-DS: Total hot run time: 198471 ms

machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/tpcds-tools
TPC-DS sf100 test result on commit f52f8fd4ca16f892d954ac4bef9f8b06cfd4e556, data reload: false

query1	1269	948	927	927
query2	6254	2064	2070	2064
query3	10905	4549	4483	4483
query4	66574	29279	23525	23525
query5	4840	468	458	458
query6	417	188	174	174
query7	5512	320	314	314
query8	296	215	233	215
query9	8738	2667	2655	2655
query10	462	289	255	255
query11	17220	15512	15774	15512
query12	154	103	102	102
query13	1445	436	446	436
query14	10355	7285	7477	7285
query15	200	182	188	182
query16	7097	455	497	455
query17	1064	567	589	567
query18	1936	316	333	316
query19	214	164	155	155
query20	118	115	108	108
query21	204	104	107	104
query22	4617	4579	4421	4421
query23	34583	34094	34585	34094
query24	6198	2986	2959	2959
query25	530	427	435	427
query26	657	178	173	173
query27	1936	351	359	351
query28	4361	2467	2462	2462
query29	732	486	475	475
query30	254	169	167	167
query31	999	846	854	846
query32	72	60	60	60
query33	446	307	290	290
query34	910	508	536	508
query35	848	749	734	734
query36	1095	955	985	955
query37	134	85	70	70
query38	4288	4055	4019	4019
query39	1529	1498	1477	1477
query40	214	109	107	107
query41	52	51	51	51
query42	120	104	102	102
query43	564	516	501	501
query44	1202	851	859	851
query45	194	170	171	170
query46	1176	756	721	721
query47	1977	1892	1916	1892
query48	472	377	389	377
query49	734	397	390	390
query50	864	430	436	430
query51	7410	7134	7153	7134
query52	98	90	87	87
query53	272	180	184	180
query54	559	443	451	443
query55	76	75	77	75
query56	251	239	238	238
query57	1256	1130	1114	1114
query58	208	209	205	205
query59	3083	3026	3078	3026
query60	283	260	250	250
query61	111	104	114	104
query62	789	685	667	667
query63	218	197	198	197
query64	1402	671	641	641
query65	3282	3260	3210	3210
query66	747	303	304	303
query67	15991	15596	15469	15469
query68	4137	579	579	579
query69	437	269	269	269
query70	1156	1146	1136	1136
query71	325	258	256	256
query72	6388	4145	4098	4098
query73	780	351	348	348
query74	10410	9036	8970	8970
query75	3382	2667	2667	2667
query76	1821	1059	1161	1059
query77	465	274	268	268
query78	10534	9610	9607	9607
query79	1417	597	584	584
query80	891	427	438	427
query81	516	239	238	238
query82	1290	117	114	114
query83	246	147	144	144
query84	284	87	76	76
query85	873	304	302	302
query86	351	303	277	277
query87	4440	4428	4278	4278
query88	3555	2398	2364	2364
query89	419	293	292	292
query90	2038	191	186	186
query91	184	147	151	147
query92	71	50	48	48
query93	1877	547	545	545
query94	893	289	297	289
query95	356	253	263	253
query96	635	278	273	273
query97	3321	3199	3160	3160
query98	227	202	195	195
query99	1570	1327	1307	1307
Total cold run time: 318357 ms
Total hot run time: 198471 ms

[doris-robot]

ClickBench: Total hot run time: 33.17 s

machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/clickbench-tools
ClickBench test result on commit f52f8fd4ca16f892d954ac4bef9f8b06cfd4e556, data reload: false

query1	0.03	0.03	0.03
query2	0.08	0.03	0.03
query3	0.24	0.06	0.07
query4	1.62	0.10	0.10
query5	0.54	0.53	0.52
query6	1.12	0.73	0.74
query7	0.02	0.02	0.01
query8	0.05	0.04	0.03
query9	0.56	0.51	0.50
query10	0.53	0.53	0.56
query11	0.14	0.10	0.10
query12	0.15	0.11	0.11
query13	0.61	0.60	0.60
query14	2.94	2.92	3.02
query15	0.91	0.83	0.82
query16	0.37	0.37	0.39
query17	1.04	1.00	1.03
query18	0.23	0.22	0.22
query19	1.91	1.82	2.06
query20	0.01	0.01	0.02
query21	15.37	0.59	0.56
query22	2.78	2.57	2.15
query23	17.08	0.90	0.81
query24	3.02	0.86	1.48
query25	0.20	0.14	0.04
query26	0.58	0.14	0.14
query27	0.06	0.04	0.05
query28	10.67	1.09	1.07
query29	12.59	3.32	3.27
query30	0.25	0.05	0.05
query31	2.87	0.39	0.38
query32	3.24	0.49	0.46
query33	2.97	2.99	3.08
query34	17.15	4.54	4.50
query35	4.56	4.53	4.54
query36	0.67	0.49	0.50
query37	0.09	0.06	0.06
query38	0.04	0.04	0.04
query39	0.04	0.02	0.02
query40	0.16	0.13	0.13
query41	0.08	0.03	0.02
query42	0.04	0.02	0.02
query43	0.04	0.03	0.03
Total cold run time: 107.65 s
Total hot run time: 33.17 s

[airborne12]

Is there any chance that tmp_segments is empty?

[jacktengg]

If it reaches line 83, it means SegmentLoader::instance()->load_segment returns OK and it assures segment_cache_handle.get_segments() is not empty.

[github-actions]

We're closing this PR because it hasn't been updated in a while.
This isn't a judgement on the merit of the PR in any way. It's just a way of keeping the PR queue manageable.
If you'd like to revive this PR, please reopen it and feel free a maintainer to remove the Stale tag!