branch-3.1: [enhance](memtable) skip memtable in queue to speed up cancel #53481

[sollhui]

pick #53481

At present, if load/update/delete is cancelled, it needs to wait for all submitted memtable task of related FlushToken to be completed, included:

• flush task wait in thread pool queue
• ongoing flushing task

It is not need wait flush task still in the thread pool queue which can speed up cancel load/update/delete.

For example, cancel load1. The current logic is to wait for all submit task(queue+flushing), but for those in the queue, they have to wait for all the previously queued ones to complete before it's their turn. Skip memtable in queue can speed up cancel for:

• If there are still memtables for load2 and load3 in front of load1 memtable, cancel block until other load memtable been flushed.

• Under more extreme conditions, the submit task count may add when wait submit task finish.

What problem does this PR solve?

Issue Number: close #xxx

Related PR: #xxx

Problem Summary:

Release note

None

Check List (For Author)

• Test

  • Regression test
  • Unit Test
  • Manual test (add detailed scripts or steps below)
  • No need to test or manual test. Explain why:
    • This is a refactor/code format and no logic has been changed.
    • Previous test can cover this change.
    • No code files have been changed.
    • Other reason

• Behavior changed:

  • No.
  • Yes.

• Does this need documentation?

  • No.
  • Yes.

Check List (For Reviewer who merge this PR)

• Confirm the release note
• Confirm test cases
• Confirm document
• Add branch pick label

[hello-stephen]

Thank you for your contribution to Apache Doris.
Don't know what should be done next? See How to process your PR.

Please clearly describe your PR:

1. What problem was fixed (it's best to include specific error reporting information). How it was fixed.
2. Which behaviors were modified. What was the previous behavior, what is it now, why was it modified, and what possible impacts might there be.
3. What features were added. Why was this function added?
4. Which code was refactored and why was this part of the code refactored?
5. Which functions were optimized and what is the difference before and after the optimization?

[sollhui]

run buildall

[doris-robot]

TPC-H: Total hot run time: 32757 ms

machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/tpch-tools
Tpch sf100 test result on commit 84890c022124d307172586e27c72d49ab79d76e8, data reload: false

------ Round 1 ----------------------------------
q1	18264	5739	5463	5463
q2	2065	298	164	164
q3	10431	1232	751	751
q4	10203	888	466	466
q5	9045	2496	2202	2202
q6	192	184	140	140
q7	959	805	652	652
q8	11295	1481	1189	1189
q9	5306	4905	4958	4905
q10	6750	2289	1824	1824
q11	467	291	267	267
q12	338	365	219	219
q13	17782	3655	2991	2991
q14	239	238	206	206
q15	512	477	493	477
q16	422	432	378	378
q17	587	857	363	363
q18	7005	6308	6484	6308
q19	1204	956	559	559
q20	334	348	213	213
q21	3049	2270	2053	2053
q22	1067	1017	967	967
Total cold run time: 107516 ms
Total hot run time: 32757 ms

----- Round 2, with runtime_filter_mode=off -----
q1	5529	5553	5537	5537
q2	241	327	229	229
q3	2253	2687	2300	2300
q4	1339	1782	1369	1369
q5	4456	5074	4993	4993
q6	182	164	128	128
q7	2075	1950	1808	1808
q8	2616	2912	2677	2677
q9	7220	7322	7132	7132
q10	3038	3224	2799	2799
q11	576	522	497	497
q12	678	737	668	668
q13	3401	3832	3165	3165
q14	270	304	272	272
q15	513	467	470	467
q16	432	486	429	429
q17	1231	1766	1266	1266
q18	7645	7451	7403	7403
q19	819	979	1142	979
q20	2005	2070	1892	1892
q21	5475	5057	4471	4471
q22	1080	1087	1033	1033
Total cold run time: 53074 ms
Total hot run time: 51514 ms

[doris-robot]

TPC-DS: Total hot run time: 196718 ms

machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/tpcds-tools
TPC-DS sf100 test result on commit 84890c022124d307172586e27c72d49ab79d76e8, data reload: false

query1	1300	915	902	902
query2	6147	1947	1880	1880
query3	10929	4365	4505	4365
query4	33133	23655	23420	23420
query5	3510	613	481	481
query6	279	201	192	192
query7	3997	495	329	329
query8	306	259	246	246
query9	9300	2578	2571	2571
query10	505	324	260	260
query11	18100	15484	15263	15263
query12	157	106	105	105
query13	1562	529	420	420
query14	10008	6932	7214	6932
query15	234	198	195	195
query16	7960	662	506	506
query17	1654	744	621	621
query18	2099	438	315	315
query19	211	190	167	167
query20	125	118	118	118
query21	206	128	110	110
query22	4557	4572	4338	4338
query23	35137	34237	34307	34237
query24	7396	2676	2691	2676
query25	520	497	468	468
query26	1184	286	173	173
query27	2046	495	357	357
query28	5268	2259	2153	2153
query29	695	592	458	458
query30	236	191	166	166
query31	1023	928	827	827
query32	71	61	59	59
query33	510	371	310	310
query34	764	863	522	522
query35	797	847	740	740
query36	989	1073	948	948
query37	106	94	70	70
query38	4064	4003	4022	4003
query39	1550	1486	1478	1478
query40	205	117	104	104
query41	48	49	47	47
query42	125	99	101	99
query43	507	507	479	479
query44	1350	849	823	823
query45	188	178	174	174
query46	885	1052	669	669
query47	1995	1971	1895	1895
query48	409	418	347	347
query49	785	502	428	428
query50	699	720	437	437
query51	7394	7418	7198	7198
query52	106	99	95	95
query53	230	261	196	196
query54	552	563	471	471
query55	85	82	87	82
query56	262	261	252	252
query57	1266	1240	1224	1224
query58	240	227	224	224
query59	3144	3138	3116	3116
query60	295	298	281	281
query61	122	113	112	112
query62	818	755	676	676
query63	232	202	193	193
query64	4055	1019	703	703
query65	3328	3201	3211	3201
query66	937	408	309	309
query67	16025	15906	15476	15476
query68	4989	831	545	545
query69	502	296	269	269
query70	1154	1133	1070	1070
query71	391	290	263	263
query72	5630	3726	3771	3726
query73	638	737	349	349
query74	10505	9314	8937	8937
query75	3169	3140	2654	2654
query76	3179	1183	776	776
query77	470	371	284	284
query78	10464	10508	9606	9606
query79	2764	888	610	610
query80	658	529	438	438
query81	496	257	221	221
query82	290	122	100	100
query83	164	162	144	144
query84	246	102	81	81
query85	823	371	312	312
query86	403	313	292	292
query87	4264	4321	4253	4253
query88	4968	2402	2396	2396
query89	411	334	285	285
query90	1796	191	194	191
query91	146	139	110	110
query92	63	56	54	54
query93	2370	873	534	534
query94	656	396	311	311
query95	338	274	269	269
query96	484	595	293	293
query97	3176	3239	3161	3161
query98	226	202	200	200
query99	1394	1404	1292	1292
Total cold run time: 292149 ms
Total hot run time: 196718 ms

[doris-robot]

ClickBench: Total hot run time: 28.92 s

machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/clickbench-tools
ClickBench test result on commit 84890c022124d307172586e27c72d49ab79d76e8, data reload: false

query1	0.03	0.03	0.02
query2	0.06	0.03	0.03
query3	0.24	0.06	0.07
query4	1.62	0.11	0.11
query5	0.52	0.50	0.52
query6	1.13	0.74	0.72
query7	0.02	0.02	0.01
query8	0.05	0.04	0.03
query9	0.57	0.51	0.50
query10	0.55	0.54	0.55
query11	0.14	0.11	0.11
query12	0.14	0.10	0.11
query13	0.61	0.60	0.61
query14	0.79	0.82	0.81
query15	0.85	0.82	0.84
query16	0.39	0.38	0.38
query17	1.06	1.07	0.94
query18	0.23	0.21	0.22
query19	1.95	1.83	1.84
query20	0.02	0.01	0.01
query21	15.39	0.91	0.59
query22	0.75	0.88	0.71
query23	14.95	1.46	0.63
query24	3.51	1.03	1.42
query25	0.20	0.23	0.06
query26	0.22	0.14	0.13
query27	0.06	0.06	0.05
query28	13.77	1.02	0.44
query29	12.57	3.98	3.26
query30	0.25	0.08	0.06
query31	2.82	0.60	0.38
query32	3.22	0.53	0.46
query33	2.95	2.99	3.01
query34	16.62	5.17	4.53
query35	4.58	4.57	4.50
query36	0.66	0.50	0.47
query37	0.09	0.06	0.06
query38	0.04	0.03	0.03
query39	0.03	0.02	0.03
query40	0.17	0.13	0.12
query41	0.08	0.03	0.02
query42	0.04	0.02	0.02
query43	0.04	0.03	0.03
Total cold run time: 103.98 s
Total hot run time: 28.92 s