Skip to content

[debug](exchange) checks on both the sending and receiving sides of the exchange. #57346

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom
Merged
BiteTheDDDDt merged 2 commits into from
Oct 28, 2025
Merged

[debug](exchange) checks on both the sending and receiving sides of the exchange. #57346

BiteTheDDDDt merged 2 commits into from
Oct 28, 2025

Conversation

@Mryange
Copy link
Contributor

@Mryange Mryange commented Oct 27, 2025

What problem does this PR solve?

We've previously had sporadic core dumps.
They can be clearly linked to the send_multi_blocks functionality.
However, it's hard to reproduce when compiled with ASAN, so we've added some checking code here (because we suspect memory has already been corrupted).
Please do not pick this into any release branches!!!

Release note

None

Check List (For Author)

  • Test

    • Regression test
    • Unit Test
    • Manual test (add detailed scripts or steps below)
    • No need to test or manual test. Explain why:
      • This is a refactor/code format and no logic has been changed.
      • Previous test can cover this change.
      • No code files have been changed.
      • Other reason

  • Behavior changed:

    • No.
    • Yes.

  • Does this need documentation?

    • No.
    • Yes.

Check List (For Reviewer who merge this PR)

  • Confirm the release note
  • Confirm test cases
  • Confirm document
  • Add branch pick label

@Mryange
Copy link
Contributor Author

Mryange commented Oct 27, 2025

run buildall

HappenLee
HappenLee approved these changes Oct 27, 2025
View reviewed changes
Copy link
Contributor

@HappenLee HappenLee left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM

@github-actions github-actions bot added the approved Indicates a PR has been approved by one committer. label Oct 27, 2025
@github-actions
Copy link
Contributor

github-actions bot commented Oct 27, 2025

PR approved by at least one committer and no changes requested.

@github-actions
Copy link
Contributor

github-actions bot commented Oct 27, 2025

PR approved by anyone and no changes requested.

@Mryange
Copy link
Contributor Author

Mryange commented Oct 28, 2025

run buildall

@doris-robot
Copy link

doris-robot commented Oct 28, 2025

TPC-DS: Total hot run time: 190355 ms 
machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/tpcds-tools
TPC-DS sf100 test result on commit 6f8addc1a7c0b63b03f4036c86d2108f785d0cc9, data reload: false

query1	1090	419	397	397
query2	6565	1713	1742	1713
query3	6754	223	221	221
query4	26776	23856	23540	23540
query5	4976	685	497	497
query6	355	266	234	234
query7	4662	502	307	307
query8	320	278	268	268
query9	8729	2615	2595	2595
query10	535	371	282	282
query11	15866	15066	14824	14824
query12	212	122	122	122
query13	1686	575	467	467
query14	12107	9258	9127	9127
query15	220	198	182	182
query16	7680	670	522	522
query17	1626	775	683	683
query18	2550	452	335	335
query19	241	231	194	194
query20	140	137	135	135
query21	216	138	122	122
query22	4694	4712	4568	4568
query23	34824	34002	33829	33829
query24	8520	2544	2774	2544
query25	618	568	498	498
query26	1315	298	173	173
query27	2753	529	388	388
query28	4396	2264	2220	2220
query29	953	671	556	556
query30	303	230	209	209
query31	968	831	793	793
query32	89	87	71	71
query33	629	379	347	347
query34	858	884	537	537
query35	792	816	745	745
query36	933	974	896	896
query37	127	109	83	83
query38	3535	3472	3505	3472
query39	1471	1420	1403	1403
query40	219	125	116	116
query41	61	56	56	56
query42	127	111	108	108
query43	489	500	464	464
query44	1215	741	746	741
query45	185	179	174	174
query46	889	998	641	641
query47	1754	1791	1721	1721
query48	401	428	330	330
query49	757	531	432	432
query50	669	697	407	407
query51	3904	3925	3792	3792
query52	115	111	102	102
query53	233	259	202	202
query54	616	601	526	526
query55	90	85	84	84
query56	322	323	311	311
query57	1163	1178	1148	1148
query58	290	284	280	280
query59	2639	2646	2487	2487
query60	369	337	335	335
query61	158	153	163	153
query62	809	741	670	670
query63	225	202	197	197
query64	4398	1186	870	870
query65	4084	3938	3965	3938
query66	1084	445	337	337
query67	15501	15194	15147	15147
query68	8061	889	601	601
query69	503	339	301	301
query70	1297	1302	1258	1258
query71	512	342	309	309
query72	5787	4980	4943	4943
query73	670	571	360	360
query74	9217	9077	8718	8718
query75	4010	3310	2852	2852
query76	3794	1210	721	721
query77	829	416	319	319
query78	9636	9570	8938	8938
query79	2475	858	594	594
query80	657	564	491	491
query81	522	265	220	220
query82	503	163	137	137
query83	314	277	254	254
query84	303	116	92	92
query85	941	503	459	459
query86	403	311	320	311
query87	3774	3757	3614	3614
query88	3876	2298	2250	2250
query89	395	334	317	317
query90	1976	213	216	213
query91	163	168	139	139
query92	79	65	64	64
query93	2088	997	655	655
query94	702	449	347	347
query95	417	352	307	307
query96	501	581	287	287
query97	2966	2972	2890	2890
query98	243	217	217	217
query99	1467	1424	1309	1309
Total cold run time: 282931 ms
Total hot run time: 190355 ms

@doris-robot
Copy link

doris-robot commented Oct 28, 2025

ClickBench: Total hot run time: 27.85 s 
machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/clickbench-tools
ClickBench test result on commit 6f8addc1a7c0b63b03f4036c86d2108f785d0cc9, data reload: false

query1	0.06	0.05	0.05
query2	0.10	0.05	0.06
query3	0.25	0.08	0.09
query4	1.60	0.12	0.12
query5	0.28	0.26	0.25
query6	1.16	0.66	0.63
query7	0.03	0.03	0.03
query8	0.06	0.04	0.04
query9	0.62	0.53	0.52
query10	0.58	0.57	0.59
query11	0.16	0.11	0.12
query12	0.16	0.13	0.12
query13	0.63	0.60	0.61
query14	1.03	1.02	1.01
query15	0.86	0.84	0.87
query16	0.39	0.39	0.38
query17	1.03	1.04	1.04
query18	0.22	0.19	0.19
query19	1.90	1.74	1.81
query20	0.02	0.01	0.01
query21	15.44	0.17	0.14
query22	5.14	0.07	0.05
query23	15.67	0.25	0.10
query24	2.75	0.77	1.33
query25	0.07	0.06	0.07
query26	0.15	0.13	0.14
query27	0.08	0.06	0.05
query28	6.00	1.20	0.94
query29	12.54	4.02	3.24
query30	0.28	0.14	0.12
query31	2.81	0.60	0.38
query32	3.23	0.57	0.47
query33	3.03	3.12	3.06
query34	15.95	5.29	4.55
query35	4.59	4.60	4.54
query36	0.68	0.52	0.50
query37	0.10	0.07	0.07
query38	0.07	0.05	0.04
query39	0.04	0.03	0.03
query40	0.18	0.15	0.15
query41	0.08	0.04	0.02
query42	0.04	0.04	0.03
query43	0.05	0.03	0.03
Total cold run time: 100.11 s
Total hot run time: 27.85 s

@hello-stephen
Copy link
Contributor

hello-stephen commented Oct 28, 2025

BE UT Coverage Report

Increment line coverage 22.22% (2/9) 🎉

Increment coverage report
Complete coverage report

Category Coverage
Function Coverage 52.76% (18056/34221)
Line Coverage 38.00% (163711/430873)
Region Coverage 32.39% (124872/385508)
Branch Coverage 33.74% (54572/161727)

@hello-stephen
Copy link
Contributor

hello-stephen commented Oct 28, 2025

BE Regression && UT Coverage Report

Increment line coverage 76.47% (13/17) 🎉

Increment coverage report
Complete coverage report

Category Coverage
Function Coverage 71.33% (23913/33526)
Line Coverage 57.77% (248790/430681)
Region Coverage 52.93% (206562/390273)
Branch Coverage 54.55% (88643/162510)

@hello-stephen
Copy link
Contributor

hello-stephen commented Oct 28, 2025

BE Regression && UT Coverage Report

Increment line coverage 76.47% (13/17) 🎉

Increment coverage report
Complete coverage report

Category Coverage
Function Coverage 71.31% (23909/33526)
Line Coverage 57.73% (248644/430681)
Region Coverage 52.88% (206370/390273)
Branch Coverage 54.52% (88594/162510)

1 similar comment
@hello-stephen
Copy link
Contributor

hello-stephen commented Oct 28, 2025

BE Regression && UT Coverage Report

Increment line coverage 76.47% (13/17) 🎉

Increment coverage report
Complete coverage report

Category Coverage
Function Coverage 71.31% (23909/33526)
Line Coverage 57.73% (248644/430681)
Region Coverage 52.88% (206370/390273)
Branch Coverage 54.52% (88594/162510)

@BiteTheDDDDt BiteTheDDDDt merged commit 04d53d4 into apache:master Oct 28, 2025
27 of 29 checks passed
dwdwqfwe pushed a commit to dwdwqfwe/doris that referenced this pull request Oct 31, 2025
@Mryange @dwdwqfwe
[debug](exchange) checks on both the sending and receiving sides of t…
e50a906 
…he exchange. (apache#57346)

We've previously had sporadic core dumps. 
They can be clearly linked to the send_multi_blocks functionality. 
However, it's hard to reproduce when compiled with ASAN, so we've added
some checking code here (because we suspect memory has already been
corrupted).
Please do not pick this into any release branches!!!
@Mryange Mryange mentioned this pull request Nov 5, 2025
Merged
zclllyybb pushed a commit that referenced this pull request Nov 6, 2025
@Mryange
Revert "[debug](exchange) checks on both the sending and receiving si…
37f2bb5 
…des of the exchange." (#57740)

Reverts #57346
We already know why the core issue is triggered—see the
#57735 description for details.

However, we want to fix the problem once and for all, so we'll complete
the TODO items directly in a subsequent PR.
wyxxxcat pushed a commit to wyxxxcat/doris that referenced this pull request Nov 18, 2025
@Mryange @wyxxxcat
Revert "[debug](exchange) checks on both the sending and receiving si…
1548c23 
…des of the exchange." (apache#57740)

Reverts apache#57346
We already know why the core issue is triggered—see the
apache#57735 description for details.

However, we want to fix the problem once and for all, so we'll complete
the TODO items directly in a subsequent PR.
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Reviewers

@BiteTheDDDDt BiteTheDDDDt BiteTheDDDDt approved these changes

@HappenLee HappenLee HappenLee approved these changes

Assignees

No one assigned

Labels

approved Indicates a PR has been approved by one committer. reviewed

Projects

None yet

Milestone

No milestone

Development

Successfully merging this pull request may close these issues.

5 participants

@Mryange @doris-robot @hello-stephen @BiteTheDDDDt @HappenLee