[BugFix] Fix exec state report lost lead to ingestion status getting stuck #36688

meegoo · 2023-12-08T06:34:58Z

Why I'm doing:

when there are more than 1000(queue size) exec report task , The thread pool will discard new tasks, which can result in the ingestion status getting stuck.

What I'm doing:

use a dedicate thread pool to handle done report task

Fixes #issue

What type of PR is this:

Does this PR entail a change in behavior?

Yes, this PR will result in a change in behavior.
No, this PR will not result in a change in behavior.

If yes, please specify the type of change:

Interface/UI changes: syntax, type conversion, expression evaluation, display information
Parameter changes: default values, similar parameters but with different default values
Policy changes: use new policy to replace old one, functionality automatically enabled
Feature removed
Miscellaneous: upgrade & downgrade compatibility, etc.

Checklist:

I have added test cases for my bug fix or my new feature
This pr needs user documentation (for new or modified features or behaviors)
- I have added documentation for my new feature or new function

Bugfix cherry-pick branch check:

be/src/exec/pipeline/exec_state_reporter.cpp

be/src/exec/pipeline/pipeline_driver_executor.cpp

be/src/exec/pipeline/exec_state_reporter.cpp

…stuck Signed-off-by: meegoo <meegoo.sr@gmail.com>

github-actions · 2024-01-05T03:30:57Z

[FE Incremental Coverage Report]

✅ pass : 0 / 0 (0%)

github-actions · 2024-01-05T03:32:58Z

[BE Incremental Coverage Report]

❌ fail : 9 / 28 (32.14%)

file detail

	path	covered_line	new_line	coverage	not_covered_line_detail
🔵	src/exec/pipeline/pipeline_driver_executor.cpp	0	17	00.00%	[323, 324, 325, 326, 327, 328, 329, 331, 332, 334, 335, 339, 340, 342, 349, 350, 351]
🔵	src/exec/pipeline/exec_state_reporter.cpp	9	11	81.82%	[312, 318]

be/src/exec/pipeline/exec_state_reporter.cpp

github-actions · 2024-01-05T11:57:15Z

@Mergifyio backport branch-3.2

github-actions · 2024-01-05T11:57:17Z

@Mergifyio backport branch-3.1

github-actions · 2024-01-05T11:57:18Z

@Mergifyio backport branch-3.0

github-actions · 2024-01-05T11:57:20Z

@Mergifyio backport branch-2.5

mergify · 2024-01-05T11:57:40Z

backport branch-3.2

✅ Backports have been created

#38564 [BugFix] Fix exec state report lost lead to ingestion status getting stuck (backport #36688) has been created for branch branch-3.2 but encountered conflicts

mergify · 2024-01-05T11:57:42Z

backport branch-3.1

✅ Backports have been created

#38565 [BugFix] Fix exec state report lost lead to ingestion status getting stuck (backport #36688) has been created for branch branch-3.1

mergify · 2024-01-05T11:57:44Z

backport branch-3.0

✅ Backports have been created

#38566 [BugFix] Fix exec state report lost lead to ingestion status getting stuck (backport #36688) has been created for branch branch-3.0 but encountered conflicts

mergify · 2024-01-05T11:57:46Z

backport branch-2.5

✅ Backports have been created

#38567 [BugFix] Fix exec state report lost lead to ingestion status getting stuck (backport #36688) has been created for branch branch-2.5 but encountered conflicts

…stuck (#36688) Signed-off-by: meegoo <meegoo.sr@gmail.com> (cherry picked from commit d088b3f) # Conflicts: # be/src/exec/pipeline/exec_state_reporter.cpp

…stuck (#36688) Signed-off-by: meegoo <meegoo.sr@gmail.com> (cherry picked from commit d088b3f)

…stuck (#36688) Signed-off-by: meegoo <meegoo.sr@gmail.com> (cherry picked from commit d088b3f) # Conflicts: # be/src/exec/pipeline/exec_state_reporter.cpp # be/src/exec/pipeline/exec_state_reporter.h

…stuck (backport #36688) (#38565) Co-authored-by: meegoo <meegoo.sr@gmail.com>

…stuck (backport #36688) (#38567) Signed-off-by: meegoo <meegoo.sr@gmail.com> Co-authored-by: meegoo <meegoo.sr@gmail.com>

…stuck (backport #36688) (#38564) Signed-off-by: meegoo <meegoo.sr@gmail.com> Co-authored-by: meegoo <meegoo.sr@gmail.com>

…stuck (backport #36688) (#38566) Signed-off-by: meegoo <meegoo.sr@gmail.com> Co-authored-by: meegoo <meegoo.sr@gmail.com>

…s getting stuck (backport StarRocks#36688) (StarRocks#38565) Co-authored-by: meegoo <meegoo.sr@gmail.com>

mergify bot assigned meegoo Dec 8, 2023

starrocks-cr bot reviewed Dec 8, 2023

View reviewed changes

be/src/exec/pipeline/exec_state_reporter.cpp Show resolved Hide resolved

starrocks-cr bot reviewed Dec 8, 2023

View reviewed changes

be/src/exec/pipeline/pipeline_driver_executor.cpp Outdated Show resolved Hide resolved

github-actions bot added 3.2 3.1 3.0 2.5 labels Dec 8, 2023

meegoo force-pushed the fix_report_lost branch 2 times, most recently from 680da23 to fd2f503 Compare December 8, 2023 06:53

meegoo marked this pull request as draft December 11, 2023 04:13

meegoo force-pushed the fix_report_lost branch 2 times, most recently from 5c9e6cc to ba32027 Compare December 12, 2023 00:12

meegoo marked this pull request as ready for review December 12, 2023 00:13

meegoo force-pushed the fix_report_lost branch from ba32027 to 26cfcdf Compare December 12, 2023 00:48

meegoo marked this pull request as draft December 12, 2023 04:33

meegoo force-pushed the fix_report_lost branch from 26cfcdf to 4003732 Compare December 13, 2023 01:39

meegoo marked this pull request as ready for review December 13, 2023 01:46

meegoo enabled auto-merge (squash) December 13, 2023 05:02

wyb reviewed Dec 20, 2023

View reviewed changes

be/src/exec/pipeline/pipeline_driver_executor.cpp Show resolved Hide resolved

be/src/exec/pipeline/pipeline_driver_executor.cpp Show resolved Hide resolved

be/src/exec/pipeline/exec_state_reporter.cpp Show resolved Hide resolved

[BugFix] Fix exec state report lost lead to ingestion status getting …

6a5363f

…stuck Signed-off-by: meegoo <meegoo.sr@gmail.com>

meegoo force-pushed the fix_report_lost branch from ef462bf to 6a5363f Compare January 5, 2024 01:55

wyb approved these changes Jan 5, 2024

View reviewed changes

banmoy reviewed Jan 5, 2024

View reviewed changes

be/src/exec/pipeline/exec_state_reporter.cpp Show resolved Hide resolved

banmoy approved these changes Jan 5, 2024

View reviewed changes

sevev approved these changes Jan 5, 2024

View reviewed changes

meegoo merged commit d088b3f into StarRocks:main Jan 5, 2024
44 of 45 checks passed

github-actions bot removed the 3.2 label Jan 5, 2024

github-actions bot removed the 3.1 label Jan 5, 2024

github-actions bot removed the 3.0 label Jan 5, 2024

github-actions bot removed the 2.5 label Jan 5, 2024

mergify bot mentioned this pull request Jan 5, 2024

[BugFix] Fix exec state report lost lead to ingestion status getting stuck (backport #36688) #38564

Merged

18 tasks

mergify bot pushed a commit that referenced this pull request Jan 5, 2024

[BugFix] Fix exec state report lost lead to ingestion status getting …

76c5832

…stuck (#36688) Signed-off-by: meegoo <meegoo.sr@gmail.com> (cherry picked from commit d088b3f)

This was referenced Jan 5, 2024

[BugFix] Fix exec state report lost lead to ingestion status getting stuck (backport #36688) #38565

Merged

[BugFix] Fix exec state report lost lead to ingestion status getting stuck (backport #36688) #38566

Merged

mergify bot mentioned this pull request Jan 5, 2024

[BugFix] Fix exec state report lost lead to ingestion status getting stuck (backport #36688) #38567

Merged

18 tasks

wanpengfei-git pushed a commit that referenced this pull request Jan 8, 2024

[BugFix] Fix exec state report lost lead to ingestion status getting …

cfbf33f

…stuck (backport #36688) (#38565) Co-authored-by: meegoo <meegoo.sr@gmail.com>

github-actions bot added the 3.1-merged label Jan 8, 2024

meegoo added a commit that referenced this pull request Jan 12, 2024

[BugFix] Fix exec state report lost lead to ingestion status getting …

6ad9632

…stuck (backport #36688) (#38567) Signed-off-by: meegoo <meegoo.sr@gmail.com> Co-authored-by: meegoo <meegoo.sr@gmail.com>

github-actions bot added the 2.5-merged label Jan 12, 2024

meegoo added a commit that referenced this pull request Jan 12, 2024

[BugFix] Fix exec state report lost lead to ingestion status getting …

9deb08c

…stuck (backport #36688) (#38564) Signed-off-by: meegoo <meegoo.sr@gmail.com> Co-authored-by: meegoo <meegoo.sr@gmail.com>

github-actions bot added the 3.2-merged label Jan 12, 2024

meegoo added a commit that referenced this pull request Jan 12, 2024

[BugFix] Fix exec state report lost lead to ingestion status getting …

9499089

…stuck (backport #36688) (#38566) Signed-off-by: meegoo <meegoo.sr@gmail.com> Co-authored-by: meegoo <meegoo.sr@gmail.com>

github-actions bot added the 3.0-merged label Jan 12, 2024

wxl24life pushed a commit to wxl24life/starrocks that referenced this pull request Feb 22, 2024

[Backport][BugFix] Fix exec state report lost lead to ingestion statu…

06c8349

…s getting stuck (backport StarRocks#36688) (StarRocks#38565) Co-authored-by: meegoo <meegoo.sr@gmail.com>

caneGuy pushed a commit to caneGuy/starrocks that referenced this pull request Mar 22, 2024

[Backport][BugFix] Fix exec state report lost lead to ingestion statu…

189e9d4

…s getting stuck (backport StarRocks#36688) (StarRocks#38565) Co-authored-by: meegoo <meegoo.sr@gmail.com>

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

[BugFix] Fix exec state report lost lead to ingestion status getting stuck #36688

[BugFix] Fix exec state report lost lead to ingestion status getting stuck #36688

meegoo commented Dec 8, 2023 •

edited

github-actions bot commented Jan 5, 2024

github-actions bot commented Jan 5, 2024

github-actions bot commented Jan 5, 2024

github-actions bot commented Jan 5, 2024

github-actions bot commented Jan 5, 2024

github-actions bot commented Jan 5, 2024

mergify bot commented Jan 5, 2024 •

edited

mergify bot commented Jan 5, 2024 •

edited

mergify bot commented Jan 5, 2024 •

edited

mergify bot commented Jan 5, 2024 •

edited

[BugFix] Fix exec state report lost lead to ingestion status getting stuck #36688

[BugFix] Fix exec state report lost lead to ingestion status getting stuck #36688

Conversation

meegoo commented Dec 8, 2023 • edited

Why I'm doing:

What I'm doing:

What type of PR is this:

Checklist:

Bugfix cherry-pick branch check:

github-actions bot commented Jan 5, 2024

[FE Incremental Coverage Report]

github-actions bot commented Jan 5, 2024

[BE Incremental Coverage Report]

file detail

github-actions bot commented Jan 5, 2024

github-actions bot commented Jan 5, 2024

github-actions bot commented Jan 5, 2024

github-actions bot commented Jan 5, 2024

mergify bot commented Jan 5, 2024 • edited

✅ Backports have been created

mergify bot commented Jan 5, 2024 • edited

✅ Backports have been created

mergify bot commented Jan 5, 2024 • edited

✅ Backports have been created

mergify bot commented Jan 5, 2024 • edited

✅ Backports have been created

meegoo commented Dec 8, 2023 •

edited

mergify bot commented Jan 5, 2024 •

edited

mergify bot commented Jan 5, 2024 •

edited

mergify bot commented Jan 5, 2024 •

edited

mergify bot commented Jan 5, 2024 •

edited