Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

[Bug][Master] fix duplicate event, avoid event stacking #14986

Merged
merged 1 commit into from
Oct 7, 2023
Merged

[Bug][Master] fix duplicate event, avoid event stacking #14986

merged 1 commit into from
Oct 7, 2023

Conversation

caishunfeng
Copy link
Contributor

@caishunfeng caishunfeng commented Oct 7, 2023 •
edited
Loading

Purpose of the pull request

When run a DAG which include more than 400+ sub process tasks, StateWheelExecuteThread will create 400+ events every 5s, and WorkflowExecuteRunnable will sleep 1s when handle event error, which cause event stacking.

BTW, it was fixed by pr #13948 so it dont have to fix in dev and 320.

Brief change log

  • add exist check before add event to WorkflowExecuteRunnable queue.
  • change sleep interval from 1s to 100ms.

Verify this pull request

  • Manually verified the change by testing locally.

@caishunfeng caishunfeng self-assigned this Oct 7, 2023
@caishunfeng caishunfeng added this to the 3.1.9 milestone Oct 7, 2023
@caishunfeng caishunfeng added 3.1.x for 3.1.x version bug Something isn't working backend and removed backend labels Oct 7, 2023
@codecov-commenter
Copy link

codecov-commenter commented Oct 7, 2023 •
edited
Loading

Codecov Report

❗ No coverage uploaded for pull request base (3.1.9-prepare@ed1cf53). Click here to learn what that means.
The diff coverage is n/a.

❗ Current head eb9eac4 differs from pull request most recent head be9780d. Consider uploading reports for the commit be9780d to get more accurate results

@@               Coverage Diff                @@
##             3.1.9-prepare   #14986   +/-   ##
================================================
  Coverage                 ?   38.22%           
  Complexity               ?     3989           
================================================
  Files                    ?     1000           
  Lines                    ?    36990           
  Branches                 ?     4278           
================================================
  Hits                     ?    14138           
  Misses                   ?    21236           
  Partials                 ?     1616

📣 We’re building smart automated test selection to slash your CI/CD build times. Learn more

@sonarcloud
Copy link

sonarcloud bot commented Oct 7, 2023

SonarCloud Quality Gate failed.    Quality Gate failed

Bug E 14 Bugs
Vulnerability B 9 Vulnerabilities
Security Hotspot E 4 Security Hotspots
Code Smell A 864 Code Smells

28.1% 28.1% Coverage
2.5% 2.5% Duplication

warning The version of Java (11.0.20.1) you have used to run this analysis is deprecated and we will stop accepting it soon. Please update to at least Java 17.
Read more here

idea Catch issues before they fail your Quality Gate with our IDE extension sonarlint SonarLint

ruanwenjun
ruanwenjun approved these changes Oct 7, 2023
View reviewed changes
.../src/main/java/org/apache/dolphinscheduler/server/master/runner/StateWheelExecuteThread.java
Comment on lines +38 to +43
import java.util.Optional;
import java.util.concurrent.ConcurrentLinkedQueue;

import javax.annotation.PostConstruct;

import lombok.NonNull;
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The code style is broken.

@ruanwenjun ruanwenjun merged commit 7a38b87 into apache:3.1.9-prepare Oct 7, 2023
36 of 38 checks passed
caishunfeng added a commit that referenced this pull request Oct 11, 2023
@caishunfeng
Revert "fix duplicate event (#14986)"
aac45af 
This reverts commit 7a38b87.
@caishunfeng caishunfeng mentioned this pull request Oct 11, 2023
Merged
lenboo pushed a commit that referenced this pull request Oct 11, 2023
@caishunfeng
Revert "fix duplicate event (#14986)" (#15006)
160fde1 
This reverts commit 7a38b87.
@zhuangchong zhuangchong added the release cherry-pick Mark this issue/PR had cherry-pick for release version label Dec 6, 2023
@fuchanghai fuchanghai mentioned this pull request Feb 27, 2024
Closed
3 tasks
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Reviewers

@ruanwenjun ruanwenjun ruanwenjun approved these changes

@SbloodyS SbloodyS Awaiting requested review from SbloodyS

@zhongjiajie zhongjiajie Awaiting requested review from zhongjiajie

@zhuangchong zhuangchong Awaiting requested review from zhuangchong

Assignees

@caishunfeng caishunfeng

Labels
3.1.x for 3.1.x version backend bug Something isn't working release cherry-pick Mark this issue/PR had cherry-pick for release version
Projects
None yet
Milestone
3.1.9
Development

Successfully merging this pull request may close these issues.

None yet
4 participants
@caishunfeng @codecov-commenter @ruanwenjun @zhuangchong