Skip to content

[Feature-#8373][MasterServer] Dependent tasks can re-run automatically in the case of complement#8496

Merged
caishunfeng merged 7 commits intoapache:devfrom
SbloodyS:feature_8373
Feb 25, 2022
Merged

[Feature-#8373][MasterServer] Dependent tasks can re-run automatically in the case of complement#8496
caishunfeng merged 7 commits intoapache:devfrom
SbloodyS:feature_8373

Conversation

@SbloodyS
Copy link
Copy Markdown
Member

@SbloodyS SbloodyS commented Feb 23, 2022

Purpose of the pull request

This pr will close #8373

Brief change log

  1. add queryDependentProcessDefinitionByProcessDefinitionCode in dolphinscheduler-dao/src/main/resources/org/apache/dolphinscheduler/dao/mapper/WorkFlowLineageMapper.xml and DependentProcessDefinition entity and query function queryDependentProcessDefinitionByProcessDefinitionCode in dolphinscheduler-service/src/main/java/org/apache/dolphinscheduler/service/process/ProcessService.java
  2. add querySchedulesByProcessDefinitionCodes in dolphinscheduler-dao/src/main/resources/org/apache/dolphinscheduler/dao/mapper/ScheduleMapper.xml and query function queryWorkerGroupByProcessDefinitionCodes in dolphinscheduler-service/src/main/java/org/apache/dolphinscheduler/service/process/ProcessService.java
  3. add createComplementDependentCommand in dolphinscheduler-api/src/main/java/org/apache/dolphinscheduler/api/service/impl/ExecutorServiceImpl.java and its related to unit test and its controller.

Execution process:

  1. Query the dependent tasks and its process definition of the current complement process definition through the database.
  2. Query the dependent task's process definition's schedule.
  3. Check whether the dependent tasks are normal.
  4. Create dependent task's command which start running from dependent node.

@codecov-commenter
Copy link
Copy Markdown

codecov-commenter commented Feb 23, 2022

Codecov Report

❌ Patch coverage is 12.26415% with 93 lines in your changes missing coverage. Please review.
✅ Project coverage is 45.22%. Comparing base (922283e) to head (6bab29c).
⚠️ Report is 2931 commits behind head on dev.

Files with missing lines Patch % Lines
...cheduler/api/service/impl/ExecutorServiceImpl.java 20.40% 37 Missing and 2 partials ⚠️
...heduler/dao/entity/DependentProcessDefinition.java 0.00% 39 Missing ⚠️
...cheduler/common/enums/ComplementDependentMode.java 0.00% 9 Missing ⚠️
...inscheduler/api/controller/ExecutorController.java 50.00% 2 Missing and 1 partial ⚠️
...lphinscheduler/service/process/ProcessService.java 0.00% 3 Missing ⚠️
Additional details and impacted files
@@             Coverage Diff              @@
##                dev    #8496      +/-   ##
============================================
- Coverage     45.38%   45.22%   -0.16%     
+ Complexity     4031     4030       -1     
============================================
  Files           686      688       +2     
  Lines         26725    26826     +101     
  Branches       2870     2882      +12     
============================================
+ Hits          12130    12133       +3     
- Misses        13451    13544      +93     
- Partials       1144     1149       +5     

☔ View full report in Codecov by Sentry.
📢 Have feedback on the report? Share it here.

🚀 New features to boost your workflow:
  • ❄️ Test Analytics: Detect flaky tests, report on failures, and find test suite problems.
  • 📦 JS Bundle Analysis: Save yourself from yourself by tracking and limiting bundle sizes in JS merges.

Copy link
Copy Markdown
Contributor

@caishunfeng caishunfeng left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Hi @SbloodyS It is better to show the brief change log of this pr, which can let others understand the content of pr more quickly.

processService.queryDependentProcessDefinitionByProcessDefinitionCode(processDefinitionCode);

for (DependentProcessDefinition dependentProcessDefinition : dependentProcessDefinitionList) {
if (dependentProcessDefinition.getDependentCycle() == processDefinitionCycle) {
Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Please add some notes here.

@sonarqubecloud
Copy link
Copy Markdown

SonarCloud Quality Gate failed.    Quality Gate failed

Bug A 0 Bugs
Vulnerability A 0 Vulnerabilities
Security Hotspot A 0 Security Hotspots
Code Smell A 2 Code Smells

15.7% 15.7% Coverage
0.0% 0.0% Duplication

@wangxj3 wangxj3 changed the title [Feature][MasterServer] Dependent tasks can re-run automatically in the case of complement [Feature-#8373][MasterServer] Dependent tasks can re-run automatically in the case of complement Feb 24, 2022
Copy link
Copy Markdown
Contributor

@lenboo lenboo left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

+1

@caishunfeng caishunfeng merged commit 891a002 into apache:dev Feb 25, 2022
@SbloodyS SbloodyS deleted the feature_8373 branch February 25, 2022 02:34
@caishunfeng caishunfeng added the miss:docs missing documents in PR label Feb 25, 2022
@caishunfeng
Copy link
Copy Markdown
Contributor

@SbloodyS Well done, thanks for your contribution.
BTW, I add the miss document label, please submit the doc after UI part finish.

@SbloodyS SbloodyS added this to the 3.1.0-alpha milestone Jun 23, 2022
@SbloodyS SbloodyS added improvement make more easy to user or prompt friendly backend feature new feature and removed miss:docs missing documents in PR improvement make more easy to user or prompt friendly labels Jun 23, 2022
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

Projects

None yet

Development

Successfully merging this pull request may close these issues.

[Feature][MasterServer] Dependent tasks can re-run automatically in the case of complement

4 participants