[Feature][Master] Run single task in workflow instance #13103

jieguangzhou · 2022-12-05T11:47:21Z

Purpose of the pull request

close: #12978

Brief change log

UI

In the Workflow Instance page, add three buttons if we Right-click the task
In the Workflow Instance page, if we save the workflow instance, will refresh instead of returning the workflow instance list page. (Because we can execute the task after editing the task definition directly)

Backend

Add an interface executeTask to execute the task
add a new function to clear data of existing task instance
fix the bug of inserting double same TaskDefinitionLogs in saveTaskDefine. (This will make some errors if we add and edit a new task definition in the workflow instance page)

Verify this pull request

This pull request is code cleanup without any test coverage.

(or)

This pull request is already covered by existing tests, such as (please describe tests).

(or)

This change added tests and can be verified as follows:

(or)

If your pull request contain incompatible change, you should also add it to docs/docs/en/guide/upgrede/incompatible.md

...heduler-api/src/main/java/org/apache/dolphinscheduler/api/controller/ExecutorController.java

...uler-api/src/main/java/org/apache/dolphinscheduler/api/service/impl/ExecutorServiceImpl.java

.../src/main/java/org/apache/dolphinscheduler/server/master/runner/WorkflowExecuteRunnable.java

+        Set<String> taskCodesString = dag.getAllNodesList();
+
+        for (String taskCodeString : taskCodesString) {
+            long taskCode = Long.parseLong(taskCodeString);


...heduler-api/src/main/java/org/apache/dolphinscheduler/api/controller/ExecutorController.java

...uler-api/src/main/java/org/apache/dolphinscheduler/api/service/impl/ExecutorServiceImpl.java

caishunfeng · 2022-12-06T02:52:03Z

.../src/main/java/org/apache/dolphinscheduler/server/master/runner/WorkflowExecuteRunnable.java

+                taskInstance = taskInstanceDao.findTaskByInstanceIdAndCode(processInstance.getId(), taskCode);
+            }
+            if (taskInstance == null) {
+                continue;


It's better to add some warn log.

This is not an anomaly. If some task failed in the last execution of the workflow, the downstream task will not create a task instance. And then, taskInstance is null, we just ignore them.

.../src/main/java/org/apache/dolphinscheduler/server/master/runner/WorkflowExecuteRunnable.java

...uler-api/src/main/java/org/apache/dolphinscheduler/api/service/impl/ExecutorServiceImpl.java

codecov-commenter · 2022-12-06T03:08:11Z

Codecov Report

Merging #13103 (e315788) into dev (26b7541) will increase coverage by 0.09%.
The diff coverage is 62.60%.

@@             Coverage Diff              @@
##                dev   #13103      +/-   ##
============================================
+ Coverage     39.28%   39.37%   +0.09%     
- Complexity     4277     4292      +15     
============================================
  Files          1069     1069              
  Lines         40305    40425     +120     
  Branches       4633     4642       +9     
============================================
+ Hits          15834    15919      +85     
- Misses        22685    22712      +27     
- Partials       1786     1794       +8

Impacted Files	Coverage Δ
...inscheduler/api/controller/ExecutorController.java	`18.98% <0.00%> (-0.75%)`	⬇️
...che/dolphinscheduler/common/enums/CommandType.java	`0.00% <0.00%> (ø)`
.../org/apache/dolphinscheduler/common/graph/DAG.java	`94.48% <0.00%> (-0.75%)`	⬇️
...duler/dao/repository/impl/TaskInstanceDaoImpl.java	`3.33% <0.00%> (-0.06%)`	⬇️
...nscheduler/service/process/ProcessServiceImpl.java	`29.58% <23.07%> (-0.11%)`	⬇️
.../server/master/runner/WorkflowExecuteRunnable.java	`10.43% <65.85%> (+2.26%)`	⬆️
...cheduler/api/service/impl/ExecutorServiceImpl.java	`46.48% <73.33%> (+3.08%)`	⬆️
...apache/dolphinscheduler/api/enums/ExecuteType.java	`66.66% <100.00%> (ø)`
.../org/apache/dolphinscheduler/api/enums/Status.java	`100.00% <100.00%> (ø)`
...er/master/dispatch/host/assign/RandomSelector.java	`77.77% <0.00%> (-5.56%)`	⬇️
... and 3 more

📣 We’re building smart automated test selection to slash your CI/CD build times. Learn more

...uler-api/src/main/java/org/apache/dolphinscheduler/api/service/impl/ExecutorServiceImpl.java

jieguangzhou · 2022-12-08T01:54:52Z

@labbomb @songjianet @Amy0104 Please help to review the code of front end, thanks

jieguangzhou · 2022-12-08T01:58:43Z

@caishunfeng @ruanwenjun Please help to review the backend code, thanks

sonarcloud · 2022-12-09T02:36:36Z

Kudos, SonarCloud Quality Gate passed!

0 Bugs
0 Vulnerabilities
0 Security Hotspots
8 Code Smells

68.8% Coverage
0.0% Duplication

caishunfeng

+1

EricGao888 · 2023-03-29T07:17:37Z

@jieguangzhou Hello, may I ask what's the relation between this PR and #10117 ? Seems there are some stuff overlapping with each other.

jieguangzhou · 2023-03-29T07:37:35Z

@jieguangzhou Hello, may I ask what the relation between this PR and #10117 ? Seems there are some stuff overlapping with each other.

10177 runs from the workflow definition, while this pr runs from the workflow instance. Previously, if we runs a single task in 10177 and wanted to continue using that workflow instance to execute the next task, it was impossible. This PR solves this problem.

EricGao888 · 2023-03-29T07:47:52Z

@jieguangzhou Hello, may I ask what the relation between this PR and #10117 ? Seems there are some stuff overlapping with each other.

10177 runs from the workflow definition, while this pr runs from the workflow instance. Previously, if we runs a single task in 10177 and wanted to continue using that workflow instance to execute the next task, it was impossible. This PR solves this problem.

Cool, thx for the explanation : )

github-actions bot added backend UI ui and front end related labels Dec 5, 2022

jieguangzhou added this to the 3.2.0 milestone Dec 5, 2022

jieguangzhou added the 3.2.0 for 3.2.0 version label Dec 5, 2022

jieguangzhou self-assigned this Dec 5, 2022

jieguangzhou added the feature new feature label Dec 5, 2022

github-advanced-security bot found potential problems Dec 5, 2022

View reviewed changes

...heduler-api/src/main/java/org/apache/dolphinscheduler/api/controller/ExecutorController.java Fixed Show fixed Hide fixed

caishunfeng reviewed Dec 6, 2022

View reviewed changes

github-advanced-security bot found potential problems Dec 6, 2022

View reviewed changes

...uler-api/src/main/java/org/apache/dolphinscheduler/api/service/impl/ExecutorServiceImpl.java Fixed Show fixed Hide fixed

jieguangzhou marked this pull request as ready for review December 7, 2022 06:52

jieguangzhou requested review from songjianet, Amy0104, SbloodyS and ruanwenjun as code owners December 7, 2022 06:52

jieguangzhou force-pushed the execute-single-task branch 2 times, most recently from e315788 to d1cc4d6 Compare December 7, 2022 13:41

jieguangzhou requested a review from labbomb December 8, 2022 01:53

Allow execute task in workflow instance

d414f36

jieguangzhou force-pushed the execute-single-task branch from d1cc4d6 to d414f36 Compare December 9, 2022 02:00

songjianet approved these changes Dec 13, 2022

View reviewed changes

caishunfeng approved these changes Dec 13, 2022

View reviewed changes

caishunfeng merged commit e4b9b67 into apache:dev Dec 13, 2022

HomminLee mentioned this pull request Aug 27, 2023

[Bug] [API] Modify deleted task will write twice version log #14805

Closed

3 tasks

HomminLee pushed a commit to HomminLee/dolphinscheduler that referenced this pull request Sep 28, 2023

cherry-pick Allow execute task in workflow instance (apache#13103)

e815ca0

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

[Feature][Master] Run single task in workflow instance #13103

[Feature][Master] Run single task in workflow instance #13103

jieguangzhou commented Dec 5, 2022 •

edited

Loading

caishunfeng Dec 6, 2022

jieguangzhou Dec 6, 2022

codecov-commenter commented Dec 6, 2022 •

edited

Loading

jieguangzhou commented Dec 8, 2022

jieguangzhou commented Dec 8, 2022

sonarcloud bot commented Dec 9, 2022

caishunfeng left a comment

EricGao888 commented Mar 29, 2023 •

edited

Loading

jieguangzhou commented Mar 29, 2023

EricGao888 commented Mar 29, 2023

[Feature][Master] Run single task in workflow instance #13103

[Feature][Master] Run single task in workflow instance #13103

Conversation

jieguangzhou commented Dec 5, 2022 • edited Loading

Purpose of the pull request

Brief change log

UI

Backend

Verify this pull request

caishunfeng Dec 6, 2022

Choose a reason for hiding this comment

jieguangzhou Dec 6, 2022

Choose a reason for hiding this comment

codecov-commenter commented Dec 6, 2022 • edited Loading

Codecov Report

jieguangzhou commented Dec 8, 2022

jieguangzhou commented Dec 8, 2022

sonarcloud bot commented Dec 9, 2022

caishunfeng left a comment

Choose a reason for hiding this comment

EricGao888 commented Mar 29, 2023 • edited Loading

jieguangzhou commented Mar 29, 2023

EricGao888 commented Mar 29, 2023

jieguangzhou commented Dec 5, 2022 •

edited

Loading

codecov-commenter commented Dec 6, 2022 •

edited

Loading

EricGao888 commented Mar 29, 2023 •

edited

Loading