Skip to content

[Feature]Blocking task UI and Doc#10397

Closed
mgsky1 wants to merge 1 commit intoapache:devfrom
mgsky1:blocking-task-ui
Closed

[Feature]Blocking task UI and Doc#10397
mgsky1 wants to merge 1 commit intoapache:devfrom
mgsky1:blocking-task-ui

Conversation

@mgsky1
Copy link
Contributor

@mgsky1 mgsky1 commented Jun 10, 2022

Purpose of the pull request

This PR adds blocking task UI and documentation. The blocking task node backed code has been merged in PR #6272.

Brief change log

  • add blocking task icon
  • add blocking task configuration pannel
  • add operations (continue or re-run) when the workflow has been blocked
  • add blocking task documentation in Chinese and English

Screen Shots

blocking_task01

blocking_task02

blocking_task03

blocking_task04

Verify this pull request

Test cases:

  • normal: i.e. Blocking node has upstream and downstram nodes.
  • blocking node in the begining of workflow.
  • blocking node in the end of workflow.

They all work fine.

Discuss

  • When config blocking node, it has a alert option. In my view, it is better to use switch plugin, but I do not know how to set switch plugin value when the switch turning on or off.
  • As for the operation after the workflow is blocked. I think re-run and continue execution is rational. When I testing the operation: continuing from faild node, this operation will skip those node which has been executed successfully. It means that blocking node will lose its function. So I disable this operation.

此PR的目的

为了完善阻断类任务的开发,后端代码已经在PR#6272中被合并

变更日志

  • 阻断类任务图标
  • 阻断节点配置面板
  • 当工作流被阻断时,提供重跑和继续执行,两种选择
  • 增加阻断节点中英文档

截图

blocking_task01

blocking_task02

blocking_task03

blocking_task04

测试

主要测试了以下几种情况

  • 正常模式:阻断节点上下游都有任务
  • 阻断节点在工作流头部(没有上游节点)
  • 阻断节点在工作流尾部(没有下游节点)

它们都能正常工作

讨论

  • 在阻断节点的配置中,有一个告警选项,我觉得将其作为switch开关更合适,但是我不知道如何给这个switch开关赋值,从代码来看,新的ui将配置项的每一条都抽出进行了高度封装,返回IJsonItem数组,不知道这种场景下,要用IJsonItem的什么字段给开关赋值。从现有的代码参考看,如果是在节点配置面板中使用了switch开关,那么开启开关后,会弹出更详细的设置信息;另外一处是在工作流空跑时出现的,那个场景是直接用HTML标签搞定的。
  • 对于工作流被阻断后的操作,我觉得继续执行和重跑是合理的。对于重新执行失败节点,在测试中发现,这种模式会跳过已经执行成功的节点,这会导致阻断节点的失效,因此不启用。

@SbloodyS
Copy link
Member

That's a cool~

@SbloodyS SbloodyS added UI ui and front end related improvement make more easy to user or prompt friendly backend document feature new feature labels Jun 10, 2022
@codecov-commenter
Copy link

codecov-commenter commented Jun 10, 2022

Codecov Report

Merging #10397 (87613d6) into dev (6d8befa) will decrease coverage by 0.16%.
The diff coverage is 0.00%.

@@             Coverage Diff              @@
##                dev   #10397      +/-   ##
============================================
- Coverage     39.46%   39.29%   -0.17%     
  Complexity     4247     4247              
============================================
  Files          1052     1061       +9     
  Lines         39860    40050     +190     
  Branches       4582     4600      +18     
============================================
+ Hits          15731    15739       +8     
- Misses        22349    22530     +181     
- Partials       1780     1781       +1     
Impacted Files Coverage Δ
...cheduler/api/service/impl/ExecutorServiceImpl.java 43.73% <0.00%> (ø)
...rver/master/runner/task/BlockingTaskProcessor.java 76.40% <0.00%> (ø)
...plugin/task/api/parameters/BlockingParameters.java 0.00% <ø> (ø)
...erver/master/processor/queue/TaskEventService.java 69.64% <0.00%> (-10.72%) ⬇️
...r/plugin/registry/zookeeper/ZookeeperRegistry.java 43.54% <0.00%> (-6.46%) ⬇️
...lphinscheduler/plugin/kubeflow/KubeflowHelper.java 81.39% <0.00%> (-5.45%) ⬇️
...dolphinscheduler/remote/future/ResponseFuture.java 81.96% <0.00%> (-1.64%) ⬇️
...r/plugin/task/sqoop/parameter/SqoopParameters.java 51.19% <0.00%> (-1.20%) ⬇️
...eduler/api/service/impl/DataSourceServiceImpl.java 50.75% <0.00%> (-1.02%) ⬇️
...che/dolphinscheduler/alert/AlertSenderService.java 45.39% <0.00%> (-0.61%) ⬇️
... and 15 more

📣 We’re building smart automated test selection to slash your CI/CD build times. Learn more

@sonarqubecloud
Copy link

SonarCloud Quality Gate failed.    Quality Gate failed

Bug A 0 Bugs
Vulnerability A 0 Vulnerabilities
Security Hotspot A 0 Security Hotspots
Code Smell A 0 Code Smells

0.0% 0.0% Coverage
0.0% 0.0% Duplication

Copy link
Member

@zhongjiajie zhongjiajie left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

backend and docs LGTM, @devosend please task a look at the frontend code


如果你有配置告警相关组件(例如email),那么将会收到一封告警邮件

## No newline at end of file
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

It seems that is unnecessary adding


This sample demonstrates the operation of the Condition task by using the [Shell](shell.md) task and [Python](python.md) task.

### 1. Create Workflow
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

should avoid using number in title, the leading number of # represent the level

Suggested change
### 1. Create Workflow
### Create Workflow

@mgsky1
Copy link
Contributor Author

mgsky1 commented Nov 14, 2022

I am very very sorry for updating this PR so late because of the busy work and training. I see projects iterating rapidly and the back-end code for the blocking node is still being maintained. I sincerely appreciated.
@zhongjiajie @caishunfeng

@EricGao888
Copy link
Member

EricGao888 commented Nov 15, 2022

I am very very sorry for updating this PR so late because of the busy work and training. I see projects iterating rapidly and the back-end code for the blocking node is still being maintained. I sincerely appreciated. @zhongjiajie @caishunfeng

@mgsky1 No worries, we are glad that you could follow up with this PR. The CI failed because of formatting errors, could u plz run mvn spotless:apply under the root directory so that spotless will automatically fix those errors for you?

@mgsky1
Copy link
Contributor Author

mgsky1 commented Nov 15, 2022

I am very very sorry for updating this PR so late because of the busy work and training. I see projects iterating rapidly and the back-end code for the blocking node is still being maintained. I sincerely appreciated. @zhongjiajie @caishunfeng

@mgsky1 No worries, we are glad that you could follow up with this PR. The CI failed because of formatting errors, could u plz run mvn spotless:apply under the root directory so that spotless will automatically fix those errors for you?

ok,I will run this command later today~

@mgsky1
Copy link
Contributor Author

mgsky1 commented Nov 15, 2022

There are still some errors occured in the Actions. I will fixed them later, maybe in this weekend

@sonarqubecloud
Copy link

SonarCloud Quality Gate failed.    Quality Gate failed

Bug A 0 Bugs
Vulnerability A 0 Vulnerabilities
Security Hotspot A 0 Security Hotspots
Code Smell A 0 Code Smells

0.0% 0.0% Coverage
0.4% 0.4% Duplication

@mgsky1
Copy link
Contributor Author

mgsky1 commented Nov 17, 2022

I fixed some error in code so that the Actions could be executed. I also clean the commits. The commit logs before are so terrible.@EricGao888

@caishunfeng caishunfeng added the 3.2.0 for 3.2.0 version label Feb 2, 2023
@caishunfeng
Copy link
Contributor

caishunfeng commented Feb 2, 2023

Hi @mgsky1 It's great to follow up the blocking task feature 👍
Please resolve the conflicts.

@caishunfeng
Copy link
Contributor

@devosend @Amy0104 @songjianet Do you have time to review this pr?

Copy link
Contributor

@devosend devosend left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

frontend LGTM

@songjianet
Copy link
Member

Please resolve the conflict. @mgsky1

@github-actions
Copy link

This pull request has been automatically marked as stale because it has not had recent activity for 120 days. It will be closed in 7 days if no further activity occurs.

@github-actions github-actions bot added the Stale label Dec 12, 2023
@github-actions
Copy link

This pull request has been closed because it has not had recent activity. You could reopen it if you try to continue your work, and anyone who are interested in it are encouraged to continue work on this pull request.

@github-actions github-actions bot closed this Feb 14, 2024
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

3.2.0 for 3.2.0 version backend document feature new feature improvement make more easy to user or prompt friendly Stale UI ui and front end related

Projects

Status: Done

Development

Successfully merging this pull request may close these issues.

8 participants