[DSIP-10][Unit Tests] Improve DolphinScheduler unit tests

[EricGao888]

Search before asking

• I had searched in the issues and found no similar feature requirement.

Description

Unit Tests of high quality could not only improve the stability of Dolphin Scheduler, but also increase the readability of Dolphin Scheduler Code.

• Currently we only have around 40% UT coverage. Although DS already has E2E tests and the community are working on the API tests, it never means we do not need high quality unit tests.
• As a DS developer, every time after submitting a PR, we will see the current UT coverage of the whole project. If the coverage is too low, developers may have concerns about the stability of the application.
• Reading unit tests is one of the best approaches for new developers to understand the code. UT cases also help reviewers to review PR more easily. Furthermore, as it is hardly possible to provide docs of every detailed feature of DS, the best way is to keep UT updated.

Use case

• We could have different standards for UT for different parts of DS. For example, we have higher bar on UT for core features but lower bar for plugins.
• We could offer a list of UT cases which need to be updated, added or refactored and label them with good first issue. For new developers to this community, writing UT is a good way to understand the code logic and it is more intriguing than refactoring docs.

Related issues

• related: [Improvement] unit test suggestion #3700
• related: [Feature][style] Replace checkstyle plugin with spotless to automatically fix formatting errors #10963

Action Items

• Add Spotless plugin into DolphinScheduler to enable developers to fix style / formatting errors incrementally: [Feature][style] Replace checkstyle plugin with spotless to automatically fix formatting errors #10963
• Remove dependency on Powermock: [Improvement][Test] Remove dependency of powermock #11405
• Upgrade jUnit dependency from 4.x to 5.x: [Improvement][UT] Upgrade junit to 5.+ #10976
• Migrate all legacy cases from jUnit 4 to jUnit 5 in the whole project: [Migration][Test] jUnit4 -> jUnit5 UT cases migration #12301
• Add CI checks to prevent developer from writing UTs with jUnit4 and Powermock.
• Refactor typical UTs in each module as examples.
• Add docs of instructions about how to write UT in DolphinScheduler project.
• Create some easy UT refactoring tasks and label them with good first issue.
• Refactor all UTs in each module complying to method coverage criteria and fix the style & formatting errors of related test code and tested code. This one will be divided into sub-tasks later.

Refactoring Guideline

• We use method coverage as refactoring metric to decouple UT cases as much as possible.
• We refactor UTs with jUnit 5 instead of jUnit 4.
• We should avoid using Powermock in UTs and remove legacy UTs which contains Powermock, see: [Improvement][Test] Remove dependency of powermock #11405 .
• Scope of each PR:

Progress

• Arranged by priority:

Module	Sub-Module	Class	issue	status
dolphinscheduler-api
dolphinscheduler-master
dolphinscheduler-worker
dolphinscheduler-common
dolphinscheduler-service
dolphinscheduler-alert	dolphinscheduler-alert-server	AlertServer
dolphinscheduler-task-plugin

Appendix: UT coverage of each module before refactoring

Are you willing to submit a PR?

• Yes I am willing to submit a PR!

Code of Conduct

• I agree to follow this project's Code of Conduct

[github-actions]

Thank you for your feedback, we have received your issue, Please wait patiently for a reply.

• In order for us to understand your request as soon as possible, please provide detailed information、version or pictures.
• If you haven't received a reply for a long time, you can join our slack and send your question to channel #troubleshooting

[SbloodyS]

That's great!

[EricGao888]

That's great!

@SbloodyS Thx. Hope this will help and attract more new developers to submit PRs for DS and offer them an approach to get familiar with the code smoothly.

[davidzollo]

great

[EricGao888]

IMHO, we need some priorities, we could apply high priority to those modules where UT might play a more significant role regarding to code quality and application stability.

[EricGao888]

@SbloodyS Could you please help add a discussion label to this issue? Much appreciated~

[SbloodyS]

IMHO, we need some priorities, we could apply high priority to those modules where UT might play a more significant role regarding to code quality and application stability.

We can make a list and mark it with priority.

[EricGao888]

I got blocked by some unit test when developing a new feature and mocking some unusual stuff for nearly a day. I would like to update the UT docs with some examples which may help new developers bypass or overcome some pitfalls in UTs. May I ask whether it is possible to migrate this docs into main repo? @zhongjiajie https://dolphinscheduler.apache.org/en-us/community/development/unit-test.html

Thanks

[zhongjiajie]

I got blocked by some unit test when developing a new feature and mocking some unusual stuff for nearly a day. I would like to update the UT docs with some examples which may help new developers bypass or overcome some pitfalls in UTs. May I ask whether it is possible to migrate this docs into main repo? @zhongjiajie https://dolphinscheduler.apache.org/en-us/community/development/unit-test.html

Thanks

Of cause, we can, do you interesting in this migration?

[EricGao888]

I got blocked by some unit test when developing a new feature and mocking some unusual stuff for nearly a day. I would like to update the UT docs with some examples which may help new developers bypass or overcome some pitfalls in UTs. May I ask whether it is possible to migrate this docs into main repo? @zhongjiajie https://dolphinscheduler.apache.org/en-us/community/development/unit-test.html
Thanks

Of cause, we can, do you interesting in this migration?

Yes, I'd love to give a shot.

[EricGao888]

BTW, we could fix the formatting and style errors incrementally, together with unit tests. See: #10963
One convenient way is when refactoring some UTs, we fix the formatting and style errors of both the testing and tested part.

[EricGao888]

I've generated test coverage reports for every module of DS using Intellij, sorted by method coverage. Here are some results:

[EricGao888]

I will update this issue with a more detailed refactoring plan as well as priorities later this week.

[stalary]

I can get involved in the junit upgrade~ @EricGao888

[EricGao888]

I can get involved in the junit upgrade~ @EricGao888

@stalary Currently we are working on removing powermock because it does not support jUnit 5 and blocks the upgrade. Would u like to participate in #11405 ? Thanks~

[stalary]

Currently we are working on removing powermock because it does not support jUnit 5 and blocks the upgrade. Would u like to participate in

Okay, I can be a part of that.

[EricGao888]

Currently we are working on removing powermock because it does not support jUnit 5 and blocks the upgrade. Would u like to participate in

Okay, I can be a part of that.

@stalary Great! We have #12150 open to contributors. Would u like to pick it up? Thanks

[stalary]

Currently we are working on removing powermock because it does not support jUnit 5 and blocks the upgrade. Would u like to participate in

Okay, I can be a part of that.

@stalary Great! We have #12150 open to contributors. Would u like to pick it up? Thanks

Please assign to me, thanks~

[EricGao888]

Usage of Powermock has been fully removed from the whole project : )

[zhongjiajie]

Usage of Powermock has been fully removed from the whole project :

Great new