Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

[Refactor] E2E test framework #4472

Closed
kezhenxu94 opened this issue Mar 9, 2020 · 1 comment · Fixed by #4510
Closed

[Refactor] E2E test framework #4472

kezhenxu94 opened this issue Mar 9, 2020 · 1 comment · Fixed by #4510
Assignees
@kezhenxu94
Labels
enhancement Enhancement on performance or codes test Test requirements about performance, feature or before release.
Milestone
7.0.0

Comments

@kezhenxu94
Copy link
Member

According to the feedbacks from our developers, committers and PMCs, especially from @JaredTan95 , the E2E framework is rather unfriendly to our contributors, especially new contributors, therefore I'm refactoring the E2E test framework, the goals are:

  • more debuggable, locally, in IDE

  • more intuitive, adopt well known technologies to let people get started more quickly

  • less verbose/duplicated codes

I've had a runnable version and may need some time to refactor the remains test cases, hope it would be ready soon, anyone who has more ideas about how the new E2E tests should work (friendly), please let me know
@kezhenxu94 kezhenxu94 added test Test requirements about performance, feature or before release. enhancement Enhancement on performance or codes labels Mar 9, 2020
@kezhenxu94 kezhenxu94 self-assigned this Mar 9, 2020
@wu-sheng
Copy link
Member

wu-sheng commented Mar 9, 2020

I think debug in IDE is the key.

@kezhenxu94 kezhenxu94 mentioned this issue Mar 14, 2020
Merged
@wu-sheng wu-sheng added this to the 7.0.0 milestone Mar 16, 2020
kezhenxu94 added a commit that referenced this issue Mar 16, 2020
@kezhenxu94
Rewrite the E2E test framework and tests (#4510)
dcd66ee 
Motivation:

According the feedback from the community, the e2e tests are hard for
them to understand and get them started, so this pull request aims to
rewrite the e2e tests with some improvements:

- debuggable locally, now you can run every individual case directly
inside the IDE, just like running a plain unit test.

- same functionalities of multiple counterparts share the same codes,
cases like different storages and cluster coordinators share the same
codes, easy to add more similar cases.

- the "end" now extends to the Doccker images, meaning that some bugs
like #4483 and #4471 can be more easily and early to be discovered.

- codes style and license headers are checked

Modifications:

- adopt jupeter (a.k.a junit 5), testcontainers as the new test
framework, all containers are orchestrated by docker-compose.yml,
well-known technology that developers should be familiar with.

- rewrite all test cases and verify them.

- reorganize the workflows in GitHub Actions.

Result:

- new e2e test should be easy to add.

- closes #4472
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Assignees

@kezhenxu94 kezhenxu94

Labels
enhancement Enhancement on performance or codes test Test requirements about performance, feature or before release.
Projects
None yet
Milestone
7.0.0
Development

Successfully merging a pull request may close this issue.

Rewrite the E2E test framework and tests
2 participants
@wu-sheng @kezhenxu94