Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

[unittest] supported condititonal testing based on env var #1701

Merged

Conversation

FrankLeeeee
Copy link
Contributor

What is the problem?

During development, we often need to temporarily skip some tests, either because they take too long or they are currently not working well. However, we can easily forget that these tests are in fact required to verify the correctness of each commit. To solve this problem, skipping a test should be controlled by a condition. Before we put up a pull request to the GitHub, we can set the condition flag locally and run a full range test without manually removing pytest.mark.skip in the code.

What does this PR do?

This PR added a util function run_on_environment_flag to colossalai.testing. This function receives a environment variable name and check if it is set to '1'. If so, this test is executed, otherwise skipped.

This PR also replaced pytest.mark.skip with run_on_environment_flag for the auto_parallel module testing.

Demo

  • Run full-range testing without skipping.
AUTO_PARALLEL=1 pytest tests/test_auto_parallel/

Screen Shot 2022-10-13 at 18 33 15

  • Run testing with some tests being skipped.
# if you do not set AUTO_PARALLEL, it is the same
AUTO_PARALLEL=0 pytest tests/test_auto_parallel/

Screen Shot 2022-10-13 at 18 33 33

@YuliangLiu0306 YuliangLiu0306 merged commit 0e52f3d into hpcaitech:main Oct 13, 2022
@FrankLeeeee FrankLeeeee deleted the feature/test-by-env-var-flag branch January 26, 2023 07:46
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

None yet

2 participants