Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

[New Feature]: Add tests that run only on demand #40406

Closed
heshanpadmasiri opened this issue May 12, 2023 · 0 comments
Closed

[New Feature]: Add tests that run only on demand #40406

heshanpadmasiri opened this issue May 12, 2023 · 0 comments
Labels
Area/TestFramework IceBox Reason/Other None of the other reasons. Team/DevTools Ballerina Developer Tooling ( CLI, Test FW, Package Management, OpenAPI, APIDocs ) Type/NewFeature

Comments

@heshanpadmasiri
Copy link
Member

Description

Make it possible to annotate tests as "optional" so that they run only with a special flag to the bal test command.

Describe your problem(s)

While it is possible to disable a set of tests using --disable-groups, for projects that have tests that are not supposed to run by default (such as property-based tests that randomly generate a large number of random tests) this means their default test command becomes bal test --disable-groups <...> and their special case (running all tests) bal test. I think it is better if it is possible to make bal test the default option even for such cases.

Describe your solution(s)

Have an annotation (similar to enabled) that mark these tests run only on demand (say optional) and have a special flag (similar to --groups, say run --run-optional) that will run all of these tests as well. (Or make the --groups causes optional to be ignored)

Related area

-> Test Framework

Related issue(s) (optional)

No response

Suggested label(s) (optional)

No response

Suggested assignee(s) (optional)

No response

@ballerina-bot ballerina-bot added Team/DevTools Ballerina Developer Tooling ( CLI, Test FW, Package Management, OpenAPI, APIDocs ) Area/TestFramework labels May 12, 2023
@Dilhasha Dilhasha added Reason/Other None of the other reasons. IceBox labels Sep 27, 2024
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
Area/TestFramework IceBox Reason/Other None of the other reasons. Team/DevTools Ballerina Developer Tooling ( CLI, Test FW, Package Management, OpenAPI, APIDocs ) Type/NewFeature
Projects
None yet
Development

No branches or pull requests

3 participants