Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Revisit CI testing strategies #100

Open
behnazh opened this issue Jan 16, 2022 · 1 comment
Open

Revisit CI testing strategies #100

behnazh opened this issue Jan 16, 2022 · 1 comment
Labels
ci Improvements or additions to CI checks configuration Changes to the configurations help wanted Extra attention is needed

Comments

@behnazh
Copy link
Collaborator

behnazh commented Jan 16, 2022

We need to revisit how to trigger the CI checks based on the events (pull_request vs. push), branches (main, staging, and the rest), and matrices (e.g., OS and Python versions).

Potentially we could remove tox, but then it might make sense to enforce the CI tests on every push event for all the branches. Currently CI tests is only enabled for push to main and pull_request for other branches.

We also need to document the branching model and how it affects running the CI checks.

@behnazh behnazh added ci Improvements or additions to CI checks configuration Changes to the configurations help wanted Extra attention is needed labels Jan 16, 2022
@jenstroeger
Copy link
Owner

jenstroeger commented Jan 17, 2022

Potentially we could remove tox, but then it might make sense to enforce the CI tests on every push event for all the branches.

I agree: we use tox to run pytest with coverage on a [py39, py310] matrix, but that matrix is already being expanded with OSs in PR #99. I think it makes little sense to run unit tests locally across different Python versions (which may not even be installed on the system) if we can do so more extensively in CI; it’s probably enough if unit tests pass locally in whatever venv the user works.

Given that, the full test matrix (three OS flavors times two Python versions) should run at least with every merge/push to the main branch. This repository also uses a staging branch into which PRs are merged, and a “release” is a merge from staging into main. So, merge/push to the staging branch should also run the full test matrix.

What about opening a PR though, and every push to an open PR — run the full test matrix every time?

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
ci Improvements or additions to CI checks configuration Changes to the configurations help wanted Extra attention is needed
Projects
None yet
Development

No branches or pull requests

2 participants