Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Improve CI #22

Merged
merged 4 commits into from
Aug 24, 2023
Merged

Improve CI #22

merged 4 commits into from
Aug 24, 2023

Conversation

gpleiss
Copy link
Collaborator

@gpleiss gpleiss commented Aug 21, 2023

Fixes #19

CI improvements

  • Allow for only torch/jax unit tests to be run
  • Set up multiple CI environments for stable/unstable torch/jax
  • Run example notebook smoke tests

Necessary fixes discovered by CI improvements

  • Create simple np_fns backend (necessary for PyTorch only mode)

@codecov
Copy link

codecov bot commented Aug 21, 2023

Codecov Report

Merging #22 (561f4b8) into main (781d609) will decrease coverage by 1.81%.
Report is 6 commits behind head on main.
The diff coverage is 25.00%.

@@            Coverage Diff             @@
##             main      #22      +/-   ##
==========================================
- Coverage   82.25%   80.44%   -1.81%     
==========================================
  Files          39       36       -3     
  Lines        3330     2890     -440     
==========================================
- Hits         2739     2325     -414     
+ Misses        591      565      -26     
Files Changed Coverage Δ
cola/ops/operator_base.py 81.36% <25.00%> (-2.44%) ⬇️

... and 6 files with indirect coverage changes

@gpleiss gpleiss force-pushed the improve_testing branch 18 times, most recently from 5405a9c to 0519e73 Compare August 23, 2023 21:58
@gpleiss gpleiss changed the title [WIP] Improve CI Improve CI Aug 24, 2023
When jax is installed, any numpy array will use the jax_fns xnp backend.
When jax is not installed, now all numpy arrays will use the np_fns xnp
backend.

on:
pull_request:
branches: [ main ]
Copy link
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

We used to run CI on all pull requests. For now I think it makes sense to just run it on main, but I could be convinced otherwise.


on:
push:
branches: [ main ]
Copy link
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Here, I feel more strongly that CI should only be run on pushes to the main branch. We were running CI twice for PRs (once on branch push, once for the PR).

These markings make it possible to run the test suite when only one of
the backends are installed.

It also skips big operator tests on on CI.

NOTE: for all future testing, neither torch, jax, cola.torch_fns, nor
cola.jax_fns can be imported in a testing file.

- With the 'not jax' mark, all non-jax tests run.
- With the 'not torch' mark, all non-torch tests run.
- With the 'not big' mark, all non-big LO tests run.
The unit tests are now run on 4 configurations:
- jax installed only
- torch (stable) installed only
- torch (latest/unstable) installed only
- both jax and torch are installed

Codecov report is updated only for the jax + torch backend
configuration.

Other updates:
- Set python-version to 3.10 to match minimum requirements
- Only run CI on PRs/pushes to main branch
@gpleiss
Copy link
Collaborator Author

gpleiss commented Aug 24, 2023

I will leave the "run example notebook smoke tests" task for a separate PR.

@AndPotap AndPotap merged commit 15e66c5 into main Aug 24, 2023
4 checks passed
@AndPotap AndPotap deleted the improve_testing branch August 24, 2023 02:01
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

Improving CI
2 participants