Use CircleCI Matrix Jobs #1043
Use CircleCI Matrix Jobs #1043
Conversation
|
Verified that Proper CI tests are being run:
|
.circleci/config.yml
Outdated
| condition: << parameters.codecov >> | ||
| condition: | ||
| and: | ||
| - equal: [ "3.8", << parameters.python_version >> ] |
There was a problem hiding this comment.
Only runs codecov when python == 3.8 and core_dependencies is true
There was a problem hiding this comment.
Why is this change necessary?
I think its fine to keep the existing code here, and simply not configure a 3.6 or 3.7 job to run codecov in the jobs section
I would prefer the jobs section below spells out the behavior of each job, rather than having to read the code inside each job (i.e. this updated check) in order to understand the behavior.
There was a problem hiding this comment.
@gsheni , on reflection, this change is incorrect and needs to be deleted.
We have unit tests which only run if core_dependencies==False, and other unit tests which only run if core_dependencies==True. So, we need to run codecov on python 3.8, both with and without core_dependencies set to True, because then when we upload both reports to codecov, they get merged. Without this, codecov will fail cryptically, as it has done on this PR.
There was a problem hiding this comment.
Changed the code to
matrix:
parameters:
python_version: ["3.8"]
core_dependencies: [false, true]
codecov: [true]
- So codecove runs with both core true, false on 3.8
Codecov Report
@@ Coverage Diff @@
## main #1043 +/- ##
=======================================
Coverage 99.91% 99.91%
=======================================
Files 183 183
Lines 10147 10147
=======================================
Hits 10138 10138
Misses 9 9 Continue to review full report at Codecov.
|
.circleci/config.yml
Outdated
| parameters: | ||
| python_version: ["3.8"] | ||
| core_dependencies: [false, true] | ||
| name: "linux python << matrix.python_version >> unit tests, core dependencies << matrix.core_dependencies >>" |
There was a problem hiding this comment.
@gsheni can we reorganize this so that we run 3.6/3.7/3.8 with core_dependencies=false, then run 3.8 with core_dependencies=True as a normal, non-matrix job?
.circleci/config.yml
Outdated
| condition: << parameters.codecov >> | ||
| condition: | ||
| and: | ||
| - equal: [ "3.8", << parameters.python_version >> ] |
There was a problem hiding this comment.
Why is this change necessary?
I think its fine to keep the existing code here, and simply not configure a 3.6 or 3.7 job to run codecov in the jobs section
I would prefer the jobs section below spells out the behavior of each job, rather than having to read the code inside each job (i.e. this updated check) in order to understand the behavior.
.circleci/config.yml
Outdated
| @@ -53,7 +52,6 @@ jobs: | |||
| parameters: | |||
| python_version: | |||
| type: string | |||
| default: "3.8" | |||
There was a problem hiding this comment.
Just to confirm: deleting these doesn't change the current behavior at all, does it?
There was a problem hiding this comment.
No, this doesn't. It just makes it so you have to specific the python version when running this job. If you don't specific it, the job will error, and CI will fail with missing parameter error
.circleci/config.yml
Outdated
| default: "3.8" | ||
| codecov: | ||
| type: boolean | ||
| default: false |
There was a problem hiding this comment.
Why delete the default for codecov?
|
I like the way the parameters always appear in the names of the test jobs now, that's helpful
|
|
@dsherry I put back the default arguments for the jobs. It seems that CircleCI matrix is fine with them having them in. In addition, I made it so codecov runs with core True and False on python 3.8 |
| python_version: ["3.8"] | ||
| core_dependencies: [false, true] | ||
| codecov: [true] | ||
| name: "linux python << matrix.python_version >> unit tests, core dependencies << matrix.core_dependencies >>, codecov << matrix.codecov >>" |
There was a problem hiding this comment.
Ohhhh, @gsheni , seeing this now, I realize I gave you a bad recommendation in my change request. Sorry about that.
This code currently runs the following jobs:
- python 3.6 lint test - Success
- python 3.7 lint test - Success
- python 3.8 lint test - Success
- linux python 3.6 unit tests, core dependencies false, codecov false - Success
- linux python 3.7 unit tests, core dependencies false, codecov false - Success
- linux python 3.8 unit tests, core dependencies false, codecov false - Success
- linux python 3.8 unit tests, core dependencies false, codecov true - Success
- linux python 3.8 unit tests, core dependencies true, codecov true - Success
Note the last 3 jobs all run python 3.8. We should not run this one, because it's unnecessary:
- linux python 3.8 unit tests, core dependencies false, codecov false - Success
So what I think we should do here
- unit_tests:
matrix:
parameters:
python_version: ["3.6", "3.7"]
core_dependencies: [false]
name: "linux python << matrix.python_version >> unit tests, core dependencies << matrix.core_dependencies >>, codecov false"
- unit_tests:
matrix:
parameters:
python_version: ["3.8"]
core_dependencies: [false, true]
codecov: [true]
name: "linux python << matrix.python_version >> unit tests, core dependencies << matrix.core_dependencies >>, codecov << matrix.codecov >>"
| core_dependencies: [false] | ||
| name: "linux python << matrix.python_version >> unit tests, core dependencies << matrix.core_dependencies >>, codecov false" | ||
| codecov: [false] |
There was a problem hiding this comment.
This way someone in the future can explicitly see the argument being passed
Use CircleCI Matrix Jobs