Fix test.sh#54
Conversation
Unfortunately, when I refactored test.sh to make it more resilient, I broke it. :-( Specifically, it was always returning 0, even if some tests failed. In turn, Github CI was not failing when tests where failing.
danielhollas
force-pushed
the
fix-tests-again
branch
from
18c1c50 to
9bb2a5a
Compare
1. Use of unitialized variable `equant` in analysis.F90 caused any non-PI test to fail when compiled with floating-point checks (uncovered in PR #50). Fortunately, this bug did not really affect anything because we don't currently print equant anywhere. 2. Non-significant numerical difference in the HARMON test, which started showing up after a fix to numdiff.py in 4620895 in #51.
Codecov Report
@@ Coverage Diff @@
## master #54 +/- ##
==========================================
- Coverage 59.78% 59.71% -0.08%
==========================================
Files 36 36
Lines 5575 5575
==========================================
- Hits 3333 3329 -4
- Misses 2242 2246 +4
|
|
@suchanj can you take a look? The coverage change in |
|
It all makes sense now, good job @danielhollas unraveling this mystery. I agree with dedicated PRs for test.sh. I wish this could be enforced by some security settings, but minding the red flags and co-reviewing should do. |
|
Yeah, there were multiple red flags that I didn't paid closer attention to, this one is on me. |
2 participants
Unfortunately, when I refactored
test.shin #50 to make it more resilient, I broke it. :-(Specifically, it was always returning 0, even if some tests failed.
In turn, Github CI was not failing when tests where failing. (as seen e.g. here
And this is also a reason why the coverage changes in #49 were so wonky...I should have taken a hint.
Apart from fixing the
test.sh, I'm also fixing the failing tests here.Use of unitialized variable
equantinanalysis.F90caused any non-Path-Integral test to fail when compiled withfloating-point checks (uncovered in PR Move source to src/ #50).
Fortunately, this bug did not really affect anything because we don't currently print equant anywhere.
Non-significant numerical difference in the HARMON test,
which started showing up after a fix to
numdiff.pyin 4620895 in MAJOR fix to tests! #51.A lesson here is, never touch
test.shagain without opening a dedicated PR that can be carefully reviewed.The error was introduced in PR #50 which unrelated to tests.