ENH #1854: Tests will check for alternate baselines using basename_[1…#1943
ENH #1854: Tests will check for alternate baselines using basename_[1…#1943doutriaux1 merged 1 commit intomasterfrom
Conversation
9f0dbbd to
e9bc399
Compare
e9bc399 to
6834146
Compare
|
LGTM 👍 |
…-9].ext Previously the pattern used was baseline.*\.ext
6834146 to
0bc3f4d
Compare
|
@doutriaux1 Please review. |
|
@danlipsa I'm cool with your changes, but in general I think it's unsafe to use alternate baselines. First it adds work to maintaining the baselines, also I'm always worried a test gets better doesn't need the laternate baseline anymore, then we fix an issue with that test. A few week later a new change breaks the test but we don't see it because the old "bad" baseline is still here and passes. |
|
@doutriaux1 I am not happy with alternate baselines but I think they are a necessary evil. It boils down to the following choice: Should we spend a week to debug a small difference in how images are rendered on a specific platform, possibly narrow it down to a OpenGL difference in how the standard is interpreted or a OpenGL bug -- or spend a minute to add a new baseline. VTK uses alternate baselines. We run on many platforms/compilers. I think we only use 3 alternate baselines at most. For uvcdat I think we have at most one. |
|
agreed |
…-9].ext
Previously the pattern used was baseline.*.ext