Add compare_target_qa script#289
Merged
Merged
Conversation
Contributor
|
This seems to be a convenient standalone script. I see no particular reason not to include it, whether or not we build on it or maintain it, so I'm merging. |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
For consideration, this PR adds a
compare_target_qascript for facilitating side-by-side comparisons ofrun_target_qaoutputs, e.g. for comparing the same code on different datasets or different target selections on the same dataset. It has some hardcoded assumptions about some of the files to expect and a naming scheme of{directory}/{prefix}-{targettype}.png, but it does have some mild robustness to additional files being added or missing files from one run to another.Examples outputs are at:
http://portal.nersc.gov/project/desi/users/sjbailey/targetQA/compare/dr4_0.19.0-18.2a_0.19.1.html
Note: these comparisons aren't taken at random; they have some distinct and important differences.
For running at NERSC, it requires the input QA directories to already be web visible because it provides relative paths to those files; it doesn't copy the images into a new location. If the original QA directories are deleted, it will break the outputs of the comparison page too.
Contributed improvements are welcome, but I'm not likely to spend much time personally on polishing this. Take it and run with it if you like, or if @geordie666 / @moustakas find this approach fundamentally flawed or unhelpful, we can also toss this PR and I can keep the code for my own private use. But it seems we want something like this to facilitate QA comparisons without toggling between a bunch of tabs and links.