Skip to content

Add compare_target_qa script#289

Merged
geordie666 merged 1 commit into
masterfrom
compare-qa
Mar 5, 2018
Merged

Add compare_target_qa script#289
geordie666 merged 1 commit into
masterfrom
compare-qa

Conversation

@sbailey
Copy link
Copy Markdown
Contributor

@sbailey sbailey commented Mar 5, 2018

For consideration, this PR adds a compare_target_qa script for facilitating side-by-side comparisons of run_target_qa outputs, e.g. for comparing the same code on different datasets or different target selections on the same dataset. It has some hardcoded assumptions about some of the files to expect and a naming scheme of {directory}/{prefix}-{targettype}.png, but it does have some mild robustness to additional files being added or missing files from one run to another.

Examples outputs are at:

Note: these comparisons aren't taken at random; they have some distinct and important differences.

For running at NERSC, it requires the input QA directories to already be web visible because it provides relative paths to those files; it doesn't copy the images into a new location. If the original QA directories are deleted, it will break the outputs of the comparison page too.

Contributed improvements are welcome, but I'm not likely to spend much time personally on polishing this. Take it and run with it if you like, or if @geordie666 / @moustakas find this approach fundamentally flawed or unhelpful, we can also toss this PR and I can keep the code for my own private use. But it seems we want something like this to facilitate QA comparisons without toggling between a bunch of tabs and links.

@geordie666
Copy link
Copy Markdown
Contributor

This seems to be a convenient standalone script. I see no particular reason not to include it, whether or not we build on it or maintain it, so I'm merging.

@geordie666 geordie666 merged commit 9d18ba2 into master Mar 5, 2018
@geordie666 geordie666 deleted the compare-qa branch March 5, 2018 17:36
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants