DM-39138: Fix cp_verify test failures

[czwa]

Comparing the image statistics against one realization of the read noise isn't correct.

[erykoff]

I'm sorry, I don't totally follow what is going on here. A couple lines are deleted but I guess the values come from elsewhere...?

[czwa]

Previously the code was setting the read noise value used in the test threshold from the as-measured serial overscan residual stdev. These values are in general smaller than the camera defined read noise values, resulting in cases where the measured image noise is smaller than the camera defined read noise, but more than 5% higher than the overscan residual and are marked as failures. I had originally thought this was fine, as the overscan residual stdev is a measurement of the read noise, but I don't think it's valid to do anymore. I think I have three reasons why a fixed read noise is better:

1. The overscan data is used to determine the overscan fit that is subtracted off, so that standard deviation of that residual should be the smallest possible value.
2. We know the per-row/per-column overscan subtraction adds some noise to the image, and forcing the test to fail because that additional noise isn't constrained by the residual stdev doesn't make sense.
3. This is also just one realization of the amplifier read noise, so depending on the size of the true scatter in the read noise, we may be forcing the test to fail because we're using a value in the tail of that scatter. From the ComCam results posted today, it looks like there's 5-10% scatter in the measured overscan residual stdev, which means this may be the largest factor in determining how many amplifiers pass with this threshold.

[erykoff]

Okay, I think I see what's going on, that verify['NOISE'] is the test but verify['READ_NOISE_CONSISTENT'] is just a check?
I think the comments above the tests could be improved based on what you posted in the PR, and make it clear which "test below (that does not trigger failure)" refers to, because there are a lot of tests below.

[czwa]

Significantly updated comments, and I've separated the advisory check in a way that should make it clearer that it is not a critical test.

[erykoff]

This is much clearer, thanks. One last question: previously, READ_NOISE_CONSISTENT was always filled, and now it's only filled if we have RESIDUAL_STDEV. Just want to confirm that this is okay.

[czwa]

That was intentional, yes. Without the metadata we can't make any claims about the read noise matching, so I felt it was better to just drop it rather than assign any value.

[erykoff]

That's what I figured, and I guess this isn't a required field so nothing will fall down if it's missing?