Update refpix step for NIR detectors to use convolution kernel

[stscijgbot-jp]

Issue JP-3622 was created on JIRA by David Law:

As part of an ongoing multi-instrument discussion of ways in which to improve 1/f noise in NIR data (see https://outerspace.stsci.edu/display/JWSTCC/JWST+Pipeline+1-over-f+Noise+Removal]) there was general agreement on a desired improvement to the refpix step which should allow reference pixels to mitigate 1/f noise as much as possible without the need for using science pixels.  In brief, instead of the side-reference pixel subtraction using a simple median smoothed signal as a function of row, it should instead use an optimized convolution kernel to do the smoothing prior to subtraction.  Algorithm and example implementation provided by Timothy Brandt (in collaboration with Bernie Rauscher).

The impact of this change on rate files is quite modest, improving the total noise by a few percent.  Likewise, there is not a large impact on the visible residual 1/f striping.  Nonetheless, this change allows us to maximize the gains that can be realized without science-specific solutions.

Details:

• This new method requires convolution of the side reference pixels by a realspace kernel.  These kernels should be provided as CRDS reference files to allow for future flexibility in changing the kernels if necessary.
• The new method should be added as a new (and default) option, but the simple-median kernel should also be kept as a user-selectable option as well.
• This method is only relevant to non-IRS2 readout modes, as IRS2 readout patterns already implement something similar on board.

A notebook is attached that demonstrates the use of this approach in a custom post-hook step appended to the refpix step (passing in the kernels for convenience as global variables- these would come from reference files instead).

This ticket will require multiple steps to implement: defining a new CRDS reference file format, implementing the necessary code changes, creating and delivering the new reference files for each NIR instrument, and potentially updating parameter reference files.  As such it is not a priority for Build 11.0, but may be for Build 11.1.