RCAL-751 Create FluxStep to apply the flux correction to Level 2 data.

[stscieisenhamer]

Resolves RCAL-751

This PR implements the flux correction to convert from native Level 2 electrons/s units to MJy/sr.

Checklist

• added entry in CHANGES.rst under the corresponding subsection
• updated relevant tests
• updated relevant documentation
• updated relevant milestone(s)
• added relevant label(s)
• ran regression tests, post a link to the Jenkins job below. How to run regression tests on a PR

[stscieisenhamer]

Of the functional items still needed, the main one is ensuring file naming is correct if step results are saved. There is also documentation and a regression test needed. However, the main code is ready for initial review.

The main design point that needs to be considered is that, currently, there is no optimization concerning memory management. I believe there are still larger discussions on the best solutions for this, so I have left the code dead-simple: All is presumed in-memory and the input model(s) are modified in-place.

[codecov]

Codecov Report

Attention: Patch coverage is 79.16667% with 10 lines in your changes are missing coverage. Please review.

Project coverage is 76.86%. Comparing base (2c5bb3a) to head (bdc8b6d).
Report is 1 commits behind head on main.

Files	Patch %	Lines
romancal/flux/flux_step.py	77.77%	10 Missing ⚠️

Additional details and impacted files

@@            Coverage Diff             @@
##             main    #1120      +/-   ##
==========================================
+ Coverage   76.84%   76.86%   +0.01%     
==========================================
  Files         105      107       +2     
  Lines        7053     7101      +48     
==========================================
+ Hits         5420     5458      +38     
- Misses       1633     1643      +10     

Flag	Coverage Δ		*Carryforward flag
nightly	62.70% <ø> (ø)		Carriedforward from 324f433

*This pull request uses carry forward flags. Click here to find out more.

☔ View full report in Codecov by Sentry.
📢 Have feedback on the report? Share it here.

[schlafly]

This looks good. Have you verified that if you plug it into the HLP and feed it an association that it does the right thing, calibrating the images and then later steps seeing those calibrated images?

The step probably needs to be registered here:

romancal/romancal/step.py

Line 16 in 78532b9

from .resample.resample_step import ResampleStep

and get some short docs.

I'm still hung up by the fact that I think the one time I used the HLP the model container contained a list of file names rather than a list of in-memory models, and so used this branch
https://github.com/spacetelescope/romancal/blob/main/romancal/datamodels/container.py#L211
where it's not clear to me that anything would get saved. Maybe not part of this PR, but it would be good if you could figure out how to call the pipeline in a way such that this step would work.

I imagine that the sky subtraction & outlier detection steps are in a similar situation, but I am not sure.

[stscieisenhamer]

HLP has some implementation issues. In its current form, it can only operate if all is in-memory and the steps work on the data in-place, and that is just to get around the syntax issues; see how result and input relate from step-to-step.

Workin' it...

[stscieisenhamer]

Status: Main work is complete, I believe. Two outstanding issues to be discussed:

Unit change of Level 2: The presumption here is still e/s. Should I update to DN/s or wait until the related rad changes are merged first?

HLP: Between RCAL-777 and other structure/memory-related discussions, I have done minimal updates and xfailed the related test. I would suggest disconnecting any further HLP work from this PR and leave it for other issues + PR's.

[schlafly]

Thanks, this looks good. The diff is a little hard to read because somehow all of Nadia's doc changes got in there too?

My understanding from the RCAL-777 discussion is that Dave has an invocation that disables step pars that should work, so let's not let that make us set an xfail.

[stscieisenhamer]

I have left units at e/s, just for test/validation to pass. However, if merged before the unit change is in, all that needs to change is this line.

[schlafly]

This looks good to me. Dave indicates in the rcal-777 discussion that you should be able to get the hlp test to pass with an invocation along the lines of
strun --disable-crds-steppar roman_hlp mosaic_asn.json --verbose
does that not work for you?

[schlafly]

LGTM, thanks!

[stscieisenhamer]

Errors are due to the unit change for Level 2 from e/s to DN/s. This should resolve with the recent rad/rdm merges.