JP-3455: Updated MIRI imager photom step

[mlibralato]

Resolves JP-3455

Closes #8048

This PR is related to proposed changes to the PHOTOM step for the MIRI Imager (and coronographs). We add a time-dependent correction to the PHOTOM value.

Checklist for maintainers

• added entry in CHANGES.rst within the relevant release section
• updated or added relevant tests
• updated relevant documentation
• added relevant milestone
• added relevant label(s)
• ran regression tests, post a link to the Jenkins job below.
  How to run regression tests on a PR
• Make sure the JIRA ticket is resolved properly

[codecov]

Codecov Report

Attention: 10 lines in your changes are missing coverage. Please review.

Comparison is base (2a70810) 75.92% compared to head (efb4e69) 75.91%.

Files	Patch %	Lines
jwst/photom/photom.py	46.15%	7 Missing ⚠️
jwst/photom/miri_imager.py	62.50%	3 Missing ⚠️

Additional details and impacted files

@@            Coverage Diff             @@
##           master    #8096      +/-   ##
==========================================
- Coverage   75.92%   75.91%   -0.02%     
==========================================
  Files         459      460       +1     
  Lines       37606    37626      +20     
==========================================
+ Hits        28554    28564      +10     
- Misses       9052     9062      +10     

Flag	Coverage Δ		*Carryforward flag
nightly	77.37% <ø> (-0.01%)	⬇️	Carriedforward from 2a70810

*This pull request uses carry forward flags. Click here to find out more.

☔ View full report in Codecov by Sentry.
📢 Have feedback on the report? Share it here.

[mlibralato]

@hbushouse Let me know if everything works now.

[hbushouse]

All of the CI failures are due to errors in the jwst/photom/tests/test_photom.py unit test module when it tries to access the new "timecoeff" attribute in the ref file model, but it's not defined here, due to the fact that the CI test environment is still using the latest stdatamodels release, rather than stdatamodels/master.

@mlibralato Can you try to verify that running "pytest test_photom.py" in your local branch, which presumably has the latest stdatamodels updates, succeeds?

[hbushouse]

Also, I still can't run the full regression test suite on this PR branch due to some kind of password protection or permission issue with your fork of the repo. So once you've made the suggested changes on your local PR branch, can you at least run a few of the regression test modules manually, just so we have some kind of assurance that this works OK? To do so, go to the /jwst/jwst/regtest directory in your local branch and execute the MIRI imager tests using:

pytest --bigdata test_miri_image.py

Note that you have to first setup the environment variable export TEST_BIGDATA=https://bytesalad.stsci.edu/artifactory so that Jenkins knows where to get the input data files used by the tests.

[mlibralato]

All of the CI failures are due to errors in the jwst/photom/tests/test_photom.py unit test module when it tries to access the new "timecoeff" attribute in the ref file model, but it's not defined here, due to the fact that the CI test environment is still using the latest stdatamodels release, rather than stdatamodels/master.

@mlibralato Can you try to verify that running "pytest test_photom.py" in your local branch, which presumably has the latest stdatamodels updates, succeeds?

I tested the code before pushing it and with the updated stdatamodels I get a "pass".

[mlibralato]

Also, I still can't run the full regression test suite on this PR branch due to some kind of password protection or permission issue with your fork of the repo. So once you've made the suggested changes on your local PR branch, can you at least run a few of the regression test modules manually, just so we have some kind of assurance that this works OK? To do so, go to the /jwst/jwst/regtest directory in your local branch and execute the MIRI imager tests using:

pytest --bigdata test_miri_image.py

Note that you have to first setup the environment variable export TEST_BIGDATA=https://bytesalad.stsci.edu/artifactory so that Jenkins knows where to get the input data files used by the tests.

The test failed but I wonder if all errors are due to the change in the PHOTOM step itself.

[hbushouse]

Also, I still can't run the full regression test suite on this PR branch due to some kind of password protection or permission issue with your fork of the repo. So once you've made the suggested changes on your local PR branch, can you at least run a few of the regression test modules manually, just so we have some kind of assurance that this works OK? To do so, go to the /jwst/jwst/regtest directory in your local branch and execute the MIRI imager tests using:
pytest --bigdata test_miri_image.py
Note that you have to first setup the environment variable export TEST_BIGDATA=https://bytesalad.stsci.edu/artifactory so that Jenkins knows where to get the input data files used by the tests.

The test failed but I wonder if all errors are due to the change in the PHOTOM step itself.

That's probably the case. I noticed that the differences start with the "cal" product and proceed downstream from there, which is exactly what you'd expect from changes in the photom correction (which is the last step before "cal" is created).

[mlibralato]

Also, I still can't run the full regression test suite on this PR branch due to some kind of password protection or permission issue with your fork of the repo. So once you've made the suggested changes on your local PR branch, can you at least run a few of the regression test modules manually, just so we have some kind of assurance that this works OK? To do so, go to the /jwst/jwst/regtest directory in your local branch and execute the MIRI imager tests using:
pytest --bigdata test_miri_image.py
Note that you have to first setup the environment variable export TEST_BIGDATA=https://bytesalad.stsci.edu/artifactory so that Jenkins knows where to get the input data files used by the tests.

The test failed but I wonder if all errors are due to the change in the PHOTOM step itself.

That's probably the case. I noticed that the differences start with the "cal" product and proceed downstream from there, which is exactly what you'd expect from changes in the photom correction (which is the last step before "cal" is created).

So, what should I do? Just ignore it for now? Have you solved the issue with the permission? If you show me the error, I can try to see on my end what I can do

[hbushouse]

All of the CI failures are due to the use of the last released version of stdatamodels, which doesn't have the updates to the MIRI imager photom data model. Note how the one using "devdeps" passes, because it's using the dev version of stdatamodels. So this all looks OK to me now.