Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

JP-3088 updated bunit_data for the top multislit model #8189

Merged
merged 5 commits into from
Jan 19, 2024

Conversation

jemorrison
Copy link
Contributor

@jemorrison jemorrison commented Jan 11, 2024

Resolves JP-3088

Closes #7792

This PR was described in JP-3088 as "exp_to_source is copying wrong and/or incomplete meta information." For a multislt data model the model.meta.bunit_data had a different value than model.slit[*].meta.bunit_data.
This even occurs in the cal file after calspec2. So I think the issue is related to calspec2.

The multslitmodel is create in extract_2d/nirspec.py:
output_model = datamodels.MultiSlitModel()
output_model.update(input_model)

The slit bunit values are copies of output_model (L95-96) in nirspec.py

Copy BUNIT values to output slit

        new_model.meta.bunit_data = input_model.meta.bunit_data
        new_model.meta.bunit_err = input_model.meta.bunit_err

In the photom step the slit bunit values are updated (starting L903) but not the upper datamodel. I added to those
line for the PR that. sets the input.meta.bunit values
if not self.inverse:
if unit_is_surface_brightness:
slit.meta.bunit_data = 'MJy/sr'
slit.meta.bunit_err = 'MJy/sr'
else:
slit.meta.bunit_data = 'MJy'
slit.meta.bunit_err = 'MJy'

Checklist for maintainers

  • added entry in CHANGES.rst within the relevant release section
  • updated or added relevant tests
  • updated relevant documentation
  • added relevant milestone
  • added relevant label(s)
  • ran regression tests, post a link to the Jenkins job below.
    How to run regression tests on a PR
  • Make sure the JIRA ticket is resolved properly

@jemorrison jemorrison requested a review from a team as a code owner January 11, 2024 19:41
@jemorrison
Copy link
Contributor Author

@stscieisenhamer Could you review this and give me some feedback if this is not what you intended.

@jemorrison jemorrison added this to the Build 10.2 milestone Jan 11, 2024
Copy link

codecov bot commented Jan 11, 2024

Codecov Report

All modified and coverable lines are covered by tests ✅

Comparison is base (649b28f) 75.24% compared to head (d4a7ca3) 75.25%.

Additional details and impacted files
@@           Coverage Diff           @@
##           master    #8189   +/-   ##
=======================================
  Coverage   75.24%   75.25%           
=======================================
  Files         470      470           
  Lines       38431    38435    +4     
=======================================
+ Hits        28919    28923    +4     
  Misses       9512     9512           
Flag Coverage Δ *Carryforward flag
nightly 77.37% <ø> (ø) Carriedforward from 649b28f

*This pull request uses carry forward flags. Click here to find out more.

☔ View full report in Codecov by Sentry.
📢 Have feedback on the report? Share it here.

Copy link
Collaborator

@stscieisenhamer stscieisenhamer left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

That looks to be what exactly needs done. LGTM

@jemorrison
Copy link
Contributor Author

@jemorrison
Copy link
Contributor Author

@hbushouse The results of the regression test are at:https://plwishmaster.stsci.edu:8081/job/RT/job/JWST-Developers-Pull-Requests/1143/
I believe all the "failed" tests are expected with this change.
Could you confirm that.

Copy link
Collaborator

@hbushouse hbushouse left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This approach will certainly fix the issue originally reported, but I also think it's probably not appropriate for us to be setting any values at all for bunit_data and bunit_err in the primary header of a MultiSlitModel. It really should only be set in each slit's meta data. I'm curious as to what would happen if instead of updating the primary values here, we simply set them to python None, so that they'd essentially disappear from the primary header. What would then happen downstream when such cal files go through exp_to_source? Would it correctly ignore the non-existent values for BUNIT in the primary header and instead only pay attention to (and copy) the values from the individual slit SCI headers?

jwst/photom/photom.py Outdated Show resolved Hide resolved
CHANGES.rst Outdated Show resolved Hide resolved
jwst/photom/photom.py Show resolved Hide resolved
@hbushouse
Copy link
Collaborator

hbushouse commented Jan 18, 2024

Full regtest run at https://plwishmaster.stsci.edu:8081/job/RT/job/JWST-Developers-Pull-Requests/1159/

These latest results reveal something odd going on. All of the (relatively few) failures are due to "BUNIT = 'DN/s'" entries that are now missing from the SCI and ERR extension headers of s2d products. At first sight this seems like a good thing, but the s2d SCI and ERR extension headers should have BUNIT keywords in them and they should have values of either "MJy" or "MJy/sr", depending on whether the individual slit instance to which they apply contains a point or extended source. The "cal" products have the correct MJy or MJy/sr settings, but apparently when the cal products go through resample_spec (to create an s2d product) the BUNIT keyword values are not getting properly carried over and revert to their original "DN/s" values. So there's another problem somewhere, in addition to the one trying to be solved by this PR.

@hbushouse
Copy link
Collaborator

OK, upon further analysis I think I can take back what I said about there being another problem that was causing s2d products to have BUNIT='DN/s' in their SCI and ERR headers. If I take a cal file and apply the changes that are in this PR (i.e. I set model.meta.bunit_data and model.meta.bunit_err to None, but don't touch the meta data in the slit instances) and then run the resample_spec step on it, all the BUNIT values in all the SCI and ERR extensions come out set to the proper "MJy" and "MJy/sr" values, instead of "DN/s", like they were previously. So that gives me confidence that the so-called "failures" seen in the regtest run for this PR branch are in fact what we want to happen. So I'm green-lighting this.

Copy link
Collaborator

@hbushouse hbushouse left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM

@hbushouse hbushouse merged commit dac66be into spacetelescope:master Jan 19, 2024
29 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
Projects
None yet
Development

Successfully merging this pull request may close these issues.

exp_to_source not transferring the slit meta appropriately
3 participants