Skip to content

Hi Processing fixes for SIT4#1785

Merged
tmplummer merged 7 commits intoIMAP-Science-Operations-Center:devfrom
tmplummer:1782-hi-l1a---fix-de-metadata
May 28, 2025
Merged

Hi Processing fixes for SIT4#1785
tmplummer merged 7 commits intoIMAP-Science-Operations-Center:devfrom
tmplummer:1782-hi-l1a---fix-de-metadata

Conversation

@tmplummer
Copy link
Copy Markdown
Contributor

@tmplummer tmplummer commented May 27, 2025

Change Summary

Overview

This is a mix of some fixes needed for Hi processing before SIT4:

  • Fix Hi L1C PSET ISTP warning
  • Add "epoch" dimension to solid_angle variable in SkyMap.to_dataset() code
  • Fix datatype for obs_date
  • Don't check schema for "label" and "delta" variables (checking causes DEPEND_0 to be added)
  • Make energy dependent exposure_factor and obs_date variables the default (Hi and Lo should not have any energy independent versions of these)
  • Get L1C dependencies as input to Hi L2 processing (duh).

Updated Files

  • imap_processing/cdf/config/imap_enamaps_l2-common_variable_attrs.yaml
    • Make energy dependent exposure_factor and obs_date default attributes be the energy dependent version
  • imap_processing/cdf/config/imap_enamaps_l2-healpix_variable_attrs.yaml
    • Make energy dependent exposure_factor and obs_date default attributes be the energy dependent version
  • imap_processing/cdf/config/imap_enamaps_l2-rectangular_variable_attrs.yaml
    • Make energy dependent exposure_factor and obs_date default attributes be the energy dependent version
  • imap_processing/cdf/config/imap_hi_variable_attrs.yaml
    • Fix L1C PSET ISTP warnings
  • imap_processing/cli.py
    • Get L1C dependencies as input to Hi L2 processing (duh).
  • imap_processing/ena_maps/ena_maps.py
    • Add "epoch" dimension to solid_angle variable in SkyMap.to_dataset() code
    • Don't check schema for "label" and "delta" variables (checking causes DEPEND_0 to be added)
  • imap_processing/hi/l2/hi_l2.py
    • Convert obs_date to int64 datatype after calculation of values
  • imap_processing/tests/ena_maps/test_ena_maps.py
    • Add check for correct solid_angle shape
  • imap_processing/tests/hi/test_hi_l2.py
    • Write L2 dataset to cdf in order to get some ISTP checking
  • imap_processing/ultra/l2/ultra_l2.py
    • Update custom attribute getter to swap energy dependent to be the default

Closes: #1782 #1783

@tmplummer tmplummer requested review from a team, nkerman and sdhoyt May 27, 2025 23:51
@tmplummer tmplummer self-assigned this May 27, 2025
@tmplummer tmplummer removed the request for review from a team May 27, 2025 23:51
@tmplummer tmplummer added the Ins: Hi Related to the IMAP-Hi instrument label May 27, 2025
@tmplummer tmplummer added the Ins: Ultra Related to the IMAP-Ultra instrument label May 27, 2025
@tmplummer tmplummer requested a review from anamanica May 27, 2025 23:51
@tmplummer tmplummer added this to IMAP May 27, 2025
greglucas
greglucas approved these changes May 28, 2025
View reviewed changes
Comment thread imap_processing/ena_maps/ena_maps.py
Comment on lines +1278 to +1279
# Don't check schema on label or delta variables
ignore_schema_substrings = ["_label", "_delta"]
Copy link
Copy Markdown
Collaborator

@greglucas greglucas May 28, 2025

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

No need to do anything here, but I wonder if there is a way to handle this upstream in SAMMI somehow 🤔 I've noticed we do this pattern quite a bit. @maxinelasp for awareness.

nkerman
nkerman approved these changes May 28, 2025
View reviewed changes
Copy link
Copy Markdown
Collaborator

@nkerman nkerman left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM as well!

Comment thread imap_processing/tests/ena_maps/test_ena_maps.py
# operations to be repeated as this test
rect_map_ds = rectangular_map.to_dataset()
assert "solid_angle" in rect_map_ds.data_vars
assert rect_map_ds.data_vars["solid_angle"].shape == (
Copy link
Copy Markdown
Collaborator

@nkerman nkerman May 28, 2025

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thanks for fixing + testing this!

@tmplummer tmplummer merged commit a39262f into IMAP-Science-Operations-Center:dev May 28, 2025
39 of 42 checks passed
@tmplummer tmplummer deleted the 1782-hi-l1a---fix-de-metadata branch May 28, 2025 17:54
@github-project-automation github-project-automation bot moved this to Done in IMAP May 28, 2025
@tmplummer tmplummer mentioned this pull request Jun 4, 2025
Closed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Reviewers

@greglucas greglucas greglucas approved these changes

@sdhoyt sdhoyt Awaiting requested review from sdhoyt

@maxinelasp maxinelasp Awaiting requested review from maxinelasp maxinelasp was automatically assigned from IMAP-Science-Operations-Center/sdc-cdf

@anamanica anamanica Awaiting requested review from anamanica anamanica was automatically assigned from IMAP-Science-Operations-Center/sdc-cdf

+1 more reviewer

@nkerman nkerman nkerman approved these changes

Reviewers whose approvals may not affect merge requirements

Assignees

@tmplummer tmplummer

Labels

Ins: Hi Related to the IMAP-Hi instrument Ins: Ultra Related to the IMAP-Ultra instrument

Projects

IMAP
Status: Done

Milestone

No milestone

Development

Successfully merging this pull request may close these issues.

Hi L1A - fix de metadata

3 participants

@tmplummer @greglucas @nkerman