Cassis Template Updated BandBin and AlphaCube Groups #4482

amystamile-usgs · 2021-05-20T22:08:01Z

Description

BandBin group uses TgoCassisMosaicBandBin.trn/TgoCassisBanBin.trn information. AlphaCube uses logic from line 449-463 from tgocassis2isis main.cpp. Also cleaned up the template by including variables.

I am unsure if I found a bug in tgocassis2isis but the BandBin is outputting different than the translation file values in TgoCassisMosaicBandBin.trn.

Example of truth data output

  Group = BandBin
    FilterName = BLU
    Center     = 485 <nm>
    Width      = 165 <nm>
    NaifIkCode = -143424
  End_Group

Versus using the translated outputs with new template

 Group = BandBin
    FilterName = BLU
    Center     = 497.4 <nm>
    Width      = 134.3 <nm>
    NaifIkCode = -143424
  End_Group

This is what the TgoCassisMosaicBandBin.trn file has

Group = Center
  Auto
  InputPosition  = (Observation_Area, Discipline_Area, img:Imaging, img:Image_Product_Information, img:Filter)
  InputKey       = img:filter_name
  OutputName     = Center
  OutputPosition = (Object, IsisCube, Group, BandBin)
  Translation    = (677.4, PAN)
  Translation    = (497.4, BLU)
  Translation    = (835.4, RED)
  Translation    = (940.2, NIR)

Group = Width
  Auto
  InputPosition  = (Observation_Area, Discipline_Area, img:Imaging, img:Image_Product_Information, img:Filter)
  InputKey       = img:filter_name
  OutputName     = Width
  OutputPosition = (Object, IsisCube, Group, BandBin)
  Translation    = (231.5, PAN)
  Translation    = (134.3, BLU)
  Translation    = (98.0, RED)
  Translation    = (120.6, NIR)

Related Issue

Motivation and Context

PDS4 Sprint

How Has This Been Tested?

Screenshots (if appropriate):

Types of changes

Bug fix (non-breaking change which fixes an issue)
New feature (non-breaking change which adds functionality)
Documentation change (update to the documentation; no code change)
Breaking change (fix or feature that would cause existing functionality to change)

Checklist:

I have read and agree to abide by the Code of Conduct
I have read the CONTRIBUTING document.
My change requires a change to the documentation.
I have updated the documentation accordingly.
I have added tests to cover my changes.
All new and existing tests passed.
I have added myself to the .zenodo.json document.
I have added any user impacting changes to the CHANGELOG.md document.

Licensing

This project is mostly composed of free and unencumbered software released into the public domain, and we are unlikely to accept contributions that are not also released into the public domain. Somewhere near the top of each file should have these words:

This work is free and unencumbered software released into the public domain. In jurisdictions that recognize copyright laws, the author or authors of this software dedicate any and all copyright interest in the software to the public domain.

I dedicate any and all copyright interest in this software to the public domain. I make this dedication for the benefit of the public at large and to the detriment of my heirs and successors. I intend this dedication to be an overt act of relinquishment in perpetuity of all present and future rights to this software under copyright law.

amystamile-usgs · 2021-05-20T22:09:37Z

isis/appdata/import/cassis.lbl.tpl

+  {% if exists(CassHeader) %}
+    {% set windowNumber=int(PO.CaSSIS_Header.FSW_HEADER.attrib_WindowCounter) + 1  %}
+    {% if windowNumber == 1 %}
+        {% set frameletStartSample = int(PO.CaSSIS_Header.PEHK_HEADER.attrib_Window1_Start_Col) + 1 %}
+        {% set frameletEndSample = int(PO.CaSSIS_Header.PEHK_HEADER.attrib_Window1_End_Col) + 1 %}
+        {% set frameletStartLine = int(PO.CaSSIS_Header.PEHK_HEADER.attrib_Window1_Start_Row) + 1 %}
+        {% set frameletEndLine = int(PO.CaSSIS_Header.PEHK_HEADER.attrib_Window1_End_Row) + 1 %}
+    {% endif %}
+    {% if windowNumber == 2 %}
+        {% set frameletStartSample = int(PO.CaSSIS_Header.PEHK_HEADER.attrib_Window2_Start_Col) + 1 %}
+        {% set frameletEndSample = int(PO.CaSSIS_Header.PEHK_HEADER.attrib_Window2_End_Col) + 1 %}
+        {% set frameletStartLine = int(PO.CaSSIS_Header.PEHK_HEADER.attrib_Window2_Start_Row) + 1 %}
+        {% set frameletEndLine = int(PO.CaSSIS_Header.PEHK_HEADER.attrib_Window2_End_Row) + 1 %}
+    {% endif %}
+    {% if windowNumber == 3 %}
+        {% set frameletStartSample = int(PO.CaSSIS_Header.PEHK_HEADER.attrib_Window3_Start_Col) + 1 %}
+        {% set frameletEndSample = int(PO.CaSSIS_Header.PEHK_HEADER.attrib_Window3_End_Col) + 1 %}
+        {% set frameletStartLine = int(PO.CaSSIS_Header.PEHK_HEADER.attrib_Window3_Start_Row) + 1 %}
+        {% set frameletEndLine = int(PO.CaSSIS_Header.PEHK_HEADER.attrib_Window3_End_Row) + 1 %}
+    {% endif %}
+    {% if windowNumber == 4 %}
+        {% set frameletStartSample = int(PO.CaSSIS_Header.PEHK_HEADER.attrib_Window4_Start_Col) + 1 %}
+        {% set frameletEndSample = int(PO.CaSSIS_Header.PEHK_HEADER.attrib_Window4_End_Col) + 1 %}
+        {% set frameletStartLine = int(PO.CaSSIS_Header.PEHK_HEADER.attrib_Window4_Start_Row) + 1 %}
+        {% set frameletEndLine = int(PO.CaSSIS_Header.PEHK_HEADER.attrib_Window4_End_Row) + 1 %}
+    {% endif %}
+    {% if windowNumber == 5 %}
+        {% set frameletStartSample = int(PO.CaSSIS_Header.PEHK_HEADER.attrib_Window5_Start_Col) + 1 %}
+        {% set frameletEndSample = int(PO.CaSSIS_Header.PEHK_HEADER.attrib_Window5_End_Col) + 1 %}
+        {% set frameletStartLine = int(PO.CaSSIS_Header.PEHK_HEADER.attrib_Window5_Start_Row) + 1 %}
+        {% set frameletEndLine = int(PO.CaSSIS_Header.PEHK_HEADER.attrib_Window5_End_Row) + 1 %}
+    {% endif %}
+    {% if windowNumber == 6 %}
+        {% set frameletStartSample = int(PO.CaSSIS_Header.PEHK_HEADER.attrib_Window6_Start_Col) + 1 %}
+        {% set frameletEndSample = int(PO.CaSSIS_Header.PEHK_HEADER.attrib_Window6_End_Col) + 1 %}
+        {% set frameletStartLine = int(PO.CaSSIS_Header.PEHK_HEADER.attrib_Window6_Start_Row) + 1 %}
+        {% set frameletEndLine = int(PO.CaSSIS_Header.PEHK_HEADER.attrib_Window6_End_Row) + 1 %}
+    {% endif %}


This is pretty ugly but its the only alternative I have figured out so far to avoid adding code for this in isisimport.

Good enough for now

kberryUSGS

This looks great to me! One comment / question.

kberryUSGS · 2021-05-24T15:40:48Z

isis/appdata/import/cassis.lbl.tpl

@@ -1,3 +1,8 @@
+{% set PO=Product_Observational %}


This is a great idea! Makes it much easier to read.

We may need to be more specific for this file name. There are likely more cassis templates to come.

kberryUSGS · 2021-05-24T15:43:23Z

isis/appdata/import/cassis.lbl.tpl

+        {% set frameletStartLine = int(PO.CaSSIS_Header.PEHK_HEADER.attrib_Window6_Start_Row) + 1 %}
+        {% set frameletEndLine = int(PO.CaSSIS_Header.PEHK_HEADER.attrib_Window6_End_Row) + 1 %}
+    {% endif %}
+
  Group = AlphaCube
    AlphaSamples        = 2048


Can these get filled in by the template, too?

These are defaulted to '2048' in tgocassis2isis so I left it as is.

That works, then!

kberryUSGS · 2021-05-24T15:46:40Z

@amystamile-usgs Thanks, that is definitely a bug! The correct values are the ones in TgoCassisMosaicBandBin.trn so:

PAN: wavelength - 677.4; bandwidth - 231.5
BLU: wavelength - 497.4; bandwidth - 134.3
RED: wavelength - 835.4; bandwidth - 98.0
NIR: wavelength - 940.2; bandwidth - 120.6

is it the non-moasic BandBin translation file that needs to be updated?

amystamile-usgs · 2021-05-24T15:59:52Z

@amystamile-usgs Thanks, that is definitely a bug! The correct values are the ones in TgoCassisMosaicBandBin.trn so:

PAN: wavelength - 677.4; bandwidth - 231.5
BLU: wavelength - 497.4; bandwidth - 134.3
RED: wavelength - 835.4; bandwidth - 98.0
NIR: wavelength - 940.2; bandwidth - 120.6

is it the non-moasic BandBin translation file that needs to be updated?

The non-moasic BandBin translation file also has these same values so I am confused where it is grabbing the incorrect values in tgocassis2isis.

amystamile-usgs · 2021-05-24T17:26:32Z

@kberryUSGS I went ahead and added a commit to this PR that adds the Pixel Group in the template. That way we can start fully testing it.

amystamile-usgs · 2021-05-24T17:29:14Z

isis/appdata/import/cassis.lbl.tpl

+    End_Group
+
+    Group = Pixels
+      {% if exists("Product_Observational.File_Area_Observational.Array_3D_Image.Element_Array.idl_data_type") %}


idl_data_type strangely does not exist in reingested cassis images. I have no idea how this is being handled correctly in tgocassis2isis. Therefore I had to put a if statement to default to "Real" for this instance.

@kberryUSGS, @amystamile-usgs, We probably don't want to track down where it disappeared, so the default is good for this round. If we find out it was just not transferred at some stage, then we can fix it later. Any objections?

scsides

merge whenever you want

scsides · 2021-05-24T20:20:46Z

isis/appdata/import/cassis.lbl.tpl

+    End_Group
+
+    Group = Pixels
+      {% if exists("Product_Observational.File_Area_Observational.Array_3D_Image.Element_Array.idl_data_type") %}


@kberryUSGS, @amystamile-usgs, We probably don't want to track down where it disappeared, so the default is good for this round. If we find out it was just not transferred at some stage, then we can fix it later. Any objections?

scsides · 2021-05-24T20:25:38Z

isis/appdata/import/cassis.lbl.tpl

@@ -1,3 +1,8 @@
+{% set PO=Product_Observational %}


We may need to be more specific for this file name. There are likely more cassis templates to come.

scsides · 2021-05-24T20:26:21Z

isis/appdata/import/cassis.lbl.tpl

+  {% if exists(CassHeader) %}
+    {% set windowNumber=int(PO.CaSSIS_Header.FSW_HEADER.attrib_WindowCounter) + 1  %}
+    {% if windowNumber == 1 %}
+        {% set frameletStartSample = int(PO.CaSSIS_Header.PEHK_HEADER.attrib_Window1_Start_Col) + 1 %}
+        {% set frameletEndSample = int(PO.CaSSIS_Header.PEHK_HEADER.attrib_Window1_End_Col) + 1 %}
+        {% set frameletStartLine = int(PO.CaSSIS_Header.PEHK_HEADER.attrib_Window1_Start_Row) + 1 %}
+        {% set frameletEndLine = int(PO.CaSSIS_Header.PEHK_HEADER.attrib_Window1_End_Row) + 1 %}
+    {% endif %}
+    {% if windowNumber == 2 %}
+        {% set frameletStartSample = int(PO.CaSSIS_Header.PEHK_HEADER.attrib_Window2_Start_Col) + 1 %}
+        {% set frameletEndSample = int(PO.CaSSIS_Header.PEHK_HEADER.attrib_Window2_End_Col) + 1 %}
+        {% set frameletStartLine = int(PO.CaSSIS_Header.PEHK_HEADER.attrib_Window2_Start_Row) + 1 %}
+        {% set frameletEndLine = int(PO.CaSSIS_Header.PEHK_HEADER.attrib_Window2_End_Row) + 1 %}
+    {% endif %}
+    {% if windowNumber == 3 %}
+        {% set frameletStartSample = int(PO.CaSSIS_Header.PEHK_HEADER.attrib_Window3_Start_Col) + 1 %}
+        {% set frameletEndSample = int(PO.CaSSIS_Header.PEHK_HEADER.attrib_Window3_End_Col) + 1 %}
+        {% set frameletStartLine = int(PO.CaSSIS_Header.PEHK_HEADER.attrib_Window3_Start_Row) + 1 %}
+        {% set frameletEndLine = int(PO.CaSSIS_Header.PEHK_HEADER.attrib_Window3_End_Row) + 1 %}
+    {% endif %}
+    {% if windowNumber == 4 %}
+        {% set frameletStartSample = int(PO.CaSSIS_Header.PEHK_HEADER.attrib_Window4_Start_Col) + 1 %}
+        {% set frameletEndSample = int(PO.CaSSIS_Header.PEHK_HEADER.attrib_Window4_End_Col) + 1 %}
+        {% set frameletStartLine = int(PO.CaSSIS_Header.PEHK_HEADER.attrib_Window4_Start_Row) + 1 %}
+        {% set frameletEndLine = int(PO.CaSSIS_Header.PEHK_HEADER.attrib_Window4_End_Row) + 1 %}
+    {% endif %}
+    {% if windowNumber == 5 %}
+        {% set frameletStartSample = int(PO.CaSSIS_Header.PEHK_HEADER.attrib_Window5_Start_Col) + 1 %}
+        {% set frameletEndSample = int(PO.CaSSIS_Header.PEHK_HEADER.attrib_Window5_End_Col) + 1 %}
+        {% set frameletStartLine = int(PO.CaSSIS_Header.PEHK_HEADER.attrib_Window5_Start_Row) + 1 %}
+        {% set frameletEndLine = int(PO.CaSSIS_Header.PEHK_HEADER.attrib_Window5_End_Row) + 1 %}
+    {% endif %}
+    {% if windowNumber == 6 %}
+        {% set frameletStartSample = int(PO.CaSSIS_Header.PEHK_HEADER.attrib_Window6_Start_Col) + 1 %}
+        {% set frameletEndSample = int(PO.CaSSIS_Header.PEHK_HEADER.attrib_Window6_End_Col) + 1 %}
+        {% set frameletStartLine = int(PO.CaSSIS_Header.PEHK_HEADER.attrib_Window6_Start_Row) + 1 %}
+        {% set frameletEndLine = int(PO.CaSSIS_Header.PEHK_HEADER.attrib_Window6_End_Row) + 1 %}
+    {% endif %}


Good enough for now

* Initial isisimport application (#4470) * Initial isisimport application modeled to work on importing tgocassis images * Add template to toisis * Draft cassis template file added to pds4 branch (#4472) * Initial isisimport application modeled to work on importing tgocassis images * Add template to toisis * actual cassis label template * Reverted accidently modified template * Add new locations of import and export templates and update cmake accordingly (#4477) * Update cmake to new locations of import and export templates * Fix mixed tabs and spaces * Removed topds4 directory from CMakeLists * Updates XmlToJson parsing to be more inja-friendly (#4478) * Updates XmlToJson parsing to indicate XML attributes and text values in a way friendly to inja parsing of the json * Update XmlToJson tests * Tgocassis Template Completed for Instruments and Archive Groups (#4481) * Updates to cassis label template * Further updates to cassis template * Updates to accomodate PR 4478 changes * Cassis template completed for Instrument and Archive Groups. * Generalize PDS4 cube creation setup (#4485) * Updates XmlToJson parsing to indicate XML attributes and text values in a way friendly to inja parsing of the json * Update XmlToJson tests * Generalized line/sample/band setup * PDS4PixelType as a callback, TEMPLATE default option, and working TGOCaSSIS.tpl (#4479) * Cassis Template Updated BandBin and AlphaCube Groups (#4482) * tgocassis Template Completed for AlphaCube and BandBin groups * Additional template cleanup * Updated Pixels Group * Removed core object from template. * protect template search and improve TGOCASSIS template (#4486) * Renamed topds4 to isisexport (#4488) * Updates to Cassis Template and XML and Minor corrections to isisimport (#4490) * Updates to cassis template. Corrections to isisimport to allow module tests to work properly. * Added disclaimer to xml file. * Added mapping group template file (#4492) * Added checks for existing keywords to be able to export an imported tgo image * Added separate mapping group translation file * Callbacks for YearDOY and ConvertUniqueIdtoObservationId (#4491) * isisimport can now default to correct template using a file template. Updated Cassis template name. Minor updates in isisimport. (#4493) * Updates mapping group in template. Added callback to handle added Z to StartTime. (#4495) * PDS4 - Fix template search and targetGroup template variable for tgocassis.tpl (#4494) * fix template search and targetGroup template variable for tgocassis.tpl * remove accidental extra tag * address feedback from amy * Updated export template to produce parsable xml (#4497) * Updated export template to be functioning xml to be used in isisimport. * Added back 3D_Array but left in comments. * Adds tests with test data and fixes off-by-one error in isisimport (#4502) * Added intial working section of tests * Update test with test data and fix off-by-one in isisimport * Add test input image * Clean up from old design of test * Updated based on feedback * Moved local function into ISIS namespace * Ticked ISIS version to 6 * More tick to 6 * Updated CHANGELOG We don't add the 6.0.0 heading until just before the actual release. I also added some missing periods. Co-authored-by: Amy Stamile <74275278+amystamile-usgs@users.noreply.github.com> Co-authored-by: Tim Giroux <62255438+tgiroux@users.noreply.github.com> Co-authored-by: Stuart Sides <ssides@usgs.gov> Co-authored-by: Kaitlyn Lee <kdl222@nau.edu> Co-authored-by: Jesse Mapel <jmapel@usgs.gov>

* Initial isisimport application (DOI-USGS#4470) * Initial isisimport application modeled to work on importing tgocassis images * Add template to toisis * Draft cassis template file added to pds4 branch (DOI-USGS#4472) * Initial isisimport application modeled to work on importing tgocassis images * Add template to toisis * actual cassis label template * Reverted accidently modified template * Add new locations of import and export templates and update cmake accordingly (DOI-USGS#4477) * Update cmake to new locations of import and export templates * Fix mixed tabs and spaces * Removed topds4 directory from CMakeLists * Updates XmlToJson parsing to be more inja-friendly (DOI-USGS#4478) * Updates XmlToJson parsing to indicate XML attributes and text values in a way friendly to inja parsing of the json * Update XmlToJson tests * Tgocassis Template Completed for Instruments and Archive Groups (DOI-USGS#4481) * Updates to cassis label template * Further updates to cassis template * Updates to accomodate PR 4478 changes * Cassis template completed for Instrument and Archive Groups. * Generalize PDS4 cube creation setup (DOI-USGS#4485) * Updates XmlToJson parsing to indicate XML attributes and text values in a way friendly to inja parsing of the json * Update XmlToJson tests * Generalized line/sample/band setup * PDS4PixelType as a callback, TEMPLATE default option, and working TGOCaSSIS.tpl (DOI-USGS#4479) * Cassis Template Updated BandBin and AlphaCube Groups (DOI-USGS#4482) * tgocassis Template Completed for AlphaCube and BandBin groups * Additional template cleanup * Updated Pixels Group * Removed core object from template. * protect template search and improve TGOCASSIS template (DOI-USGS#4486) * Renamed topds4 to isisexport (DOI-USGS#4488) * Updates to Cassis Template and XML and Minor corrections to isisimport (DOI-USGS#4490) * Updates to cassis template. Corrections to isisimport to allow module tests to work properly. * Added disclaimer to xml file. * Added mapping group template file (DOI-USGS#4492) * Added checks for existing keywords to be able to export an imported tgo image * Added separate mapping group translation file * Callbacks for YearDOY and ConvertUniqueIdtoObservationId (DOI-USGS#4491) * isisimport can now default to correct template using a file template. Updated Cassis template name. Minor updates in isisimport. (DOI-USGS#4493) * Updates mapping group in template. Added callback to handle added Z to StartTime. (DOI-USGS#4495) * PDS4 - Fix template search and targetGroup template variable for tgocassis.tpl (DOI-USGS#4494) * fix template search and targetGroup template variable for tgocassis.tpl * remove accidental extra tag * address feedback from amy * Updated export template to produce parsable xml (DOI-USGS#4497) * Updated export template to be functioning xml to be used in isisimport. * Added back 3D_Array but left in comments. * Adds tests with test data and fixes off-by-one error in isisimport (DOI-USGS#4502) * Added intial working section of tests * Update test with test data and fix off-by-one in isisimport * Add test input image * Clean up from old design of test * Updated based on feedback * Moved local function into ISIS namespace * Ticked ISIS version to 6 * More tick to 6 * Updated CHANGELOG We don't add the 6.0.0 heading until just before the actual release. I also added some missing periods. Co-authored-by: Amy Stamile <74275278+amystamile-usgs@users.noreply.github.com> Co-authored-by: Tim Giroux <62255438+tgiroux@users.noreply.github.com> Co-authored-by: Stuart Sides <ssides@usgs.gov> Co-authored-by: Kaitlyn Lee <kdl222@nau.edu> Co-authored-by: Jesse Mapel <jmapel@usgs.gov>

Amy Stamile added 2 commits May 20, 2021 14:52

tgocassis Template Completed for AlphaCube and BandBin groups

8bfe0ca

Additional template cleanup

8797358

amystamile-usgs commented May 20, 2021

View reviewed changes

kberryUSGS approved these changes May 24, 2021

View reviewed changes

Updated Pixels Group

c5cf258

amystamile-usgs requested a review from kberryUSGS May 24, 2021 16:58

amystamile-usgs commented May 24, 2021

View reviewed changes

Removed core object from template.

7173586

scsides approved these changes May 24, 2021

View reviewed changes

amystamile-usgs merged commit 0fd4d51 into DOI-USGS:pds4 May 24, 2021

amystamile-usgs deleted the cassisTemp branch May 26, 2021 14:43

This was referenced Jul 8, 2021

(Enhanced) Import PDS4 image USGS-Astrogeology/softwaremanagement#71

Closed

PDS4 Image Export Application USGS-Astrogeology/softwaremanagement#69

Closed

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Cassis Template Updated BandBin and AlphaCube Groups #4482

Cassis Template Updated BandBin and AlphaCube Groups #4482

amystamile-usgs commented May 20, 2021 •

edited

amystamile-usgs May 20, 2021

scsides May 24, 2021

kberryUSGS left a comment

kberryUSGS May 24, 2021

scsides May 24, 2021

kberryUSGS May 24, 2021

amystamile-usgs May 24, 2021

kberryUSGS May 24, 2021

kberryUSGS commented May 24, 2021

amystamile-usgs commented May 24, 2021

amystamile-usgs commented May 24, 2021

amystamile-usgs May 24, 2021

scsides May 24, 2021

scsides left a comment

scsides May 24, 2021

scsides May 24, 2021

scsides May 24, 2021

Cassis Template Updated BandBin and AlphaCube Groups #4482

Cassis Template Updated BandBin and AlphaCube Groups #4482

Conversation

amystamile-usgs commented May 20, 2021 • edited

Description

Related Issue

Motivation and Context

How Has This Been Tested?

Screenshots (if appropriate):

Types of changes

Checklist:

Licensing

Choose a reason for hiding this comment

Choose a reason for hiding this comment

kberryUSGS left a comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

kberryUSGS commented May 24, 2021

amystamile-usgs commented May 24, 2021

amystamile-usgs commented May 24, 2021

Choose a reason for hiding this comment

Choose a reason for hiding this comment

scsides left a comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

amystamile-usgs commented May 20, 2021 •

edited