Add writing a cube out to topds4 #4241

kberryUSGS · 2021-01-06T05:18:22Z

Description

Converts cubeatt to a callable and then uses it in topds4 to generate an output cube.

Related Issue

PDS4 work.

Motivation and Context

So that topds4 can output a cube in addition to a label.

How Has This Been Tested?

Screenshots (if appropriate):

Types of changes

Bug fix (non-breaking change which fixes an issue)
New feature (non-breaking change which adds functionality)
Documentation change (update to the documentation; no code change)
Breaking change (fix or feature that would cause existing functionality to change)

Checklist:

I have read and agree to abide by the Code of Conduct
I have read the CONTRIBUTING document.
My change requires a change to the documentation.
I have updated the documentation accordingly.
I have added tests to cover my changes.
All new and existing tests passed.
I have added myself to the .zenodo.json document.
I have added any user impacting changes to the CHANGELOG.md document.

Licensing

This project is mostly composed of free and unencumbered software released into the public domain, and we are unlikely to accept contributions that are not also released into the public domain. Somewhere near the top of each file should have these words:

This work is free and unencumbered software released into the public domain. In jurisdictions that recognize copyright laws, the author or authors of this software dedicate any and all copyright interest in the software to the public domain.

I dedicate any and all copyright interest in this software to the public domain. I make this dedication for the benefit of the public at large and to the detriment of my heirs and successors. I intend this dedication to be an overt act of relinquishment in perpetuity of all present and future rights to this software under copyright law.

kberryUSGS · 2021-01-06T05:22:45Z

isis/src/base/apps/topds4/topds4.cpp

+    // NEED TO WRITE AND CLOSE THE OUTPUT FILE BEFORE RENDERING SO THE FILE SIZE CALLBACK CAN GET THE FINAL FILE SIZE
+
+    // Need a better solution for this.
+    QString outputCubePath = outputFile+".cub";


How to handle one output filepath indicated by "TO" vs. two output files: the label and the cube itself. tgocassisrdrgen uses the one input for both and appends ".xml" or ".img" as needed. Thoughts for this application?

I would have to specify the xml label and then strip that off to get the cube name. Ideally it would check if the extension is .xml, .cub, or neither. Then it will strip and add extensions as needed.

Most if not all of the apps that deal with .img .xml .lbl, ... ask for the label file then change the extension to get at the other files.

kberryUSGS · 2021-01-06T05:23:04Z

isis/tests/FunctionalTestsCubeatt.cpp

+  EXPECT_NEAR(pixelGroup["Base"][0].toDouble(), -0.003952569, 0.00001);
+  EXPECT_NEAR(pixelGroup["Multiplier"][0].toDouble(), 0.003952569, 0.00001);
+
+  // Test the DNs


I need to add this part of the tests still.

I would pull the Cube statistics and check that the min/max values are between the specified values in the cubeatt call. The small cube fixture has values ranging from 0 to 1000, so I would pick something other than 0:1 for your range in this test. Maybe something like 100:460 that will cover about half of the pixels?

isis/tests/FunctionalTestsCubeatt.cpp

jessemapel

Conversion looks good. I added some comments about how to improve the tests and how to check DNs

isis/tests/FunctionalTestsCubeatt.cpp

jessemapel · 2021-01-06T15:20:53Z

isis/tests/FunctionalTestsCubeatt.cpp

+  EXPECT_NEAR(pixelGroup["Base"][0].toDouble(), -0.003952569, 0.00001);
+  EXPECT_NEAR(pixelGroup["Multiplier"][0].toDouble(), 0.003952569, 0.00001);
+
+  // Test the DNs


I would pull the Cube statistics and check that the min/max values are between the specified values in the cubeatt call. The small cube fixture has values ranging from 0 to 1000, so I would pick something other than 0:1 for your range in this test. Maybe something like 100:460 that will cover about half of the pixels?

isis/tests/FunctionalTestsCubeatt.cpp

scsides · 2021-01-06T16:00:22Z

isis/src/base/apps/cubeatt/cubeatt.cpp

+    ProcessByLine p;
+
+    // Should we propagate tables
+    if(propTables) {


This seems backwards.

Good catch! Also, I wrote this weirdly anyway. I've cleaned it up now to just do: p.PropagateTables(propTables);

scsides · 2021-01-06T16:05:32Z

isis/src/base/apps/topds4/topds4.cpp

+    // NEED TO WRITE AND CLOSE THE OUTPUT FILE BEFORE RENDERING SO THE FILE SIZE CALLBACK CAN GET THE FINAL FILE SIZE
+
+    // Need a better solution for this.
+    QString outputCubePath = outputFile+".cub";


Most if not all of the apps that deal with .img .xml .lbl, ... ask for the label file then change the extension to get at the other files.

isis/tests/FunctionalTestsCubeatt.cpp

jessemapel

Looks good, just one issue that's causing other tests to fail.

isis/tests/Fixtures.cpp

jessemapel · 2021-01-07T15:25:06Z

Also, when we do test conversion we are deleting the old test makefiles

* Removed compile and deprication warnings. Fixes #4110 * temp add of to pds 4 test * Initial topds4 app and cmake updates to include inja (#4207) * Removed compile and deprication warnings. Fixes #4110 * temp add of to pds 4 test * Inintal topds4 app and cmake updates to include inja * Delete extra headers * Fixed inja include Co-authored-by: ssides <ssides@usgs.gov> * Added conda-forge inja * add template file * Adds PVL to JSON converters (#4212) * Pvl to JSON conversion * Renamed file * Added repeated elements and examples * Topds4 real parameters (#4223) * now has some real parameters * removed inja files * Add full orex pds4 template (#4233) * now has some real parameters * removed inja files * full orex xml * add full orex template * Added callbacks and tests (#4226) * Adds Xml2Json capability to topds4 branch (#4228) * Inintal topds4 app and cmake updates to include inja * Delete extra headers * Fixed inja include * Added basic working XMLToJson functionality * Updated to work with many nested levels of different amounts of repeated tags * Add support for unsupported xml edge cases * Add tests that work to test mutliple nested repeated values conversion * cleanup * Update to include no-text value cases to work in repeated-value cases * Update test * Break up tests * Fix typoes in comments * Normalized topds4 and added tests (#4234) * Removed data source args and added app log * Added simple tests for topds4 * Moved line * Added more template elements and callback for file size (#4236) * now has some real parameters * removed inja files * full orex xml * add full orex template * More template elements, and file size callback * Added Extra data source parameters to topds4 (#4239) * Added extra parameters * Added duplicate warnings * Fixed warning * Add writing a cube out to topds4 (#4241) * Basic cubeatt conversion * Add more useful function signature * Removed hardcoded path * Update based on feedback * Add better band test * Updated to fix failing tests and remove old testfiles * Add copy of topds4 templates to CMakeLists.txt (#4242) * Fixed callbacks and doc updates (#4243) * Fixed callbacks and doc updates * added some extra docs * Added original xml label and tests * Update cubeattribute to include an option to supply CubeAttributeInput directly (#4244) * Update cubeatt to add cubeattributeinput option. Also add tests for all function signatures available * Remove unnecessary comments * Update to use explicit array conversion. * Remove commented out code * Update CHANGELOG to include topds4 addition * Delete osirisrex.xml.tpl Removed temporary file * Delete 20190612T090019S776_map_L0pan.xml.full Remove temporary file * Delete 20190612T090019S776_map_L0pan.xml.tpl Remove temp file * Added mis-deleted file (#4249) * Update XmlToJson.cpp * Update topds4.xml * Update topds4.xml * Update topds4.xml * Add setting of virtual bands to topds4 and remove unneeded header file includes Co-authored-by: ssides <ssides@usgs.gov> Co-authored-by: Kristin Berry <kberry@usgs.gov>

kberryUSGS added 2 commits January 5, 2021 21:43

Basic cubeatt conversion

0353bf7

Add more useful function signature

5a4359c

kberryUSGS requested review from scsides and jessemapel January 6, 2021 05:18

kberryUSGS commented Jan 6, 2021

View reviewed changes

isis/tests/FunctionalTestsCubeatt.cpp Outdated Show resolved Hide resolved

Removed hardcoded path

51c976e

jessemapel reviewed Jan 6, 2021

View reviewed changes

scsides reviewed Jan 6, 2021

View reviewed changes

Update based on feedback

e2be73e

jessemapel reviewed Jan 6, 2021

View reviewed changes

isis/tests/FunctionalTestsCubeatt.cpp Outdated Show resolved Hide resolved

jessemapel reviewed Jan 6, 2021

View reviewed changes

isis/tests/FunctionalTestsCubeatt.cpp Show resolved Hide resolved

Add better band test

995e8e6

kberryUSGS requested review from scsides and jessemapel January 6, 2021 23:15

Merge conflicts

34b4e92

jessemapel reviewed Jan 7, 2021

View reviewed changes

isis/tests/Fixtures.cpp Outdated Show resolved Hide resolved

Updated to fix failing tests and remove old testfiles

35121d5

kberryUSGS requested a review from jessemapel January 7, 2021 16:50

jessemapel approved these changes Jan 7, 2021

View reviewed changes

kberryUSGS merged commit 5a6586c into DOI-USGS:topds4 Jan 7, 2021

scsides mentioned this pull request Jul 8, 2021

PDS4 Image Export Application USGS-Astrogeology/softwaremanagement#69

Closed

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Add writing a cube out to topds4 #4241

Add writing a cube out to topds4 #4241

kberryUSGS commented Jan 6, 2021 •

edited

Loading

kberryUSGS Jan 6, 2021

jessemapel Jan 6, 2021

scsides Jan 6, 2021

kberryUSGS Jan 6, 2021

jessemapel Jan 6, 2021

jessemapel left a comment

jessemapel Jan 6, 2021

scsides Jan 6, 2021

kberryUSGS Jan 6, 2021 •

edited

Loading

scsides Jan 6, 2021

jessemapel left a comment

jessemapel commented Jan 7, 2021

Add writing a cube out to topds4 #4241

Add writing a cube out to topds4 #4241

Conversation

kberryUSGS commented Jan 6, 2021 • edited Loading

Description

Related Issue

Motivation and Context

How Has This Been Tested?

Screenshots (if appropriate):

Types of changes

Checklist:

Licensing

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

jessemapel left a comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

kberryUSGS Jan 6, 2021 • edited Loading

Choose a reason for hiding this comment

Choose a reason for hiding this comment

jessemapel left a comment

Choose a reason for hiding this comment

jessemapel commented Jan 7, 2021

kberryUSGS commented Jan 6, 2021 •

edited

Loading

kberryUSGS Jan 6, 2021 •

edited

Loading