Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Added Extra data source parameters to topds4 #4239

Merged
merged 3 commits into from
Jan 6, 2021

Conversation

jessemapel
Copy link
Contributor

Added extra pvl, xml, and json data source parameters. They can be a single file or multiple files. In the case of multiple files, duplicate keys/elements in later files overwrite previous files. If this happens, a user warning is logged like this:

Group = Warning
  Duplicate = "Duplicate key [TestValue] in extra Pvl file
               [/var/folders/s2/r22kzkt91ms_t4j5b90g5k1w002dkh/T/qt_temp-bLxZd-
               t/extra2.pvl]. Previous value [{"Value":"b"}] will be
               overwritten."
End_Group
End

I also re-arranged the included in topds4.cpp and added the xml to json header because I needed it for the extra xml data.

}
dataSource["ExtraJson"].update(extraJson);
}
}
Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

There is a lot of duplicate code between these three sections, but things are subtly different enough that I couldn't just pull it out into a function.


<group name="Data">
Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I moved the DATA parameter into this group because it made sense to me, but I'm not 100% on it.

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

totally reasonable

QString xmlFile = tempDir.path()+"/extra.xml";
std::ofstream xmlStream;
xmlStream.open(xmlFile.toStdString());
xmlStream << "<TestValue>a</TestValue>";
Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Writing raw XML here because it's just easier than using the QT stuff.

Copy link
Contributor

@scsides scsides left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The new tests are painful to look at, but great


<group name="Data">
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

totally reasonable

@jessemapel
Copy link
Contributor Author

So I botched the warning and put the new value in the warning instead of the old value, here's the updated warning from the top comment:

Group = Warning
  Duplicate = "Duplicate key [TestValue] in extra Pvl file
               [/var/folders/s2/r22kzkt91ms_t4j5b90g5k1w002dkh/T/qt_temp-KxImP-
               j/extra2.pvl]. Previous value [{"Value":"a"}] will be
               overwritten."
End_Group
End

Copy link
Contributor

@kberryUSGS kberryUSGS left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

👍 Looks good!

@jessemapel jessemapel merged commit d5d4e73 into DOI-USGS:topds4 Jan 6, 2021
scsides added a commit that referenced this pull request Jan 12, 2021
* Removed compile and deprication warnings. Fixes #4110

* temp add of to pds 4 test

* Initial topds4 app and cmake updates to include inja (#4207)

* Removed compile and deprication warnings. Fixes #4110

* temp add of to pds 4 test

* Inintal topds4 app and cmake updates to include inja

* Delete extra headers

* Fixed inja include

Co-authored-by: ssides <ssides@usgs.gov>

* Added conda-forge inja

* add template file

* Adds PVL to JSON converters (#4212)

* Pvl to JSON conversion

* Renamed file

* Added repeated elements and examples

* Topds4 real parameters (#4223)

* now has some real parameters

* removed inja files

* Add full orex pds4 template (#4233)

* now has some real parameters

* removed inja files

* full orex xml

* add full orex template

* Added callbacks and tests (#4226)

* Adds Xml2Json capability to topds4 branch (#4228)

* Inintal topds4 app and cmake updates to include inja

* Delete extra headers

* Fixed inja include

* Added basic working XMLToJson functionality

* Updated to work with many nested levels of different amounts of repeated tags

* Add support for unsupported xml edge cases

* Add tests that work to test mutliple nested repeated values conversion

* cleanup

* Update to include no-text value cases to work in repeated-value cases

* Update test

* Break up tests

* Fix typoes in comments

* Normalized topds4 and added tests (#4234)

* Removed data source args and added app log

* Added simple tests for topds4

* Moved line

* Added more template elements and callback for file size (#4236)

* now has some real parameters

* removed inja files

* full orex xml

* add full orex template

* More template elements, and file size callback

* Added Extra data source parameters to topds4 (#4239)

* Added extra parameters

* Added duplicate warnings

* Fixed warning

* Add writing a cube out to topds4 (#4241)

* Basic cubeatt conversion

* Add more useful function signature

* Removed hardcoded path

* Update based on feedback

* Add better band test

* Updated to fix failing tests and remove old testfiles

* Add copy of topds4 templates to CMakeLists.txt (#4242)

* Fixed callbacks and doc updates (#4243)

* Fixed callbacks and doc updates

* added some extra docs

* Added original xml label and tests

* Update cubeattribute to include an option to supply CubeAttributeInput directly (#4244)

* Update cubeatt to add cubeattributeinput option. Also add tests for all function signatures available

* Remove unnecessary comments

* Update to use explicit array conversion.

* Remove commented out code

* Update CHANGELOG to include topds4 addition

* Delete osirisrex.xml.tpl

Removed temporary file

* Delete 20190612T090019S776_map_L0pan.xml.full

Remove temporary file

* Delete 20190612T090019S776_map_L0pan.xml.tpl

Remove temp file

* Added mis-deleted file (#4249)

* Update XmlToJson.cpp

* Update topds4.xml

* Update topds4.xml

* Update topds4.xml

* Add setting of virtual bands to topds4 and remove unneeded header file includes

Co-authored-by: ssides <ssides@usgs.gov>
Co-authored-by: Kristin Berry <kberry@usgs.gov>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

3 participants