Skip to content

HTTPS clone URL

Subversion checkout URL

You can clone with
or
.
Download ZIP

Loading…

"Same as" in metadata dictionary should just default instead of requiring copies? #70

Closed
rayg-ssec opened this Issue · 7 comments

3 participants

@rayg-ssec
Collaborator

Having the same value in 2-3 places in the metadata/band dictionary seems to be error-prone 27B-6 silliness. How about some nice defaults inherited from upstream scopes?

@davidh-ssec
Owner

It was defensive programming that can probably be cleaned up.

Does this include the "Copy of metadata dict entry" like swath_rows, swath_cols, swaths_scans, rows_per_scan? I'll refer to these as "copies" from now on.

The copies were "needed" because in certain cases only the band dictionary is passed to polar2grid processing, like remapping. The copies could probably be removed and passed as arguments from the top level of the dictionary. Which we could also get rid of "swath_scans" since that can just be calculated from swath_rows/rows_per_scan.

The 'remap_data_as' could probably be used as a default I suppose (defaults to data_kind), but if it wasn't required I'm not sure anyone would ever set it which would defeat the purpose of having it. So I'll probably keep this one or come up with a performance enhancement that makes this unnecessary...wait I did fix this in fornav so this shouldn't be necessary but I think performance was better if you split up the jobs anyway. Hhhhmmm decisions, decisions.

The "kind" and "band" in each band dictionary could probably be passed as arguments to whatever is using them. This was more useful back when only band_kind was the key in the dictionary.

I'm not seeing any other "Same as" keys in the documentation I wrote. Any other suggestions or other feedback on this topic?

@evas-ssec
Collaborator

I do want to note that I may end up in a position with the geocat work where I have to process files with different rows_per_scan values. I don't think anything you're discussing will cause problems with that, but it's a limitation that none of the other data sets have so far.

@davidh-ssec
Owner

@evas-ssec would these files be in different navigation sets (they each have their own unique lat/lon arrays)? If so, then this should be fine. I could see a problem if, for example, the VIIRS M bands had a different number of rows per scan for bands 1-5 vs 6-15 or something like that. But even then I can't imagine that they would have the same navigation arrays.

@evas-ssec
Collaborator

I'm expecting them to be different navigation sets, but the organization of variable selection is kind of different in the geocat files. They're producing similar products using different satellite/instrument inputs and I'm not yet entirely sure how it's going to shake out in the final guidebook. Currently I don't have a lot of sample data.

@davidh-ssec
Owner

Ok, sounds like once we get a better understanding of the CrIS cases and geocat cases we will have to sit down for a developer meeting and hash out a couple things. Maybe when closer to version 1.5.0, probably jump to version 2.0 if it's awesome enough.

Not really the place, but things to think about:

  • Frontend interface, what's logical, what's easy. There's something I just don't like about it.
  • Best way to describe a band. Is "band" the best defining term or do we go more generic
    • Along with this, how are components called, per band, per nav set, other?
    • What is the minimum amount of describing information any one component will be called with. Right now I think it's (sat, inst, nav_set, band_kind, band_id, data_kind)
  • What config files are needed, do we get rid of some, do we add others
  • What needs to be encapsulated into a class
  • Remapping; rewrite fornav, get rid of fornav (scipy or pytroll or gdal)
  • Better/more official names (Cartographer -> other, kind -> band_kind)
@evas-ssec
Collaborator

Since we're now discussing config files (in a massive tangent), I'd really like the option for wild cards or sets in some slots in the files. There are a bunch of duplicated entries for the MODIS stuff that are only there because I have to list "aqua" and "terra" on separate lines. There will probably be times where we need to discriminate by both satellite and instrument, but I'd like the option of saying "this piece of selection criteria isn't important, whatever you find is fine".

@davidh-ssec
Owner

In v2.0 of polar2grid these "metadata dictionaries" have completely changed. They should be a little more sensible (although I'm sure they could be done better).

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Something went wrong with that request. Please try again.