P2G fails to produce VIIRS AWIPS AOD EDR files

[kathys]

Just tried to produce AWIPS tiles from the VIIRS AOD EDR product files for the first time and the software failed.

polar2grid.sh -r viirs_edr -w awips_tiled -p AOD550 -g lcc_conus_300 --sector-id LCC --letters --compress -f ../edr/JRR-AOD*.nc
........
ValueError: The truth value of an array with more than one element is ambiguous. Use a.any() or a.all()

I was able to create a really nice GeoTIFF from this input data set.

Log files is here: bumi:/data/users/kathys/test_data/viirs_edr/j01/awips

[djhoese]

Oh! Sorry for taking so long to address this. I thought this was a lower-level issue related to reading the file, but completely missed the "AWIPS" specific part of this issue. While this error is unfortunate I think it stems from the AOD product not being configured in the AWIPS tiled writer OR it could also be related to missing or misconfigured metadata coming from the files. Based on the error happening in the "valid_range" portion of the writer (based on the traceback of the error) I'm guess valid_range from the file is being loaded as an array when it should be something like a tuple/list (a native python type). Let me see if I can reproduce this locally and fix it.

[djhoese]

We may need to talk to GINA (CC @spruceboy, does Carl have a github account?) about if/what they might want in the Satpy AWIPS writer configuration file for the AOD (or other) products. Or since they already customize a lot of their stuff maybe we just leave it unconfigured. Regardless I'll try to fix this error being produced.

[spruceboy]

@cfdierking is Carl's github account, but he doesn't really do github much. Let me point him to this and see if he will chim in.

[djhoese]

Related Satpy PR: pytroll/satpy#2833

Basically the reader was loading the NetCDF variable attribute valid_range as a numpy array (NetCDF library's fault) and the AWIPS writer assumes it can do something like if valid_range:. This type of boolean check on a numpy array is not allowed so you get the except you're seeing. My PR for Satpy updates the reader to store the valid_range as a tuple which does allow this boolean check.

[cfdierking]

Dave... I've read the comments so far. Is there anything I should weigh in on?

[djhoese]

Honestly, not sure. The biggest thing is that I could configure, in Satpy, an entry for VIIRS AOD in the AWIPS tiled writer. That would make it so GINA (theoretically) wouldn't have to modify your custom AWIPS config to use that product as we talked about in our email thread. But I think it is almost less work for you guys to do it in your own config the more I think about it.

[cfdierking]

So would we still be using p2g 3.1 (beta) but with our own AWIPS configs?

[djhoese]

Yes.

[djhoese]

Or rather, whatever you're doing now.

[djhoese]

This should be fixed because this Satpy PR was merged:

pytroll/satpy#2833

After @kathys verifies we can close this.

[spruceboy]

Thanks @djhoese !!

[djhoese]

Kathy has had this working for a while. We just forgot to come back and comment. This should be working. I don't know off the top of my head what specific beta version includes this fix, but the one that was just sent to Carl includes it for sure.