Consider a more obvious stale data warning

[astrobokonon]

I only noticed issue #1 because I wanted to see the longer term radar data for the current storm, and I realized that the plot was showing clear skies when I could literally hear the rain pounding on my house/roof.

There should therefore probably be a much more obvious "HEY THERE'S NO DATA TO PLOT" type of indication, or individual status indicators for each major subsystem. TBD which is easier and which is actually better.

[astrobokonon]

Happened again today, this time due to a lag in the AWS data; all updates (on the AWS side, not me) stopped around 16:00 UTC, but I didn't notice until around 19:00 UTC.

[astrobokonon]

The sticky part of this is that the code is completely fine, it's just the datasource goes slightly stale and no new files are found. As I sit here it seems like they're trickling in to AWS again, though slowly.

Looking around, places like the CoD site still have data flowing; I think they're using the Unidata source(s) but from a cursory look, those are much more hands-on to get up and running. Specifically I was able to download and compile their LDM tool, but I didn't bother to try to configure it. That really is screaming for containerization since it's so specific in its setup.

And from a cursory google search - here's the official container :)
https://github.com/Unidata/ldm-docker

and

https://www.unidata.ucar.edu/data/goesr.html

Ultimately I don't want to open unless we really have to consider it. I'm content to just slap a warning on it and let the users fall back to their favorite alternate source, since the purpose of NightWatch wasn't to build a weather data site.

[astrobokonon]

😃

[astrobokonon]

And we're live! Now testing. I'll watch to see if it un-triggers when the data starts flowing on the AWS side again.

[astrobokonon]

It works!

At (or around) 19:49 UTC I picked up an image from 18:44 UTC, and it didn't have the errorStamp on it. Closing this as it seems to be working fine now.

[astrobokonon]

Final note:

After literally almost a day of searching through various NOAA websites, I think I found something important:

https://www.ospo.noaa.gov/Operations/messages.html

This must be the place that is uploading the AWS data, because they list the following:

Topic: Planned ESPDS PDA Maintenance at the NSOF OPS primary site may cause brief connectivity and/or processing Interruptions.

Date/Time Issued: May 30, 2019 2024Z

Product(s) or Data Impacted: All or some products being Ingested and Disseminated from the NSOF OPS primary site may be affected.

Date/Time of Initial Impact: May 29, 2019 1330Z

Date/Time of Expected End: May 30, 2019 2030Z

Length of Event: 31 hours

Details/Specifics of Change: The PDA OPS primary environment at the NSOF in Suitland, MD is undergoing planned maintenance to upgrade the ESPDS application software. During this upgrade there may be interruptions to connectivity and/or processing for connections to PDA of up to 15 minutes at the NSOF OPS site.

Contact Information for Further Information: ESPC Operations at ESPCOperations@noaa.gov and 301-817-3880.

Web Site(s) for applicable information: N/A

This message was sent by ESPC.Notification@noaa.gov. You have been sent this and other notifications because you have opted in to receive it. If for any reason, you wish to unsubscribe, please contact ESPC Help Desk at ESPCOperations@noaa.gov or (301) 817-3880. Please note: it may take up to two business days to process your unsubscribe request.

and they gave the following updates:

Update # 1: Reminder M-bM-^@M-^S The ESPDS planned maintenance at the NSOF primary site is a two day event and must continue tomorrow, May 30th, even during a Critical Weather declaration. The upgrade activities for tomorrow, Thursday May 30th will begin at 1200Z and continue until the upgrade is complete and verified, about 2100Z. During this upgrade there may be interruptions to processing and/or brief connectivity issues to PDA.

Update # 2: A PDA anomaly is affecting the processing and delivery of GOES-16 and GOES-17 data since approximately 1253Z this morning. This is affecting all GOES-16 and GOES-17 products delivered from PDA. The PDA Support team has identified the issue and is working on a corrective action. Updates will be provided every 2 hours or immediately when the issue is resolved.

Update # 3: The PDA anomaly affecting processing and delivery of GOES-16 and GOES-17 data has been resolved as of 1835Z. The length of the event was 5 hours and 42 minutes. The ESPDS Release 3.3 upgrade is continuing and expected to end at the scheduled 2100Z. A completion notification will be sent after the post upgrade verification activities are complete.

that almost exactly corresponds to the gaps I just saw, so I'll check back here periodically when there are problems.

OPS: 😧
PDA: Product Distribution and Access
OSPO: Office of Satellite and Product Operations
NSOF: NOAA Satellite Operations Facility (located in Suitland, MD)
ESPC: Environmental Satellite Processing Center
ESPDS: NOAA/NESDIS/OSGS Environmental Satellite Processing and Distribution System