Stops working silently

[fishcharlie]

For some reason every once in a while the system will just stop logging anything, and stop working. My guess is that it's not making requests anymore. Maybe an await statement is getting stuck or something??

We need to investigate this and fix the bug.

[stucka]

Dunno if this might help narrow things down, or whether you have any visibility into CPU load, memory usage, etc. to see what's happening ... https://developer.ibm.com/articles/6-reasons-your-node-js-apps-are-failing/

[stucka]

For giggles, likely would not hurt to bump the node in docker-compose.yml to 20.11.1 from 20.10.0.

[fishcharlie]

@stucka

Dunno if this might help narrow things down, or whether you have any visibility into CPU load, memory usage, etc. to see what's happening ... https://developer.ibm.com/articles/6-reasons-your-node-js-apps-are-failing/

Yeah. Nothing standing out. I'm pretty sure it's a promise that isn't resolving. But not 100% sure yet. It's pretty rare that it happens. Just adding more logging to see if I can see where it's failing.

For giggles, likely would not hurt to bump the node in docker-compose.yml to 20.11.1 from 20.10.0.

Done in cc6e6ad

[fishcharlie]

app-1  | Start run: 1715455763789
app-1  | Got XML <AIRPORT_STATUS_INFORMATION><Update_Time>Sat May 11 19:28:42 2024 GMT</Update_Time><Dtd_File>http://www.fly.faa.gov/AirportStatus.dtd</Dtd_File><Delay_type><Name>Ground Delay Programs</Name><Ground_Delay_List><Ground_Delay><ARPT>YUL</ARPT><Reason>runway construction</Reason><Avg>14 minutes</Avg><Max>48 minutes</Max></Ground_Delay></Ground_Delay_List></Delay_type><Delay_type><Name>Airport Closures</Name><Airport_Closure_List><Airport><ARPT>ASE</ARPT><Reason>!ASE 05/032 ASE AD AP CLSD EXC SAR AND LIFE FLT AND HEL 2405061500-2405240100</Reason><Start>May 06 at 15:00 UTC.</Start><Reopen>May 24 at 01:00 UTC.</Reopen></Airport><Airport><ARPT>LGA</ARPT><Reason>!LGA 04/094 LGA AD AP CLSD EXC 4HR &#xd;PPR 718-533-3700 DLY 0401-1000 2404150401-2411021000</Reason><Start>Apr 15 at 04:01 UTC.</Start><Reopen>Nov 02 at 10:00 UTC.</Reopen></Airport></Airport_Closure_List></Delay_type><Delay_type><Name>Airport Closures</Name><Airport_Closure_List><Airport><ARPT>SBA</ARPT><Reason>!SBA 04/124 SBA AD AP CLSD TO FIXED WING EXC 30 MIN PPR 805-681-4803 MON-FRI 0700-1200 2404250700-2407241200</Reason><Start>Apr 25 at 07:00 UTC.</Start><Reopen>Jul 24 at 12:00 UTC.</Reopen></Airport><Airport><ARPT>BOS</ARPT><Reason>!BOS 04/191 BOS AD AP CLSD TO NON SKED TRANSIENT GA ACFT PPR 617-561-2500 2404110900-2406152359</Reason><Start>Apr 11 at 09:00 UTC.</Start><Reopen>Jun 15 at 23:59 UTC.</Reopen></Airport><Airport><ARPT>LAS</ARPT><Reason>!LAS 03/098 LAS AD AP CLSD TO NON SKED TRANSIENT GA ACFT EXC PPR 702-261-7775 2403132300-2405152300</Reason><Start>Mar 13 at 23:00 UTC.</Start><Reopen>May 15 at 23:00 UTC.</Reopen></Airport></Airport_Closure_List></Delay_type></AIRPORT_STATUS_INFORMATION>
app-1  | Failed to find airport with code YUL
app-1  | Failed to find airport with code YUL
app-1  |
app-1  |
app-1  | All delays:
app-1  | Failed to find airport with code YUL
app-1  | [
app-1  |   'An airport closure has been issued for Aspen-Pitkin County Airport (Sardy Field) (#ASE). The airport is expected to reopen May 5 at 7:00 PM.',
app-1  |   'An airport closure has been issued for La Guardia Airport (#LGA). The airport is expected to reopen November 11 at 6:00 AM.',
app-1  |   'An airport closure has been issued for Santa Barbara Municipal Airport (#SBA). The airport is expected to reopen July 7 at 5:00 AM.',
app-1  |   'An airport closure has been issued for Logan International Airport (#BOS). The airport is expected to reopen June 6 at 7:59 PM.',
app-1  |   'An airport closure has been issued for Harry Reid International Airport (#LAS). The airport is expected to reopen May 5 at 4:00 PM.'
app-1  | ]
app-1  | New delays:
app-1  | []
app-1  | Removed delays:
app-1  | []
app-1  | Updated delays:
app-1  | []
app-1  | Run Parse: 80.878ms
app-1  | Posting 0 new delays.
app-1  | Posting 0 removed delays.
app-1  | Done posting social messages.
app-1  | Done running: 1715455763986
app-1  | Start run: 1715455823994
app-1  | Got XML <AIRPORT_STATUS_INFORMATION><Update_Time>Sat May 11 19:30:16 2024 GMT</Update_Time><Dtd_File>http://www.fly.faa.gov/AirportStatus.dtd</Dtd_File><Delay_type><Name>Ground Delay Programs</Name><Ground_Delay_List><Ground_Delay><ARPT>YUL</ARPT><Reason>runway construction</Reason><Avg>28 minutes</Avg><Max>1 hour and 19 minutes</Max></Ground_Delay></Ground_Delay_List></Delay_type><Delay_type><Name>Airport Closures</Name><Airport_Closure_List><Airport><ARPT>ASE</ARPT><Reason>!ASE 05/032 ASE AD AP CLSD EXC SAR AND LIFE FLT AND HEL 2405061500-2405240100</Reason><Start>May 06 at 15:00 UTC.</Start><Reopen>May 24 at 01:00 UTC.</Reopen></Airport><Airport><ARPT>LGA</ARPT><Reason>!LGA 04/094 LGA AD AP CLSD EXC 4HR &#xd;PPR 718-533-3700 DLY 0401-1000 2404150401-2411021000</Reason><Start>Apr 15 at 04:01 UTC.</Start><Reopen>Nov 02 at 10:00 UTC.</Reopen></Airport></Airport_Closure_List></Delay_type><Delay_type><Name>Airport Closures</Name><Airport_Closure_List><Airport><ARPT>SBA</ARPT><Reason>!SBA 04/124 SBA AD AP CLSD TO FIXED WING EXC 30 MIN PPR 805-681-4803 MON-FRI 0700-1200 2404250700-2407241200</Reason><Start>Apr 25 at 07:00 UTC.</Start><Reopen>Jul 24 at 12:00 UTC.</Reopen></Airport><Airport><ARPT>BOS</ARPT><Reason>!BOS 04/191 BOS AD AP CLSD TO NON SKED TRANSIENT GA ACFT PPR 617-561-2500 2404110900-2406152359</Reason><Start>Apr 11 at 09:00 UTC.</Start><Reopen>Jun 15 at 23:59 UTC.</Reopen></Airport><Airport><ARPT>LAS</ARPT><Reason>!LAS 03/098 LAS AD AP CLSD TO NON SKED TRANSIENT GA ACFT EXC PPR 702-261-7775 2403132300-2405152300</Reason><Start>Mar 13 at 23:00 UTC.</Start><Reopen>May 15 at 23:00 UTC.</Reopen></Airport></Airport_Closure_List></Delay_type></AIRPORT_STATUS_INFORMATION>
app-1  | Failed to find airport with code YUL
app-1  | Failed to find airport with code YUL
app-1  |
app-1  |
app-1  | All delays:
app-1  | Failed to find airport with code YUL
app-1  | [
app-1  |   'An airport closure has been issued for Aspen-Pitkin County Airport (Sardy Field) (#ASE). The airport is expected to reopen May 5 at 7:00 PM.',
app-1  |   'An airport closure has been issued for La Guardia Airport (#LGA). The airport is expected to reopen November 11 at 6:00 AM.',
app-1  |   'An airport closure has been issued for Santa Barbara Municipal Airport (#SBA). The airport is expected to reopen July 7 at 5:00 AM.',
app-1  |   'An airport closure has been issued for Logan International Airport (#BOS). The airport is expected to reopen June 6 at 7:59 PM.',
app-1  |   'An airport closure has been issued for Harry Reid International Airport (#LAS). The airport is expected to reopen May 5 at 4:00 PM.'
app-1  | ]
app-1  | New delays:
app-1  | []
app-1  | Removed delays:
app-1  | []
app-1  | Updated delays:
app-1  | []
app-1  | Run Parse: 84.633ms
app-1  | Posting 0 new delays.
app-1  | Posting 0 removed delays.
app-1  | Done posting social messages.
app-1  | Done running: 1715455824205
app-1  | Exiting gracefully...
app-1  | Done. Exiting.