Allmon3 Daemon Error Overnight

[tsawyer]

Service was continuously printing errors. Apparent cause is node 27437 and its multimodes went down and is still down. This one node going down caused all nodes to not appear on the site, rather than indication that 27437 and its multinodes were not responding.

Additionally, service would not restart I assume due to the continuous printing of errors. Had to stop and then start service.

Web site now correctly shows 27437 and its multinodes 'Websocket is not available or went away'.

Jun 22 07:42:06 Allmon allmon3[148463]: INFO: (asl_allmon.node_db) Updated node database in 0.56 seconds
Jun 22 07:57:06 Allmon allmon3[148463]: INFO: (asl_allmon.node_db) Retrieving database from http://allmondb.allstarlink.org/
Jun 22 07:57:06 Allmon allmon3[148463]: INFO: (asl_allmon.node_db) Updated node database in 0.60 seconds
Jun 22 08:08:24 Allmon allmon3[148463]: ERROR: (asl_allmon.ami_conn) asl_cmd_response() socket.timeout
Jun 22 08:08:24 Allmon allmon3[148463]: ERROR: (asl_allmon.ami_parser) asl_cmd() exception <class 'socket.timeout'>
Jun 22 08:08:24 Allmon allmon3[148463]: ERROR: (asl_allmon.ami_parser) asl_cmd() message <class 'socket.timeout'>
Jun 22 08:08:24 Allmon allmon3[148463]: ERROR: (asl_allmon.allmon3_ws_voter) received socket.timeout; trying to reconnect
Jun 22 08:08:29 Allmon allmon3[148463]: ERROR: (asl_allmon.ami_conn) asl_cmd_response() socket.timeout
Jun 22 08:08:29 Allmon allmon3[148463]: ERROR: (asl_allmon.ami_parser) asl_cmd() exception <class 'socket.timeout'>
Jun 22 08:08:29 Allmon allmon3[148463]: ERROR: (asl_allmon.ami_parser) asl_cmd() message <class 'socket.timeout'>
Jun 22 08:08:29 Allmon allmon3[148463]: ERROR: (asl_allmon.allmon3_ws_status) received socket.timeout; trying to reconnect
Jun 22 08:08:50 Allmon allmon3[148463]: ERROR: (asl_allmon.ami_conn) connection failed to 27437.nodes.allstarlink.org:5038: timed out
Jun 22 08:08:50 Allmon allmon3[148463]: INFO: (asl_allmon.allmon3_ws_voter) connection reestablished after 1 retries
Jun 22 08:08:50 Allmon allmon3[148463]: Future exception was never retrieved
Jun 22 08:08:50 Allmon allmon3[148463]: future: <Future finished exception=ConnectionClosedError(None, None, None)>
Jun 22 08:08:50 Allmon allmon3[148463]: websockets.exceptions.ConnectionClosedError: no close frame received or sent
Jun 22 08:08:50 Allmon allmon3[148463]: Future exception was never retrieved
Jun 22 08:08:50 Allmon allmon3[148463]: future: <Future finished exception=ConnectionClosedError(None, None, None)>
Jun 22 08:08:50 Allmon allmon3[148463]: websockets.exceptions.ConnectionClosedError: no close frame received or sent
Jun 22 08:08:50 Allmon allmon3[148463]: ERROR: (asl_allmon.ami_conn) asl_cmd_response() OSError: [Errno 9] Bad file descriptor
Jun 22 08:08:50 Allmon allmon3[148463]: ERROR: (asl_allmon.ami_parser) asl_cmd() exception <class 'OSError'>
Jun 22 08:08:50 Allmon allmon3[148463]: ERROR: (asl_allmon.ami_parser) asl_cmd() message <class 'OSError'>
Jun 22 08:08:50 Allmon allmon3[148463]: ERROR: (asl_allmon.allmon3_ws_voter) received OSError: [Errno 9] Bad file descriptor; trying to>
Jun 22 08:09:10 Allmon allmon3[148463]: ERROR: (asl_allmon.ami_conn) connection failed to 27437.nodes.allstarlink.org:5038: timed out
Jun 22 08:09:10 Allmon allmon3[148463]: INFO: (asl_allmon.allmon3_ws_status) connection reestablished after 1 retries
Jun 22 08:09:10 Allmon allmon3[148463]: ERROR: (asl_allmon.ami_conn) asl_cmd_response() OSError: [Errno 9] Bad file descriptor
Jun 22 08:09:10 Allmon allmon3[148463]: ERROR: (asl_allmon.ami_parser) asl_cmd() exception <class 'OSError'>
Jun 22 08:09:10 Allmon allmon3[148463]: ERROR: (asl_allmon.ami_parser) asl_cmd() message <class 'OSError'>
Jun 22 08:09:10 Allmon allmon3[148463]: ERROR: (asl_allmon.allmon3_ws_status) received OSError: [Errno 9] Bad file descriptor; trying t>
Jun 22 08:09:10 Allmon allmon3[148463]: ERROR: (asl_allmon.ami_conn) [Errno 9] Bad file descriptor
Jun 22 08:09:30 Allmon allmon3[148463]: ERROR: (as

[jxmx]

Should be fixed by pull #150