Network watchdog

[mansr]

To do

• Write helper script to reset modem
• Write watchdog script to check VRM status
• Add to the gui

Once released

• Add to manual
• Add to the 'how to build a reliable system' blog-post from years ago.

Summary of the functionality as implemented in the watchdog recipes:

• It creates a new setting: com.victronenergy.settings /Settings/Watchdog/VrmTimeout.
• Defaults to 0; disabled. So; the functional minimum is a 1 second time-out; Maximum is 86400 = 24 hours. A one second time-out is insane btw; minimum user-configurable logger interval is 60 seconds.
• The hardware watchdog counter will be reset (preventing a reboot); as long as
  a. vrmlogger has been disabled, or
  b. VrmLastContact + log-interval + timeout >= now
  c. there has been no contact yet; and boot-time + log-interval + time-out >= now()

Code is here.

Proposal for the gui

• Add a setting, to the vrm-logger menu (?)
• An enable/disable switch, and when enabled, show an item that allows configuring the time-out.
• Depending on how complicated it all is; consider adding a text that explains how the feature works.

Summary of the full requirement.

If internet connectivity is lost, a local malfunction may be the cause. It is thus desirable to detect such a condition and take appropriate action which may include rebooting the system. When connectivity is provided by the GX GSM or similar, this shall also be restarted.

vrmlogger TimeOfLastContact will be used to detect network loss

Using VRM TimeLastContact value avoids creating additional network traffic for the connectivity check. The logging interval setting must be taken into account. The check will be performed periodically by the watchdog daemon.

Making sure (usb connected-) modems also reboot

The Victron USB modem can be rebooted by stopping updates of its watchdog and waiting for it to reset. An onboard modem can be simply power cycled. A subsequent full system reboot ensures that a hung pppd or similar does not block communication.

Timeout

To allow for an unreliable external network, a relatively long timeout shall be used. 60 minutes is likely a suitable choice. This timeout is in addition to the VRM logging interval, so the final amount can be significantly longer depending on VRM settings.

[mpvader]

Some additional points:

• it must be possible to disable the network watchdog. (I'm not sure if by default it should be enabled, or disabled).
• A related customer request is to cycle a relay; to power cycle comms equipment: Add relay function: powercycle comms device based on ping #222. Could be done, implementation probably not so difficult; but yet more options can also be confusing for the user. People that want reliable comms for Venus only can get it by using the GX GSM. People that want reliable comms for more than just the Venus can get themselves a professional cellular router that includes a reboot function aka network watchdog itself.. So we decided not to do it.

[mansr]

https://github.com/victronenergy/meta-victronenergy/tree/mans/watchdog

[mpvader]

Thanks.

I do miss an on/off setting. Perhaps better, move the time-out to dbus and when 0 => disabled.

[mpvader]

On off setting has been implemented. Waiting for inclusion in Venus.

[mpvader]

The functionality has been added to Venus v2.30~8, with this commit. Now needs adding to the gui; and testing.

[jhofstee]

comments?

[mpvader]

Its in the ccgx manual