Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Network watchdog #287

Closed
4 of 5 tasks
mansr opened this issue Jun 22, 2018 · 7 comments
Closed
4 of 5 tasks

Network watchdog #287

mansr opened this issue Jun 22, 2018 · 7 comments
Assignees

Comments

@mansr
Copy link
Collaborator

mansr commented Jun 22, 2018

To do

  • Write helper script to reset modem
  • Write watchdog script to check VRM status
  • Add to the gui

Once released

  • Add to manual
  • Add to the 'how to build a reliable system' blog-post from years ago.

Summary of the functionality as implemented in the watchdog recipes:

  • It creates a new setting: com.victronenergy.settings /Settings/Watchdog/VrmTimeout.
  • Defaults to 0; disabled. So; the functional minimum is a 1 second time-out; Maximum is 86400 = 24 hours. A one second time-out is insane btw; minimum user-configurable logger interval is 60 seconds.
  • The hardware watchdog counter will be reset (preventing a reboot); as long as
    a. vrmlogger has been disabled, or
    b. VrmLastContact + log-interval + timeout >= now
    c. there has been no contact yet; and boot-time + log-interval + time-out >= now()

Code is here.

Proposal for the gui

  • Add a setting, to the vrm-logger menu (?)
  • An enable/disable switch, and when enabled, show an item that allows configuring the time-out.
  • Depending on how complicated it all is; consider adding a text that explains how the feature works.

Summary of the full requirement.

If internet connectivity is lost, a local malfunction may be the cause. It is thus desirable to detect such a condition and take appropriate action which may include rebooting the system. When connectivity is provided by the GX GSM or similar, this shall also be restarted.

vrmlogger TimeOfLastContact will be used to detect network loss

Using VRM TimeLastContact value avoids creating additional network traffic for the connectivity check. The logging interval setting must be taken into account. The check will be performed periodically by the watchdog daemon.

Making sure (usb connected-) modems also reboot

The Victron USB modem can be rebooted by stopping updates of its watchdog and waiting for it to reset. An onboard modem can be simply power cycled. A subsequent full system reboot ensures that a hung pppd or similar does not block communication.

Timeout

To allow for an unreliable external network, a relatively long timeout shall be used. 60 minutes is likely a suitable choice. This timeout is in addition to the VRM logging interval, so the final amount can be significantly longer depending on VRM settings.

@mpvader
Copy link
Contributor

mpvader commented Jun 24, 2018

Some additional points:

  • it must be possible to disable the network watchdog. (I'm not sure if by default it should be enabled, or disabled).
  • A related customer request is to cycle a relay; to power cycle comms equipment: Add relay function: powercycle comms device based on ping #222. Could be done, implementation probably not so difficult; but yet more options can also be confusing for the user. People that want reliable comms for Venus only can get it by using the GX GSM. People that want reliable comms for more than just the Venus can get themselves a professional cellular router that includes a reboot function aka network watchdog itself.. So we decided not to do it.

@mansr
Copy link
Collaborator Author

mansr commented Jul 20, 2018

@mpvader
Copy link
Contributor

mpvader commented Jul 21, 2018

Thanks.

I do miss an on/off setting. Perhaps better, move the time-out to dbus and when 0 => disabled.

@mpvader
Copy link
Contributor

mpvader commented Oct 31, 2018

On off setting has been implemented. Waiting for inclusion in Venus.

@mpvader
Copy link
Contributor

mpvader commented Jan 7, 2019

The functionality has been added to Venus v2.30~8, with this commit. Now needs adding to the gui; and testing.

@jhofstee
Copy link
Contributor

image

comments?

@jhofstee jhofstee assigned mpvader and unassigned jhofstee Mar 13, 2019
@mpvader
Copy link
Contributor

mpvader commented Nov 28, 2019

Its in the ccgx manual

@mpvader mpvader closed this as completed Nov 28, 2019
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

No branches or pull requests

4 participants