Steps to reproduce
- Boot a machine with a freshly flashed image, or reset an existing one with
firstboot && reboot
- Wait for the boot to complete
- Inspect the boot messages:
tail -f /var/log/messages | grep -E "nginx|netifyd|vmalert|telegraf|victoria"
Expected behavior
Every service starts once, after the services it needs are already listening. The boot
log contains no connection errors between local services and no duplicate service
starts.
Actual behavior
Several services start before the services they talk to, so they either get restarted
later in the boot sequence or keep retrying until their dependency is up. Both cases
fill the log with avoidable messages at every boot:
nginx starts before ns-ui and ns-reverse-proxy have generated its location
files, so it starts with an incomplete configuration and ns-ui restarts it right
after — nginx is started twice on every boot
netifyd starts before ns-flows and ns-stats, so the traffic data it sends to
them fails until they are listening
rsyslog starts forwarding to victoria-logs long before victoria-logs is up, so
early boot messages are lost and the forwarding errors are logged
vmalert and telegraf start before victoria-metrics, so their queries and metric
writes fail until it is up
Components
Image version: 8.8.0
netifyd 5.2.6
nginx-ssl 1.26.3
ns-monitoring 1.2.1 (ns-flows, ns-stats)
ns-reverse-proxy 1.0.0
ns-ui 2.23.2
rsyslog 8.2506.0
telegraf 1.39.1
victoria-logs 1.51.0
victoria-metrics 1.146.0
Steps to reproduce
firstboot && rebootExpected behavior
Every service starts once, after the services it needs are already listening. The boot
log contains no connection errors between local services and no duplicate service
starts.
Actual behavior
Several services start before the services they talk to, so they either get restarted
later in the boot sequence or keep retrying until their dependency is up. Both cases
fill the log with avoidable messages at every boot:
nginxstarts beforens-uiandns-reverse-proxyhave generated its locationfiles, so it starts with an incomplete configuration and
ns-uirestarts it rightafter — nginx is started twice on every boot
netifydstarts beforens-flowsandns-stats, so the traffic data it sends tothem fails until they are listening
rsyslogstarts forwarding tovictoria-logslong beforevictoria-logsis up, soearly boot messages are lost and the forwarding errors are logged
vmalertandtelegrafstart beforevictoria-metrics, so their queries and metricwrites fail until it is up
Components
Image version: 8.8.0
netifyd 5.2.6
nginx-ssl 1.26.3
ns-monitoring 1.2.1 (ns-flows, ns-stats)
ns-reverse-proxy 1.0.0
ns-ui 2.23.2
rsyslog 8.2506.0
telegraf 1.39.1
victoria-logs 1.51.0
victoria-metrics 1.146.0