containers fail to boot after reboot. so-status shows highstate is stuck for a long time - "no highstate has completed..." for over an hour

[zeeker-lang]

Version

2.4.200

Installation Method

Security Onion ISO image

Description

other (please provide detail below)

Installation Type

Standalone

Location

airgap

Hardware Specs

Exceeds minimum requirements

CPU

16

RAM

64

Storage for /

2TB

Storage for /nsm

650GB

Network Traffic Collection

tap

Network Traffic Speeds

1Gbps to 10Gbps

Status

No, one or more services are failed (please provide detail below)

Salt Status

Yes, there are salt failures (please provide detail below)

Logs

No, there are no additional clues

Detail

After shutdown during testing, I booted the SO up and it showed highstate was in progress, but when I ran 'docker ps' it showed only 3 containers were up, the rest were exited and so-strelka was stuck in a restarting loop. I rebooted and tried to restart the salt-master and salt-minion using systemctl after a while. salt-master restarted successfully however the minion failed to start again and shows timeout errors. my salt key is accepted, it worked in the past. the system is airgapped and directly deployed on a HP z2 g9 PC as a standalone deployment with the web UI being accessible from an accompanying device.

Guidelines

• I have read the discussion guidelines at Read before posting! #1720 and assert that I have followed the guidelines.

[cm-ops]

What happens if you run the below commands? Does the node come up?

sudo salt-call saltutil.kill_all_jobs
sudo salt-call state.highstate