so-strelka-backend keeps restarting

[gihtub666]

Version

3.0.0

Installation Method

Security Onion ISO image

Description

configuration

Installation Type

Standalone

Location

airgap

Hardware Specs

Meets minimum requirements

CPU

16

RAM

64 GB

Storage for /

155 GB (23% used)

Storage for /nsm

311 GB (30% used)

Network Traffic Collection

tap

Network Traffic Speeds

Less than 1Gbps

Status

No, one or more services are failed (please provide detail below)

Salt Status

No, there are no failures

Logs

No, there are no additional clues

Detail

Hi everyone,
I have installed the SO on my Acer laptop as a bare metal installation. The entire installation process seemed to be OK. However, when the management interface is not connected to my other “SOC” laptop (that I use to check Kibana etc. ) the so-strelka-backend container dies (starts to restart in an endless loop to be more precise) and it seems that entire SO stops ingesting traffic (seen on Kibana graphs). When I start my laptop and connection between the laptop and SO is established again the so-strelka-backend container starts and all seems to work as expected. The result is that my SO is only monitoring traffic when my “SOC” laptop is up and connected.

All services are up and running. There is sometimes problem with the salt-minion.service when the SO is started without connection to a management “SOC” laptop.

I have tried to use AI but without any usable results so now it seems I need somebody’s real intelligence to help me out. Any help will be much appreciated.

Side note: the problem is now on version 3.0.0-20260331 but was same for version 2.4.211-20260312 (both freshly installed no attempts to upgrade).

Here is what docker logs so-strelka-backend returns:

  __import__('pkg_resources').run_script('strelka==0.0.0', 'strelka-backend')
2026-05-19 15:33:15 - [DEBUG] root [strelka.__init__]: coordinator up
2026-05-19 15:33:15 - [INFO] root [strelka.work]: starting up
2026-05-19 15:33:15 - [INFO] root [strelka.check_scanners]: checking scanners
/usr/local/lib/python3.10/dist-packages/oletools/oleobj.py:581: SyntaxWarning: "is" with a literal. Did you mean "=="?
  if idx is -1:
Traceback (most recent call last):
  File "/usr/local/lib/python3.10/dist-packages/redis/connection.py", line 865, in read_response
    response = self._parser.read_response(disable_decoding=disable_decoding)
  File "/usr/local/lib/python3.10/dist-packages/redis/connection.py", line 346, in read_response
    result = self._read_response(disable_decoding=disable_decoding)
  File "/usr/local/lib/python3.10/dist-packages/redis/connection.py", line 356, in _read_response
    raw = self._buffer.readline()
  File "/usr/local/lib/python3.10/dist-packages/redis/connection.py", line 259, in readline
    self._read_from_socket()
  File "/usr/local/lib/python3.10/dist-packages/redis/connection.py", line 209, in _read_from_socket
    data = self._sock.recv(socket_read_size)
ConnectionResetError: [Errno 104] Connection reset by peer

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
  File "/usr/local/bin/strelka-backend", line 4, in <module>
    __import__('pkg_resources').run_script('strelka==0.0.0', 'strelka-backend')
  File "/usr/local/lib/python3.10/dist-packages/pkg_resources/__init__.py", line 747, in run_script
    self.require(requires)[0].run_script(script_name, ns)
  File "/usr/local/lib/python3.10/dist-packages/pkg_resources/__init__.py", line 1734, in run_script
    exec(code, namespace, namespace)
  File "/usr/local/lib/python3.10/dist-packages/strelka-0.0.0-py3.10.egg/EGG-INFO/scripts/strelka-backend", line 41, in <module>
    main()
  File "/usr/local/lib/python3.10/dist-packages/strelka-0.0.0-py3.10.egg/EGG-INFO/scripts/strelka-backend", line 35, in main
    backend.work()
  File "/usr/local/lib/python3.10/dist-packages/strelka-0.0.0-py3.10.egg/strelka/strelka.py", line 296, in work
    task = self.coordinator.zpopmin("tasks", count=1)
  File "/usr/local/lib/python3.10/dist-packages/redis/commands/core.py", line 4229, in zpopmin
    return self.execute_command("ZPOPMIN", name, *args, **options)
  File "/usr/local/lib/python3.10/dist-packages/redis/client.py", line 1258, in execute_command
    return conn.retry.call_with_retry(
  File "/usr/local/lib/python3.10/dist-packages/redis/retry.py", line 49, in call_with_retry
    fail(error)
  File "/usr/local/lib/python3.10/dist-packages/redis/client.py", line 1262, in <lambda>
    lambda error: self._disconnect_raise(conn, error),
  File "/usr/local/lib/python3.10/dist-packages/redis/client.py", line 1248, in _disconnect_raise
    raise error
  File "/usr/local/lib/python3.10/dist-packages/redis/retry.py", line 46, in call_with_retry
    return do()
  File "/usr/local/lib/python3.10/dist-packages/redis/client.py", line 1259, in <lambda>
    lambda: self._send_command_parse_response(
  File "/usr/local/lib/python3.10/dist-packages/redis/client.py", line 1235, in _send_command_parse_response
    return self.parse_response(conn, command_name, **options)
  File "/usr/local/lib/python3.10/dist-packages/redis/client.py", line 1275, in parse_response
    response = connection.read_response()
  File "/usr/local/lib/python3.10/dist-packages/redis/connection.py", line 871, in read_response
    raise ConnectionError(
redis.exceptions.ConnectionError: Error while reading from sauron:6380 : (104, 'Connection reset by peer')
/usr/local/bin/strelka-backend:4: DeprecationWarning: pkg_resources is deprecated as an API. See https://setuptools.pypa.io/en/latest/pkg_resources.html
  __import__('pkg_resources').run_script('strelka==0.0.0', 'strelka-backend')
2026-05-19 18:03:35 - [DEBUG] root [strelka.__init__]: coordinator up
/usr/local/lib/python3.10/dist-packages/oletools/oleobj.py:581: SyntaxWarning: "is" with a literal. Did you mean "=="?
  if idx is -1:
2026-05-19 18:03:35 - [INFO] root [strelka.work]: starting up
2026-05-19 18:03:35 - [INFO] root [strelka.check_scanners]: checking scanners



### Guidelines

- [X] I have read the discussion guidelines at https://github.com/Security-Onion-Solutions/securityonion/discussions/1720 and assert that I have followed the guidelines.

[cm-ops]

Would you be able to share you salt minion log? /opt/so/log/salt/minion

[gihtub666]

Yes, here it is. Thank you. :-)

[gihtub666]

Sorry, the file did not append. I'll try again...
so_minion_20260521.gz

[cm-ops]

I see some salt errors for the component ssl states, but looks like they did run after the errors. If you run sudo so-checkin does that complete without error?

Also, what is the output of these? Do they show the correct interfaces and IPs?

sudo salt-call mine.get '*' network.ip_addrs
sudo salt-call pillar.get host:mainint
sudo salt-call grains.get ip_interfaces

Can you expand on how your management interface is connected or not connected?

[gihtub666]

Hello,
sudo so-checkin - finishes without errors:

---------------
Succeeded: 1668 (changed=41)
Failed:       0
---------------

sudo salt-call mine.get '*' network.ip_addrs - returns management IP of the SO:

    ----------
    [hostname]_standalone:
        - [management_IP_v4]

sudo salt-call pillar.get host:mainint - returns list of all interface

    ----------
    bond0:
    docker0:
        - 172.17.0.1
    enp0s13f0u1:
        - [management_IP_v4]
        - [management_IP_v6]
    enp0s20f0u1:
    enp0s20f0u2:
    lo:
        - 127.0.0.1
    sobridge:
        - 172.17.1.1
    veth09164a4:
    veth0e581d1:
    veth121384d:
    veth24aa5dc:
    veth2f505ec:
    veth4fa41f4:
    veth67227dd:
    veth6d95633:
    veth8567b0d:
    veth8e99bfc:
    veth92a3851:
    vethb3fd43a:
    vethb40fb66:
    vethd464771:
    vethd756483:
    vethe6020b6:
    vethf767572:
    vethfbd6ee6:

I will try to explain my situation once more: I have Security Onion standalone version installed as a bare metal installation on my laptop. It has three USB to ETH interfaces. Two are for monitoring and one is for management (say IP 1.1.1.1). The management interface is connected directly via Ethernet cable to my other laptop (say IP 1.1.1.2). When the second laptop is shut down or disconnected, the so-strelka-backend starts an endless loop of restarting itself and entire Security Onion stops ingesting traffic (or maybe processing it - there is nothing to be seen during that period in Kibana). Once my second laptop is up and connected so-strelka-backend starts up and runs OK and SO starts processing data once again.

[gihtub666]

This is what makes me think, that when the management interface is down, no data is collected.

[cm-ops]

That is correct, when you take down the management interface your ingest pipeline will stop. The bond interfaces will still see the traffic if they are up and the raw logs will be written, but your Elastic Agent can't send them because the management interface is disconnected.

[gihtub666]

Does it mean that each time the management is down SO does not monitor my network? Shouldn't it be able to collect and monitor network even though the management is down? It is an all in one (Standalone) installation so the Kibana etc. is installed locally - as can be seen from the so-status command output:

                       Security Onion Status
                         Container │ Status  │              Details
───────────────────────────────────┼─────────┼──────────────────────
                 so-dockerregistry │ running │           Up 10 days
                     so-elastalert │ running │           Up 10 days
                  so-elastic-fleet │ running │           Up 10 days
 so-elastic-fleet-package-registry │ running │ Up 10 days (healthy)
                  so-elasticsearch │ running │           Up 10 days
                       so-influxdb │ running │ Up 10 days (healthy)
                         so-kibana │ running │           Up 10 days
                         so-kratos │ running │           Up 10 days
                       so-logstash │ running │           Up 10 days
                          so-nginx │ running │ Up 10 days (healthy)
                          so-redis │ running │           Up 10 days
                      so-sensoroni │ running │           Up 10 days
                            so-soc │ running │           Up 10 days
                so-strelka-backend │ running │     Up About an hour
            so-strelka-coordinator │ running │           Up 10 days
             so-strelka-filestream │ running │           Up 10 days
               so-strelka-frontend │ running │           Up 10 days
             so-strelka-gatekeeper │ running │           Up 10 days
                so-strelka-manager │ running │           Up 10 days
                       so-suricata │ running │           Up 10 days
                       so-telegraf │ running │           Up 10 days
                           so-zeek │ running │ Up 10 days (healthy)

All the data should be collected and processed on this one SO machine. The laptop connected to the management interface is only used to connect to the monitoring (SO) laptop and to manage it and visualize information about monitored network. I guess if I could install graphical interface and e.g. firefox on the monitoring laptop I could open the https://127.0.0.1/kibana and analyse all data without the need for second laptop. Or is there something I am missing?

Would it help if I set somehow somewhere that the management IP is not only IP of my second laptop but also 127.0.0.1 (which is always reachable)? Maybe then the so-strelka and data collection would stop failing and all would work OK?

[cm-ops]

You should only need an analyst host firewall allowance to log in to the SOC GUI from a browser on a computer in your network. For your second laptop, as long as the IP is in the host firewall, you should be able to login to the GUI. There is no need for the ethernet connection between laptop one and two.

When you management NIC is down your raw network logs which are collected by the monitoring interface will still get written. Those logs won't be sent through the ingest pipeline. That is why you don't see any logging in the GUI when it is down.

[gihtub666]

Hello and thanks. I read your response multiple times but it seems I do not know enough to understand what you mean.

"You should only need an analyst host firewall allowance to log in to the SOC GUI from a browser on a computer in your network" - my analyst host is my laptop 2. I do not have all network allowed to connect to the management of the laptop 1 (all-in-one standalone machine), but only a single IP of the laptop 2.

"For your second laptop, as long as the IP is in the host firewall, you should be able to login to the GUI. " - Yes, I can log in from laptop 2.

"There is no need for the ethernet connection between laptop one and two." - I do not get this. If I need to connect from laptop 2 to laptop 1 so I can see what is going on on my network I need to have a connection established between these two machines.

"When you management NIC is down your raw network logs which are collected by the monitoring interface will still get written. Those logs won't be sent through the ingest pipeline. That is why you don't see any logging in the GUI when it is down." - Does this mean that all logs are written but never processed? Is that intentional? Are the logs processed when the management interface is up again or will they never be processed? If the second case is right - can I change this behaviour so that my network monitoring is up and running even though the management interface is down?

And how about failing so-strelka-backend container? How can I make it run even without the management interface up?

Thank you for your patience. :-)

[cm-ops]

What is in this setting in SOC > Administration > Configuration >firewall > hostgroups > analyst?

[gihtub666]

There is just the IP v4 of my analyst laptop (laptop 2). Not CIDR but only a single IP.

[cm-ops]

So you using your above, that IP is 1.1.1.2 (your example from above)? What is your management interface IP on your SO node? Is is 1.1.1.1 or 1.1.1.2?

If your management interface on SO is 1.1.1.1 and you have a firewall allowance for 1.1.1.2 you should only need to open a web browser, Chrome for example, and use your IP (or what you set as your url_base) and connect without a direct ethernet connection.

https://docs.securityonion.net/en/3/main/security-onion-console/ shows this, analyst machine to SO via browser.