soup fails with error 160 on standalone 3.0.0 — reads Elasticsearch target version from wrong path (salt/manager/defaults.yaml instead of salt/elasticsearch/defaults.yaml)

[baileyphil]

Version

3.0.0

Installation Method

Security Onion ISO image

Description

upgrading

Installation Type

Standalone

Location

on-prem with Internet access

Hardware Specs

Exceeds minimum requirements

CPU

8

RAM

64

Storage for /

8000

Storage for /nsm

8000

Network Traffic Collection

span port

Network Traffic Speeds

1Gbps to 10Gbps

Status

Yes, all services on all nodes are running OK

Salt Status

Yes, there are salt failures (please provide detail below)

Logs

No, there are no additional clues

Detail

Running sudo soup fails during the pre-upgrade Elasticsearch compatibility check with error 160. Relevant output:
Verifying Elasticsearch version compatibility across the grid before upgrading.
Missing filename or key arg
Usage: /sbin/so-yaml.py <YAML_FILE> [ARGS...]
...
Couldn't determine the target Elasticsearch version (post soup version) to ensure compatibility with current Elasticsearch version. Exiting
Soup failed with error 160: Incompatible Elasticsearch upgrade
The soup script version check reports "up to date" and the salt master / pillars render fine — it only fails at the ES version-compatibility step.
Diagnosis (what I found):
soup appears to read the target Elasticsearch version from the wrong file. The key it queries does not exist where it's looking:

Where soup appears to look — key absent:

$ sudo /sbin/so-yaml.py get /tmp/sogh/securityonion/salt/manager/defaults.yaml elasticsearch.version
Key 'elasticsearch.version' not found by so-yaml.py

Where the value actually lives:

$ sudo find /tmp/sogh/securityonion -name 'defaults.yaml' -path 'elasticsearch'
/tmp/sogh/securityonion/salt/elasticsearch/defaults.yaml

$ sudo head -5 /tmp/sogh/securityonion/salt/elasticsearch/defaults.yaml
elasticsearch:
enabled: false
version: 9.3.3
index_clean: true
...
So the target version (9.3.3) is present at elasticsearch.version in salt/elasticsearch/defaults.yaml, but the compatibility check queries salt/manager/defaults.yaml, where that key doesn't exist — producing the empty argument to so-yaml.py and the error 160 abort.

Guidelines

• I have read the discussion guidelines at Read before posting! #1720 and assert that I have followed the guidelines.

[rock0ne]

Potential Solution;; Run the below commands in order;;
sudo cp -av /usr/sbin/so-yaml.py /root/so-yaml.py.$(date +%Y%m%d-%H%M%S).bak

sudo salt-call state.apply common.soup_scripts
queue=True -l info
--file-root=/tmp/sogh/securityonion/salt
--local

sudo /usr/sbin/so-yaml.py get -r /tmp/sogh/securityonion/salt/elasticsearch/defaults.yaml elasticsearch.version
sudo /sbin/so-yaml.py get -r /tmp/sogh/securityonion/salt/elasticsearch/defaults.yaml elasticsearch.version

OUTPUT: 9.3.3 and then run sudo soup. worked on my machine.

I hit the same SOUP failure:

Missing filename or key arg
Couldn't determine the target Elasticsearch version
Soup failed with error 160: Incompatible Elasticsearch upgrade

In my case the issue was not the Elasticsearch defaults file path. The target version existed here:

/tmp/sogh/securityonion/salt/elasticsearch/defaults.yaml

[baileyphil]

Thank you so much this worked perfectly, fixed my issue 👍

[reyesj2]

Glad your issue was resolved, it appears like you ran into an issue with a helper utility not being updated as expected. Trying to pinpoint where the initial error occurs, if you could provide some additional info it may help.

Was this your first attempt to soup to 3.1.0? Or did you first soup attempt error out, resulting in you re-running soup?

How did you run soup? Was it from the ISO or network install ? ie did you run sudo soup -y -f /home/YourUser/securityonion-3.X.Y-YYYYMMDD.iso or sudo soup ?

Are you able to provide your soup logs for review? They would be located in /root and you can pack them all into 1 tar.gz with the following

sudo bash -c 'cd /root && tar -czvf /tmp/soup-logs.tar.gz soup.log*'

that will create /tmp/soup-logs.tar.gz file with the /root/soup.log* files in it