Please note! This wiki is no longer maintained. Our documentation has moved to https://securityonion.net/docs/. Please update your bookmarks. You can find the latest version of this page at: https://securityonion.net/docs/Salt.
What is Salt?
"Salt delivers a dynamic communication bus for infrastructures that can be used for orchestration, remote execution, configuration management and much more."
What is OnionSalt?
"OnionSalt is a tool created to manage multiple Security Onion sensors."
For the Security Onion 14.04 ISO,
securityonion-onionsalt is pre-installed (via
securityonion-iso syslog-ng-core) , and Salt is configured by default when choosing Best Practices during setup.
Salt and OnionSalt are optional packages
If you choose to install Security Onion via PPA without installing
securityonion-iso syslog-ng-core, please note that Salt is totally optional. If you're happy with your current method of sensor management, then you don't have to install
securityonion-onionsalt, and nothing will change for you. Otherwise, install
securityonion-onionsalt before running setup to enable Salt for your deployment.
Sensors need to be able to connect to the master server on ports 4505/tcp and 4506/tcp:
For new deployments, Best Practices (Production Mode) checks to see if the
securityonion-onionsalt package is installed and, if so, enables Salt by default. If choosing the "Custom" configuration option (Production Mode), simply answer "Yes" at the prompt (where applicable), and setup will configure salt-master and/or salt-minion services and open firewall ports as necessary.
For existing deployments, please see:
Want to verify all your sensors are up?
sudo salt '*' test.ping
Want to execute a command on all your sensors at once?
sudo salt '*' cmd.run 'InsertYourCommandHere'
When you install and enable securityonion-onionsalt, the following data will replicate from the master server out to the sensors every 15 minutes:
- NIDS rules in /etc/nsm/rules/ (Snort/Suricata/barnyard will automatically restart as necessary)
- HIDS rules in /var/ossec/rules/local_rules.xml (OSSEC will automatically restart as necessary)
- Bro scripts in /opt/bro/share/bro/policy/
- Bro does not restart automatically, but you can easily use salt on your master server to tell all your Bro instances to update and restart:
# Force all Salt minions to update Bro scripts sudo salt '*' state.highstate # Restart Bro sudo salt '*' cmd.run 'nsm_sensor_ps-restart --only-bro'
- Bro intel in /opt/bro/share/bro/intel/
- You'll need to restart Bro as shown above if you add any intel files to the default intel.dat. After that initial Bro restart, Bro should be watching the intel files with the Input framework which should automatically notice if the files ever change (new intel is added). In many cases, you won't need to restart Bro if you're just adding intel to the existing intel file(s).
- user accounts and sudoers in /opt/onionsalt/pillar/users/init.sls
- user ssh keys in /opt/onionsalt/salt/users/keys/
- For each user account in /opt/onionsalt/pillar/users/init.sls, you can add an SSH Public Key to /opt/onionsalt/salt/users/keys/USERNAME.id_rsa.pub (replacing USERNAME with the user's actual username)
In addition, Salt is a full configuration management system, so you can script anything that you want to deploy across your army of sensors.
Using Salt to Install Updates Across Your Entire Deployment
You can use Salt and Soup to install updates across your entire deployment, but please remember to always update your master server first:
# Update Master first # If MySQL and/or kernel updates are installed, it will reboot sudo soup -y # After Master server is fully updated, now update the rest of the deployment # If MySQL and/or kernel updates are installed, the sensors will reboot sudo salt '*' cmd.run 'soup -y'
Also, please keep in mind that occasionally Ubuntu will release updates that prompt for user input which would cause that last command to hang. If you experience this, you should be able to ssh to each sensor and run
soup interactively. For more information, please see Issue 1108.
Modifying Salt config files
If you need to modify the values in /etc/salt/master or /etc/salt/minion, please pay attention to this note at the top of each file:
# /etc/salt/master # Per default, the master will automatically include all config files # from master.d/*.conf (master.d is a directory in the same directory # as the main master config file) #default_include: master.d/*.conf
# /etc/salt/minion # Per default the minion will automatically include all config files # from minion.d/*.conf (minion.d is a directory in the same directory # as the main minion config file). #default_include: minion.d/*.conf
Instead of modifying /etc/salt/master or /etc/salt/minion directly, please add your custom settings in /etc/salt/master.d/
*.conf or /etc/salt/minion.d/
Changing Minion ID
If you need to change the ID for a minion, do the following:
On the minion machine:
# Stop salt-minion sudo service salt-minion stop # Edit /etc/salt/minion_id, modifying the ID as necessary. # Start salt-minion sudo service salt-minion start
On the master machine:
# Restart salt-master sudo service salt-master restart # List the salt keys sudo salt-key -L # Accept the new key for the modified minion sudo salt-key -A # Delete the old minion key sudo salt-key -d OLD_MINION_NAME # Test the configuration -- minion should return "TRUE" sudo salt "MINION_NAME" test.ping
Salting an Existing Deployment
Configure the Master Server first
# Make sure the necessary packages are installed and updated sudo apt-get update && sudo apt-get install securityonion-onionsalt # Create a starting /opt/onionsalt/pillar/users/init.sls and /opt/onionsalt/salt/top.sls file from the template. sudo cp /opt/onionsalt/salt/top.sls.template /opt/onionsalt/salt/top.sls sudo cp /opt/onionsalt/pillar/users/init.sls.template /opt/onionsalt/pillar/users/init.sls # Edit /opt/onionsalt/salt/top.sls and add your master as a "backend". # For example, if your SO master server's hostname is so-master, then replace: # My Onion Backend: 'C*': - backend with: # My Onion Backend: 'so-master': - backend # Open salt ports in firewall: # sudo ufw allow salt # OR preferably just allow from your sensor IP addresses like this: # sudo ufw allow proto tcp from a.b.c.d to any port 4505,4506 # Also see our Firewall page: # https://github.com/Security-Onion-Solutions/security-onion/wiki/Firewall # Configure minion echo "master: localhost" | sudo tee -a /etc/salt/minion.d/onionsalt.conf # Allow salt-master and salt-minion to start on boot if they had previously been disabled [ -f /etc/init/salt-master.DISABLED ] && sudo mv /etc/init/salt-master.DISABLED /etc/init/salt-master.conf [ -f /etc/init/salt-master.override ] && sudo rm -f /etc/init/salt-master.override [ -f /etc/init/salt-minion.DISABLED ] && sudo mv /etc/init/salt-minion.DISABLED /etc/init/salt-minion.conf [ -f /etc/init/salt-minion.override ] && sudo rm -f /etc/init/salt-minion.override # Restart minion sudo service salt-minion restart # list the salt keys: sudo salt-key -L # You should see an unaccepted salt key for the minion, add it: sudo salt-key -a '*' # Verify that the master can communicate with the minion: sudo salt '*' test.ping # Tell salt to do an update sudo salt '*' state.highstate
Now configure salt-minion on a Sensor
# Make sure the necessary packages are installed and updated sudo apt-get update && sudo apt-get install securityonion-onionsalt # Stop the running salt-master sudo service salt-master stop # Disable salt-master [ -f /etc/init/salt-master.conf ] && echo "manual" | sudo tee /etc/init/salt-master.override # Allow salt-minion to start on boot if it had previously been disabled [ -f /etc/init/salt-minion.DISABLED ] && sudo mv /etc/init/salt-minion.DISABLED /etc/init/salt-minion.conf [ -f /etc/init/salt-minion.override ] && sudo rm -f /etc/init/salt-minion.override # Configure minion MASTER=`grep SENSOR_SERVER_HOST /etc/nsm/*/sensor.conf |head -1 |cut -d\" -f2` echo "master: $MASTER" | sudo tee -a /etc/salt/minion.d/onionsalt.conf # Restart minion sudo service salt-minion restart
Now return to the Master and accept the new minion
# Edit /opt/onionsalt/salt/top.sls and add the new minion as a "sensor" # list the salt keys: sudo salt-key -L # You should see an unaccepted salt key for the sensor, add it: sudo salt-key -a '*' # Verify that the master can communicate with all minions: sudo salt '*' test.ping # Tell all minions to do an update sudo salt '*' state.highstate
Maximum Event Size
Salt-master uses a default
max_event_size of 1048576 bytes (1 Mebibyte). For some Security Onion deployments, this may need to be change to a larger value to avoid receiving a
VALUE_TRIMMED error (if the output of a command run on a minion is too large to be passed back to the master).
This setting should be changed in
/etc/salt/master.d/onionsalt.conf, as opposed to directly in /etc/salt/master.
On a distributed Security Onion deployment
/etc/salt/master.d/onionsalt.conf (on the master) should look like the following:
file_roots: base: - /opt/onionsalt/salt pillar_roots: base: - /opt/onionsalt/pillar max_event_size: YOUR_NEW_VALUE
After making changes, ensure salt-master has been started/restarted:
sudo service salt-master restart