Skip to content
This repository has been archived by the owner on Apr 16, 2021. It is now read-only.

IntroductionWalkthrough

Jump to bottom
doug edited this page Aug 27, 2019 · 17 revisions

Please note! This wiki is no longer maintained. Our documentation has moved to https://securityonion.net/docs/. Please update your bookmarks. You can find the latest version of this page at: https://securityonion.net/docs/IntroductionWalkthrough.

Please note this walkthrough was written several years ago for the previous version of Security Onion (12.04). The current version of Security Onion is 16.04 and you can follow one of the updated Installation guides.

Walkthrough

Security Onion is configured to run on version 12.04 of any Ubuntu-based Linux server or desktop distribution, such as Ubuntu, Lubuntu, Xubuntu, and Kubuntu. Your base operating system choice really depends on personal preference, your hardware and how you intend to interact with Security Onion. If you're experienced with the flavors of Ubuntu you probably have already made this decision. We're going to walkthrough setting up the Security Onion Live Xubuntu 12.04 distribution in a virtual machine (VM) and installing Security Onion using the Quick Setup option. Having Security Onion installed in a VM gives you an isolated environment which can act as a "client" for interacting with a remote Security Onion server. In an Ubuntu Server deployment, where access to the server is limited to SSH and command line, the client VM will let us setup remote servers and sensors graphically. It is also recommended for analysts to run Security Onion in a virtual machine for client access to ensure you have all the tools needed to manage and monitor a deployment in an isolated environment. You'll need a computer with at least 4GB of RAM (ideally 8GB) for best results. We'll use VirtualBox, a free desktop virtualization tool, but the process is very similar for VMware or others. You can download a copy of VirtualBox for Windows, Mac OS X or Linux at http://www.virtualbox.org. We'll also need to download the Security Onion 12.04 Live distribution from the Download page.

Once downloaded, install VirtualBox, then launch it and click the "New" button. First we'll provide a name for our virtual machine ("Security Onion Standalone Client" for example) and specify the type ("Linux") and version ("Ubuntu" or "Ubuntu 64 bit"), then click "Continue." We'll next define how much memory we want to make available to our virtual machine. For a Security Onion client, you should dedicate at least 2GB RAM, but ideally 4GB if possible. 2GB RAM will provide decent performance for the Sguil client and retrieving packet captures from the server but also enough to run Security Onion in standalone mode for monitoring the local client and testing packet captures with tools like tcpreplay, but you could potentially experience some delays under heavy use or retrieving large packet captures.

Next we'll create a virtual hard drive for our client. Specify "Create a virtual hard drive now" then click "Create" to choose the hard drive file type "VDI (VirtualBox Disk Image)" and "Continue." For storage, we have the options of "Dynamically allocated" or "Fixed size." For a client virtual machine, "Dynamically allocated" is the best choice as it will grow the hard disk up to whatever we define as the maximum size on an as needed basis until full, at which point Security Onion's disk cleanup routines will work to keep disk space available. If you happen to be running a dedicated sensor in a virtual machine, I would suggest using "Fixed size," which will allocate all of the disk space you define up front and save you some disk performance early on. Once you've settled on the storage allocation, click "Continue" and provide a name from your hard disk image file and specify the location where you want the disk file to be created if other than the default location. For disk size, you'll want enough disk capacity for retrieving/testing packet captures and downloading system updates. At a minimum for a client, I would designate at least 20GB, with 40GB recommended. Click "Create" and your Security Onion client virtual machine will be created.

At this point, you can click "Settings" for your new virtual machine so we can get it configured. You might want to increase your display virtual memory to 128MB of RAM, but most other settings should be fine. We do, however, need to do a couple of things. First, mount the Security Onion Live 12.04 .iso file we downloaded so our virtual machine can boot from it to install Linux. Click the "Storage" icon, then under "Controller: IDE" select the "Empty" CD icon. To the right, you'll see "CD/DVD Drive" with "IDE Secondary" specified with another CD icon. Click the icon, then select "Choose a virtual CD/DVD disk file" and browse to where you downloaded the Security Onion Live 12.04 .iso file, select it then choose "Open." Next click "Network" then "Adapter 2." You'll need to click the checkbox to enable it then attach it to "Internal Network." Under the "Advanced" options, set "Promiscuous Mode" to "Allow All." Click "Ok" and we are ready to install the operating system.

Hit the "Start" button with your new virtual machine selected and after a few seconds the boot menu will load. Choose "live - boot the Live System" or wait and the Security Onion desktop will load. Double-click the "Install SecurityOnion 12.04" icon on the desktop to initiate the first of a handful of operating system setup screens. After language selection we'll see some information about our system in "Preparing to install SecurityOnion." Go ahead and check the box to "Download updates while installing" if you have an active network connection, then click "Continue." On the "Installation type" screen we want to "Erase disk and install SecurityOnion" which will partition and format the virtual disk we created. Click "Continue" again then "Install Now." After a moment, you'll be prompted to specify a time zone, which isn't important for us as Security Onion will default the system to UTC time, so you can just click "Continue" to move on to "Keyboard layout" selection where the defaults are usually sufficient. "Continue" again to create a user account for accessing the system and set the virtual machine's hostname. The username and password defined here will be the only account that can access the system once the installation is complete until you add additional accounts. Click "Continue" and after a few minutes the installation with complete and you'll be prompted to "Restart Now."

There are two things we need to do once our virtual machine boots up on it's own for the first time to prepare for Security Onion setup: install all Ubuntu/SO updates and install VirtualBox Guest Additions (or VMware Tools if you're using VMware). Go ahead and login with the username and password you created during the install and double-click the "Terminal Emulator" icon on the desktop. Now that we have a command prompt, type:

sudo soup

You'll be asked to enter your password since we're executing a "sudo" command. By default, "root" access is disabled in Ubuntu. The "sudo" command allows authorized users, like system administrators, to execute commands that would otherwise require root access without having to know the root password. It is similar to how recent versions of Microsoft Windows may prompt you to confirm administrative actions, when installing new applications for example. The command we execute tells Linux: update the package and repository listings with root access without prompting me to tell you yes & again with root access upgrade this Linux distribution with the latest patches & again with root access, reboot. Patching will take a few minutes on a fresh installation and when it's done the system will restart. When the system comes back up, log in again and open another terminal window.

At the top of your virtual machine window you'll notice menu items for VirtualBox. Click on your virtual machine window, then on the menu click "Devices" then "Install Guest Additions..." Doing so will mount the VirtualBox guest additions CD on your virtual machine and it will open the folder showing you the files now available. Click on your terminal window and type "cd /media/VBOX" then press the Tab key key to autofill the folder name and press Enter to change to that directory. To install the Guest Additions type:

sudo ./VBoxLinuxAdditions.run

You'll again be prompted for your password since we're running sudo for the first time after a reboot. The installation will launch and after a couple minutes you'll return to the command prompt when it's complete. In the upper right hand corner of your Xubuntu desktop, click your username then "Shut down" to shut down the system.

Before we install Security Onion, this is an excellent time to take a snapshot of your virtual machine. While the system is shutdown, you'll notice two icons on the top right in VirtualBox Manager when you select your virtual machine: Details and Snapshots. Click "Snapshots" then click the camera icon and give your snapshot a name and description. I recommend something descriptive here, perhaps naming it "New Build SO Client" with a description including details that the system was patched and updated with VirtualBox Guest Additions installed and provide the date. Once we have a snapshot, we'll be able to make changes to the system and revert those changes back to the state we are preserving. For a Security Onion client this is useful, as we can setup Security Onion as a standalone server for testing, then later revert to the snapshot and reinstall Security Onion to only use the client tools. Boot up the system again once you've completed the snapshot and we'll install Security Onion.

At this point, without running the Security Onion setup script, you have a fully functioning Security Onion client workstation environment with which to access a Security Onion server. Next we're going to install Security Onion using the Quick Setup to familiarize ourselves with the setup and get started learning the tools. Once we're done experimenting we can revert our VM to the snapshot we just took and be back to a clean, Security Onion, client-only state.

When you're logged in again, double-click the "Setup" icon on the desktop to begin Security Onion setup. You'll first be prompted for your password then asked to confirm that you want to continue with the installation. Choose "Yes, Continue!" and you'll be asked if you would like to configure /etc/network/interfaces now. You'll first be asked "Which network interface should be the management interface?" Choose "eth0" and you'll then be asked whether to use static or DHCP IP addressing, with static being highly recommended. When installing a production server or sensor you should make sure to use static IP addressing, but for our client VM we can use DHCP if static addressing isn't available or the VM is going to change networks. Just be aware that IP address changes can cause issues with some of the Security Onion agents, so if you have a test VM using DHCP, you may need to re-run the Security Onion setup. If you can assign a static IP, choose static and you'll be asked to provide the IP address, gateway, netmask, broadcast address and DNS servers. You'll then be asked to configure the monitor interface. Choose "Yes, configure monitor interfaces" then choose "eth1" and click OK. Once you've made your network selections, click "Yes, make changes and reboot!"

Log back in and double-click the "Setup" icon again. Security Onion setup will detect that we've already configured the network interfaces, so choose "Yes, skip network configuration!" when prompted. You'll next be asked whether you want to install Security Onion using "Advanced Setup" or "Quick Setup." For this purpose, we'll use "Quick Setup" which will automatically configure most of your system to monitor one network interface. You'll first be asked "Which network interface should snort listen on?" Choose "eth1." You'll then be prompted for usernames and passwords for Sguil, Squert, ELSA and Snorby and whether or not you want to enable ELSA and that's it. "Advanced Setup" lets you specify whether the Security Onion instance will be running as a Server, Sensor or in Standalone mode, which IDS engine you'd prefer (Snort or Suricata), how many CPU cores you want to assign to Snort/Suricata and Bro IDS, which Snort/Suricata rule set you want to use, your username/passwords and whether you want to install ELSA. When you're done, click "Yes, proceed with the changes!" and Security Onion Setup will do the following:

  • Set the OS timezone to UTC;
  • Delete any existing NSM data/configuration;
  • Create a Sguil server and user;
  • Create a Snorby user;
  • Configure Snort and Bro to monitor interface eth0;
  • Run a single IDS process per interface;
  • Run a single Bro process per interface; and
  • Configure ELSA as both a Log Node and Web Node. It will take a minute for the setup to complete, but trust me it's a lot less time than it would take if you didn't have Security Onion guiding the way.

When setup is finished you'll see several informative prompts providing you valuable information about your new Security Onion host, which we'll cover here in more detail as a future reference.

  • "Setup log can be found here: /var/log/nsm/sosetup.log" - sosetup.log documents changes to the system and details about what occurred during the setup process. If you have any suspicions that some part of the setup failed, this should be your first stop.
  • "You may view IDS alerts using Sguil, Squert, Snorby or ELSA (if enabled)" - Sguil is a client application while Squert, Snorby and ELSA are web interfaces.
  • "Bro logs can be found in ELSA (if enabled) and the following location: /nsm/bro/" - Bro logs are rotated on an hourly basis and provide a wealth of information about the network being monitored, which can be invaluable during an investigation.
  • "Rules downloaded by PulledPork are stored in /etc/nsm/rules/downloaded.rules" - PulledPork is a tool to update and manage Snort IDS signatures. All rules that PulledPork downloads from Snort or Emerging Threats are stored in the downloaded.rules file.
  • "Local rules can be added to: /etc/nsm/rules/local.rules" - Local rules allow you to deploy custom signatures that will not be overwritten when PulledPork's scheduled updates occur.
  • "You can have PulledPork modify the downloaded rules by modifying the files in: /etc/nsm/pulledpork/" - There are a number of configuration file options for PulledPork:
    • pulledpork.conf
    • enablesid.conf
    • disablesid.conf
    • dropsid.conf
    • modifysid.conf
  • "Rules will be updated every day at 7:01 AM UTC. You can manually update them by running: rule-update" - The rule-update script allows you to manually run PulledPork to update signatures, which is most useful when tuning signatures by modifying the /etc/nsm/pulledpork .conf files.
  • "Sensors can be tuned by modifying the files in: /etc/nsm/HOSTNAME-INTERFACE/" - There are multiple configuration files in this path:
    • sensor.conf - contains a number of variables that are used throughout the Security Onion network monitoring services, such as which interfaces are being monitored, paths to config files, and more.
  • Typically you'll only need to modify the following files if you are monitoring IP address ranges other than private RFC1918 address space (192.168.0.0/16, 10.0.0.0/8, 172.16.0.0/12).

You can also type "sudo nsm_server_ps-status" to check the status of Sguil server. If any of the agents or server fail, a reference to the log file will be included that will be useful for troubleshooting.

A useful script to provide performance and health status of your Security Onion hosts is "sostat." You should run it periodically on any deployment with the command "sudo sostat | less" to review and monitor all aspects of Security Onion. It includes nsm_server|sensor_ps-status results, network interface status, disk usage, network sockets, IDS rule update status, CPU usage, log archive size, IDS engine packet drops, pf_ring stats, Sguil uncategorized events and summaries, top 50 URLs for previous day, and Snorby events and summaries. It provides powerful visibility into the health of Security Onion and should be adopted as part of your monitoring routine.

If everything looks ok, we can quickly test Sguil and Snort/Suricata detections. Double-click the Sguil icon on the desktop and enter your Sguil username and password (created during the Security Onion Setup). You'll be prompted to choose which network(s) to monitor: the monitored network interface(s) and/or OSSEC events. Choose "Select All" then "Start SGUIL" and the Sguil client will load. You might already have some events showing up, but just to confirm type "curl http://testmyids.com" in a terminal window and you should see an event appear in Sguil for "GPL ATTACK_RESPONSE id check returned root."

Security Onion includes a number of useful links on the desktop in addition to the Security Onion application menu which provides access to man pages for tools included in Security Onion. The "README" icon on the desktop is a good starting point and will open https://localhost in a web browser with local links to Squert and ELSA and external links to additional useful Security Onion information. Sguil, Squert and ELSA all share the same username/password. Here's a brief description of the primary tools available in Security Onion for security monitoring:

  • Sguil (http://sguil.sourceforge.net/) - THE analyst console for security monitoring. There isn't a more powerful and capable solution available for event analysis, correlation and review.
  • Squert (http://www.squertproject.org/) - A web interface to query and view Sguil event data that was designed to supplement Sguil by providing additional context around events.
  • ELSA (http://code.google.com/p/enterprise-log-search-and-archive/) - Enterprise Log Search and Archive is a distributed log archive system similar to Splunk and allows for event searching and visualization of all the log data Security Onion consumes, including OSSEC, Snort/Suricata, and most importantly, Bro IDS.

If you want to take some time to experiment with the tools, Security Onion includes some sample packet capture files we can replay to generate event data. I highly recommend physically disabling the network connection from your host operating system prior to replaying the packets. Since we're using a VM, simply unplugging the physical network connection or disabling wireless on the host system where your VM is running will work. We will be replaying samples of live traffic, some of which are malicious, so use caution. The samples provided are available in /opt/samples/ and once we're disconnected from the network, we can replay them by opening up a terminal and typing:

sudo tcpreplay -i eth1 -M10 /opt/samples/*.pcap

If you have the Sguil client up and running, you'll see events start to roll in to the console. You now have a decent sample of data to play with. Let's dig into one example to get a feeling for the capabilities at our disposal.

You'll notice several events with a source IP of 188.72.243.72 and a destination IP of 192.168.3.65 that indicate a potentially suspicious executable file has been downloaded causing several Snort IDS ET (Emerging Threats) signatures to fire: "ET INFO Packed Executable Download," "ET POLICY PE EXE or DLL Windows file download" and "ET INFO EXE IsDebuggerPresent (Used in Malware Anti-Debugging)." If we look at "ET INFO Packed Executable Download" we can see under the CNT column the number of correlated events and if you right-click on the value you can open a new tab showing all the correlated events. Additionally, you can right-click on the source or destination IP and port columns to query for additional IDS, SANCP or PADS events or perform DShield lookups. At this point, about all we know is a file was downloaded. We can click on the event entry for "ET INFO Packed Executable Download" then in the bottom right corner of the Sguil client we can click the "Show Packet Data" and "Show Rule" checkboxes. "Show Rule" will display the Snort/Suricata rule that created the event, while "Show Packet Data" will load the TCP headers and packet data. Poking through the packet data, we really only get confirmation of what we already know: Content-Type: application/x-msdownload tells us it was an exe, but we don't know what it was at this point.

A better option with Sguil is to right-click on the Alert ID column for the event and choose "Transcript." The transcript reaches out to the full packet captured events stored in /nsm and rebuilds a transcript of the session from the archived packet captures. Now we have context. Just from the top header information we almost immediately know the domain name (hosted-by.leaseweb.com) and the fingerprinted OS of our host involved (Windows XP SP1+, 2000 SP3). Following the header, we see the color coded conversation between the source (SRC) and destination (DST) IPs involved. First we see our SRC make a HTTP POST request to ishi-bati . com/kartos/youyou.php followed by the server’s response. Then our SRC issues a GET to ishi-bati . com/kartos/krt.exe. The DST then replies with file content-type that is application/x-msdownload, indicating a DOS executable. (the string "This program cannot be run in DOS mode" is also a giveaway.)

In a matter of seconds, we know a client downloaded krt.exe from a suspect domain. Want to know more about krt.exe? Right-click on the "ET INFO Packed Executable Download" event Alert ID field again, but this time choose "Network Miner." Sguil will load the packet captures we just reviewed in Network Miner, which is useful for getting details on the hosts involved and, in this case, for it's ability to extract files and certificates from packet captures. When the packet capture loads, you'll notice a "Files" tab. You might have to expand the window and column names a bit. In the "Filename" column you'll see "krt.exe.x-msdownload." Right-click the file and choose "Open folder" and you now have the file that triggered the alert. From here you can scan a copy of the file with whatever endpoint security product you use to see if it might have been detected or submit the file or check it's MD5 hash with VirusTotal to identify what security vendors might be detecting it. You could also use a CIF (Collective Intelligence Framework) server to check the MD5 against known threats based on community intel. Or you might look into Cuckoo Sandbox or Razorback as an option for local automated analysis.

The choice is yours but the options are available. Lack of financial support for network security monitoring is no longer excusable in information security. The tools are there, and in most cases are easier to support and maintain than vendor-based solutions, and the capabilities rival or surpass best-of-breed commercial solutions. If you care about security, take what I've written, practice it and learn the process, then show someone else. There's a community working very hard to make this type of monitoring possible for such a small expense as time and hardware on your part. But there's also a community out there that needs your help and needs you to take these tools that have been harmoniously woven together with a most complex thread and learn how they work. Then learn to use them and teach others. I call it "crash, burn and learn" but it's what it's all about. Try hard to learn and improve and advance and share what Doug Burks and the Security Onion community is offering. You'll fail, but you will also have a dedicated and faithful group of people committed to Security Onion helping you succeed.

Getting Started

Update/Upgrade

Analyst Tools

Network Visibility

Host Visibility

Elastic Stack

Customizing for your network

Tuning

Tricks and Tips

Help

Issues

Other

Scripts

Clone this wiki locally