nwatch

Simple network monitoring tool using Elasticsearch, nmap and Python

Grafana visualization of data generated by nwatch

Dependencies and requirements

This tool required nmap installed and available via calling 'nmap' in shell. Access to an elasticsearch database is required to store parsed data outside JSON files. Elastic upload script requires the python module 'elasticsearch' which can be installed with pip.

Usage

The repo contains two scripts and json file for mapping an elasticsearch index for storage. Before uploading anything to the database it is important you first create the index with the right mapping. This can for example be done with:

curl -XPOST "http://<db-ip>:<db-port>/<index>" --data-binary "@mapping.json"

Once the index is created you will want to produce a parsed nmap log by running nmap.py. This script will run nmap, parse the result, and output result to stdout as a JSON object.

This example will search the subnet 192.168.1.0/24 with the nmap option -F and output the result to the file log.json

sudo python nmap.py --nmap-option F --search-address 192.168.1.0 --search-mask 24 > log.json

Note: Running the nmap script without root means you will not be able to retrieve MAC addresses!

When you have a parsed log file you can upload it to a elasticsearch database with elastic_upload.py, this script will read a JSON object from stdin and push the content to an elasticsearch database of your choosing.

This example will push the data from the previous example to a local elasticsearch database on index 'nwatch':

python elastic_upload.py --address localhost --port 9200 --index nwatch < log.json

You can look up all of the script arguments with python <scriptname> --help

Automation of script running can be done with for example adding it as a cronjob. All scripts will exit with status code 1 on failure and 0 on success.

Index structure

The index generated has one type called 'entry' consisting of the following fields:

• 'ip': The IP address of the host
• 'mac': The MAC address of the host
• 'timestamp': ISO formatted date the data was retrieved
• 'mac_provider': The name of the organization assigned the MAC address as discovered by nmap
• 'open_ports': An array of ports that were open on the host at time of analysis