DNS Threat Investigation

elopezsa edited this page Sep 21, 2016 · 3 revisions

Purpose and Audience

This section contains a walk-through of the Threat Investigation analyst view. The intended audience is Security Analysts responsible for reviewing the results for potential threats. The Threat Investigation notebook provides a way to perform a more detailed analysis of the connections previously scored as high risk.
Users will select a day to investigate, starting at the Suspicious Connects section to later get to the detailed analysis performed with a Threat Investigation Jupyter notebook.

###Walk-through Access the analyst view for DNS Suspicious Connects. Select the date that you want to review. Your view should now look like this:

The analyst must previously score the suspicious connections before moving into Threat Investigation View, please refer to DNS Suspicious Connects Analyst View walk-through.

Select DNS > Threat Investigation from Open Network Insight Menu.

Threat Investigation Web Page will be opened, loading the embedded Jupyter notebook. A list with all IPs and DNS Names scored as High risk will be presented

####Expanded Search Select any value from the list and press the "Search" button. The system will execute a query to the dns table, looking into the raw data initially collected to find additional activity of the selected IP or DNS Name according to the following criteria:

Expanded Search for a particular Domain Name
The query results will provide the different unique IP Addresses list that have queried this particular Domain, the list will be sorted by the quantity of connections.

Expanded Search for a particular IP
The expanded search will provide the different unique Domains list that this particular IP queried in one day, they will be sorted by the quantity of connections made to each specific Domain Name.

The full output of this query is stored into the threat-dendro-<threat>.csv file, from which the top 'n' results will be extracted and displayed in a table. If an expanded search was previously executed on this IP or Domain, the system will extract the results from the preexisting file to reduce the execution time by avoiding another query to the table. Query execution time is long and will vary depending on whether Hive or Impala is being used, so please monitor the notebooks status icon for completion. The quantity of results displayed on screen can be set by modifying the top_results variable.

####Save comments.
In addition, a web form is displayed under the title of 'Threat summary', where the analyst can enter a Title & Description on the kind of attack/behavior described by the particular IP address that is under investigation.

Clicking the "Save" button, will create/update the threats.csv file, adding a new line with the contents of the form. This file is used at the Storyboard section to display all the comments entered by the user, as well it will serve as a index of the threats analyzed.

####Continue to the Storyboard.
Once you have saved comments on any suspicious IP or domain, you can continue to the Storyboard to check the results.

Input files

ipython/dns/user/<date>/dns_scores.csv

Output files

ipython/dns/user/<date>/threats.csv
ipython/dns/user/<date>/threat-dendro-<threat>.csv

HDFS tables consumed

dns
You can’t perform that action at this time.
You signed in with another tab or window. Reload to refresh your session. You signed out in another tab or window. Reload to refresh your session.
Press h to open a hovercard with more details.