Splunk Defensive Solution

Description

This project involves setting up and utilizing Splunk as a security monitoring solution to investigate cyberattacks. The objective will be to analyze logs from Windows and Apache servers, detect suspicious activities, and assess the effectiveness of security measures. Tasks include log ingestion, report generation, alert configuration, dashboard creation, and incident analysis to identify threats and anomalies.

Key Components

• Splunk for log analysis, reporting, and alert configuration
• Apache Server Logs and Windows Logs for investigating web and system-based attacks
• Linux Server for hosting the environment and analyzing logs
• SIEM System for threat detection and optimization

Skills Gained and Exercised

• Log Analysis and Threat Detection

  • Learned to ingest, search, and analyze logs using Splunk to detect security incidents, unauthorized access, and system anomalies.

• Security Monitoring and Incident Response

  • Developed skills in monitoring critical systems, identifying attack patterns, and responding to security breaches by analyzing attack logs.

• Report Generation and Data Interpretation

  • Created and analyzed Splunk reports to understand security trends, detect anomalies, and assess the severity of threats.

• Alert Configuration and Tuning

  • Configured Splunk alerts to detect suspicious login attempts, failed access activities, and privilege escalation attempts.

• Dashboard Creation and Visualization

  • Built interactive dashboards to visualize attack data, monitor critical security metrics, and enhance decision-making for threat mitigation.

• Account Security and Suspicious Activity Detection

  • Monitored logs for unusual user behavior, such as failed logins and unauthorized access, to detect potential account compromises.

• HTTP Traffic and Web Server Attack Analysis

  • Investigated Windows and Apache attack logs, analyzed HTTP methods, referrer domains, and response codes to detect potential web-based attacks.

• SIEM Configuration and Optimization

  • Gained hands-on experience in configuring a Security Information and Event Management (SIEM) solution, optimizing search queries, and ensuring accurate threat detection.

---

Splunk Defensive Solution: Normal Activity

Part 1: Loaded and Analyzed Windows Logs

• Uploaded Windows security logs: Used the "Add Data" option in Splunk to upload the windows_server_logs.csv file from the /splunk/logs/Week-2-Day-3-Logs/ directory.
• Configured data input: Set the host name to “Windows_server_logs” and successfully submitted the data.
• Analyzed logs: Briefly reviewed logs and fields, focusing on the following:
  • signature_id
  • signature
  • user
  • status
  • severity

Part 2: Created Reports, Alerts, and Dashboards for Windows Logs

Reports

• Signature Report: Created a report with a table of signatures and associated signature IDs to identify specific Windows activity.
  • Removed duplicate values in the SPL search for accurate results.
• Severity Report: Designed a report to display severity levels with counts and percentages of each.
• Success/Failure Report: Developed a report comparing successful and failed Windows activities using the "status" field.

Alerts

• Failed Activity Alert: Established a baseline and threshold for hourly failed Windows activity, triggering an alert when exceeded. This alert is configured to send an email to SOC@VSI-company.com.
• Successful Login Alert: Defined a baseline and threshold for the hourly count of the signature “an account was successfully logged on,” creating an alert based on signature ID.
• Account Deletion Alert: Set up an alert for the hourly count of the signature “a user account was deleted,” based on the signature ID.

Visualizations and Dashboards

• Line Chart for Signatures: Created a line chart displaying different “signature” field values over time (timechart span=1h count by signature).
• Line Chart for Users: Developed a line chart displaying different “user” field values over time.
• Signature Count Visualization: Added a visualization showing the count of different signatures.
• User Count Visualization: Created a visualization illustrating the count of different users.
• Single-Value Visualization: Designed a custom visualization (e.g., radial gauge or marker gauge) analyzing a single data point.

All visualizations were added to the Windows Server Monitoring dashboard with the ability to change the time range. Panels were appropriately titled and organized.

Part 3: Loaded and Analyzed Apache Logs

• Uploaded Apache logs: Used the "Add Data" option in Splunk to upload the apache_logs.txt file from the /splunk/logs/Week-2-Day-3-Logs/ directory.
• Configured data input: Set the host name to “Apache_logs” and submitted the data.
• Analyzed logs: Reviewed logs and fields, focusing on:
  • method
  • referer_domain
  • status
  • clientip
  • useragent

Part 4: Created Reports, Alerts, and Dashboards for Apache Logs

Reports

• HTTP Methods Report: Created a report displaying a table of different HTTP methods (GET, POST, HEAD, etc.).
• Top Referrers Report: Designed a report to show the top 10 domains referring to VSI’s website.
• HTTP Response Codes Report: Developed a report showing the count of each HTTP response code.

Alerts

• Non-US Activity Alert: Established a baseline and threshold for hourly activity from countries other than the United States. Configured the alert to trigger when the threshold is exceeded, sending an email to SOC@VSI-company.com.
• HTTP POST Method Alert: Set a baseline and threshold for the hourly count of HTTP POST method activity and configured the alert accordingly.

Visualizations and Dashboards

• Line Chart for HTTP Methods: Created a line chart displaying different HTTP “methods” field values over time (timechart span=1h count by method).
• Geographical Map for Client IPs: Developed a geographical map visualizing the location based on the “clientip” field.
• URI Count Visualization: Added a visualization displaying the count of different URIs.
• Top Countries Visualization: Designed a visualization illustrating the top 10 countries from the log data.
• User Agents Visualization: Created a visualization showing the count of different user agents.

These visualizations were added to the Apache Web Server Monitoring dashboard, with the ability to change the time range for each panel. Panels were titled and organized.

Part 5: Installed an Add-On Splunk Application for Additional Monitoring

• Chosen Add-On App: Selected the Splunk Add-On for Apache Servers for additional security monitoring of VSI’s systems.
• Installed and Configured Add-On: Installed the add-on app and described how its features would help protect VSI.
• Scenario: Provided a use case showing how the add-on app helps with security monitoring.

---

Splunk Defensive Solution: Attack Activity

Part 1: Analyze Windows Attack Logs

Task 1: Report Analysis for Severity

Findings

• There was a signficant increase in high severity events during the attack (329 to 1111).

Task 2: Report Analysis for Failed Activities

Findings

• There was a notable increase in success events during the attack (4622 to 5856).
• There was a slight decrease in failure events during the attack (142 to 93)

Task 3: Alert Analysis for Failed Windows Activity

Findings

• There was a spike in failure events during the attack, specifically at 8:00 am. The alert for this actvity would have been triggered as the threshold was set to >11 events.

Task 4: Alert Analysis for Successful Logins

Findings

• There was no evidence of an unusual amount of successful logins based on signature ID. The threshold for this alert was set to >20 events.

Task 5: Alert Analysis for Deleted Accounts

Findings

• There was no evidence of an unusual amount of deleted accounts. The threshold for this alert was set to >22 events.

Windows Attack Dashboard

Task 7: Dashboard Analysis for Time Chart of Signatures

Findings

• Notable signatures:
  • A user account was locked out | Timeframe: 1:00 am - 2:00 am | Peak Count: 896
  • An attempt was made to reset an accounts password | Timeframe: 9:00 am - 10:00 am | Peak Count: 1258
  • An account was successfully logged on | Timeframe: 11:00 am | Peak Count: 196
• The signature ID for Windows Successful Login did not match the label for An account was successfully logged on in the dataset and therefore did not trigger an alert.

Task 8: Dashboard Analysis for Users

Findings

• Notable users:
  • user_a | Timeframe: 1:00 am - 2:00 am | Peak Count: 984
  • user_k | Timeframe: 9:00 am - 10:00 am | Peak Count: 1256
  • user_j | Timeframe: 11:00 am | Peak Count: 196

Task 9: Dashboard Analysis for Signatures with Bar, Graph, and Pie Charts

Findings

• The piechart breakdown of the signature count aligns with the findings from the time chart data for signature activity.

Task 10: Dashboard Analysis for Users with Bar, Graph, and Pie Charts

Findings

• The piechart breakdown of the user activity aligns with the findings from the time chart data for user activity.

Task 11: Dashboard Analysis for Users with Statistical Charts: Pros & Cons

Pros

• Detailed View – Shows exact numbers and timestamps for user activity.
• Easy Filtering – You can sort and filter data for deeper analysis.
• Long-Term Tracking – Helps monitor trends over time.
• Consistent Data – Uses signature IDs, avoiding issues with Windows updates.

Cons

• Harder to Read – A table isn’t as easy to understand as a chart.
• No Visual Trends – Doesn’t highlight patterns at a glance.
• More Manual Work – Requires filtering and analysis to find insights.

Comparison to Other Panels

• Charts make it easier to spot trends and spikes quickly.
• Pie/Bar Graphs show comparisons at a glance without manual sorting.
• Single-Value Metrics provide instant key stats like failed logins.

---

Splunk Defensive Solution: Attack Activity

Part 2: Analyze Apache Attack Logs

Task 1: Report Analysis for Methods

Findings

• There was a significant increase in POST HTTP methods during the attack timeframe (106 to 1324).

Task 2: Report Analysis for Referrer Domains

Findings

• There was a significant deacrease in the referring domain count during the attack timeframe.
• Top 5 notable changes:
  • http://www.semicomplete.com (3038 to 764)
  • http://semicomplete.com (2001 to 572)
  • http://www.google.com (123 to 37)
  • https://www.google.com (105 to 25)
  • http://stackoverflow.com (34 to 15)

Task 3: Report Analysis for HTTP Response Codes

Findings

• There was a significant increase in the 404 HTTP response code count during the attack timeframe (213 to 679).

Task 4: Alert Analysis for International Activity

Findings

• There were significant spikes in international activity at 6:00 pm (730) and at 8:00 pm (1415).
• The alert for this activity would have been triggered as the threshold was set to >137 events.

Task 5: Alert Analysis for HTTP POST Activity

Findings

• There was a significant spike in HTTP POST activity at 8:00 pm (1296).
• The alert for this activity would have been triggered as the threshold was set to >4 events.

Apache Attack Dashboard

Task 6: Dashboard Analysis for Time Chart of HTTP Methods

Findings

• There was a significant spike in HTTP GET activity at 6:00 pm (729).
• There was a significant spike in HTTP POST activity at 8:00 pm (1296).

Task 7: Dashboard Analysis for Cluster Map

Findings

• There was a large amount of activity in Ukraine during the attack timeframe. This country does not have a history of high web traffic to the VSI servers and is considered to be abnormal.
• The total number of activities from Ukraine was 432.
• Additional research illustrates that the web traffic originated from Kharkiv, Ukraine.

Task 8: Dashboard Analysis for URI Data

Findings

• The most targeted URI was /VSI_Account_logon.php. Additionally the URI /files/logstash/logstash-1.3.2-monolithic.jar had a notably higher count relative to the rest of the most frequently accessed URIs.

---

VSI Web Server Attack Summary

VSI’s web server was hit by a coordinated cyber attack involving brute-force login attempts, reconnaissance scanning, and possible exploitation efforts. A sharp increase in HTTP POST requests, particularly targeting /VSI_Account_logon.php, pointed to a credential stuffing or brute-force attack. Additionally, an unusual spike in traffic from Ukraine suggested the involvement of a foreign threat actor or botnet. The attack also triggered a surge in 404 (Not Found) response codes, indicating that attackers were actively probing for restricted directories and vulnerabilities. The total volume of HTTP requests exceeded normal thresholds, raising concerns about a DDoS or automated high-volume attack. Furthermore, attackers appeared to focus on sensitive URIs and files related to logins and system configurations, potentially aiming for data exfiltration or privilege escalation. Notably, user_a, user_k, and user_j exhibited significant unusual activity, suggesting they were key targets during the attack.

VSI Future Mitigation Breakdown

To protect VSI from future attacks, several mitigation strategies should be implemented. Strengthening authentication and access controls is crucial—this includes enforcing multi-factor authentication (MFA), increasing password complexity requirements, and limiting login attempts per IP address to prevent credential stuffing and brute-force attacks. Enhancing web application security is also essential, which can be achieved by deploying a Web Application Firewall (WAF) to filter malicious requests and restricting access to sensitive files and endpoints to prevent unauthorized access. Given that the attack involved high-volume traffic from Ukraine, geo-based security policies should be enforced, such as enabling geo-blocking for high-risk countries and monitoring international traffic patterns for anomalies.

To detect and respond to threats in real-time, VSI should improve logging and monitoring by enabling real-time log analysis and deploying an Intrusion Detection System (IDS) to identify suspicious activity. Since the attack involved DDoS-like traffic spikes, mitigation strategies should include leveraging Content Delivery Networks (CDN) to absorb high-volume traffic and using anomaly-based traffic filtering to detect and block unusual request patterns. Additionally, regular security audits and penetration testing should be conducted to proactively identify vulnerabilities, simulate attacks on VSI’s infrastructure, and ensure employees are trained in phishing and social engineering awareness to reduce human-related security risks. Implementing these measures will help VSI harden its security posture and prevent similar attacks in the future.