NGWAF Traffic Visualization Dashboard

Overview :

This project visualizes NGWAF request traffic using timeseries data and request logs in Elasticsearch and Kibana.

It helps you:

• Understand traffic trends (total / blocked / attack)

• Analyze attack patterns (OWASP, scanners, anomalies)

• Investigate specific IPs, URLs, and payloads

• Correlate attack signals with block rules

Dashboard Highlights

Traffic Trends

• Requests_total/Requests_total_blocked/requests_attack

• OWASP Injection Attacks

• Scanners

Anomaly Detection

• Traffic Source Anomalies

• IPs flagged

• Request Anomalies

• Response Anomalies

• Bot Detection

• Authentication

Geo & Blocking Insights

• Geo distribution of blocked traffic

Block by block rule

• Block rule breakdown

Attack Analysis

• Attack signal breakdown
• CVE & payload analysis
• Attack rate vs block rate

Anomaly signal insight

Advanced Insights

• Anomaly signal analysis
• 4xx / 5xx correlation
• Sigsci-IP / SANS / ATO

Anomaly signal breakdown and 4xx/5xx insight

Bot Management

Investigation (Drill-down)

Filter by IP / URL:

• Identify attack signals per IP
• See which block rule triggered
• Detect suspicious sources (e.g., Azure, bot traffic)

You can inspect raw logs and export CSV and use a GenAI:

Quick Start

1. Install dependencies

pip install -r requirements.txt

2. Start Elasticsearch & Kibana

brew install colima
brew install docker-compose

colima start --memory 6 --cpu 4
docker context use colima

docker-compose up -d

---

3. Export NGWAF logs

python3 export_waf_data.py --customerid $customerid$ --site $SITE$ --start YYYY-MM-DDTHH:MM:SS --end YYYY-MM-DDTHH:MM:SS --query_block '' --query_allow '' --apitoken $apitoken$

You can filter requests using query_block/query_allow. (e.g., 4xx, 5xx).

4. Ingest into Elasticsearch

python3 sigsci_ingestion_pipeline.py --customerid $customerid$ --site $SITE$ --f 20260328 --t 20260329

5. Import Kibana Dashboard

python3 import_kibana_dashboard.py --customerid $customerid$ --site $SITE$

Prerequisites

1. Install rclone (for uploading NGWAF logs to Google Drive)

brew install rclone
rclone config

Configuration example:

name> gdrive
Storage> drive
client_id> (press Enter)
client_secret> (press Enter)
scope> 1  (Full access)
root_folder_id> (press Enter)
service_account_file> (press Enter)
Edit advanced config? n
Use auto config? y

2. Start Elasticsearch & Kibana

brew install colima docker-compose

colima start --memory 6 --cpu 4 docker context use colima

docker-compose up -d