Shadowserver Vulnerability Monitoring Scraper

Scrapes the Shadowserver vulnerability monitoring dashboard and exports the table data to a CSV file.

The page is dynamically rendered via JavaScript, so the script uses a headless Chromium browser (Playwright) to load the full page before extracting the data.

---

Quick Start

# 1. Install Python dependencies
pip install playwright beautifulsoup4

# 2. Install the Chromium browser used by Playwright
python -m playwright install chromium

# 3. Run the scraper (output file generated in the current directory)
python scrape_monitoring.py

---

Requirements

• Python 3.9+
• playwright — headless browser automation
• beautifulsoup4 — HTML parsing

Install all at once:

pip install playwright beautifulsoup4
python -m playwright install chromium

---

Output

The generated CSV contains the following columns, all values quoted and ASCII-only:

Column	Description
Vulnerability	CVE or vulnerability identifier (e.g. CVE-2017-17215)
Vendor	Affected vendor name
Product	Affected product name
Details	Fixed URL pointing to the source dashboard page

Example rows:

"Vulnerability","Vendor","Product","Details"
"CVE-2017-17215","Huawei","Huawei Home Gateway HG532","https://dashboard.shadowserver.org/statistics/honeypot/vulnerability/monitoring/"
"CVE-2014-8361","Realtek","Realtek SDK","https://dashboard.shadowserver.org/statistics/honeypot/vulnerability/monitoring/"

---

Usage Examples

Generate output with default filename (shadowserver_vulnerabilities_YYYYMMDD_HHMMSS.csv) in the current directory:

python scrape_monitoring.py

Specify a custom output path:

python scrape_monitoring.py C:\reports\vulnerabilities.csv

python scrape_monitoring.py /tmp/vulns.csv

Terminal output during execution:

[START] 2026-05-26 18:24:10
[INFO]  Source : https://dashboard.shadowserver.org/statistics/honeypot/vulnerability/monitoring/
[INFO]  Output : C:\reports\vulnerabilities.csv

[INFO] Launching headless browser...
[INFO] Navigating to: https://dashboard.shadowserver.org/...
[INFO] Waiting for table data to load...
[INFO] Table loaded in 7.7s

[INFO] Found 100 rows in table

[INFO] CSV written: C:\reports\vulnerabilities.csv
[INFO] Total records: 100

[DONE] 2026-05-26 18:24:18

---

Notes

• The dashboard defaults to showing the top 100 vulnerabilities. This limit is controlled by the filter on the source page and is not configurable via script arguments.
• On first run, if no filter state is stored in the browser's localStorage, the script automatically injects today's date into the filter and clicks Update to force the table to load.
• All text values are normalized to ASCII (e.g. accented characters are transliterated) to prevent encoding issues downstream.

---

License

This project has no license. Use it at your own risk.