Python SIEM

A lightweight Security Information and Event Management tool you run from the command line. It parses log files, runs detection rules against them, stores alerts in MySQL, and optionally cross-references every IP it sees against your IOC database.

It works without MySQL too — if you just want to run it against a log file and see what triggers, it will print everything to the terminal without needing any database setup.

---

What it actually does

You point it at a log file. It reads every line, figures out what kind of log it is automatically, parses each line into a structured event, and then runs a set of detection rules across all the events. If anything suspicious is found, it creates an alert. Alerts get saved to MySQL so you can query them, filter them, and export them later.

If you also have the IOC database tool set up, the SIEM will automatically check every IP address it sees in your logs against your IOC database and fire a critical alert if there is a match.

---

Detection rules

Rule	Severity	What triggers it
SSH Brute Force	High	5 or more failed SSH logins from the same IP
SSH Login Outside Business Hours	Medium	Successful SSH login before 7am or after 10pm
Web Scanner / 404 Flood	Medium	20 or more HTTP 404 responses to the same IP
Port Scan Detected	High	Same IP hitting 10 or more unique ports in firewall logs
Windows New User Created	High	Windows Event ID 4720
Windows New Service Installed	Critical	Windows Event ID 7045
Windows Brute Force	High	5 or more Event ID 4625 failed logins from same IP
Internal IP in Web Log	Low	Private RFC1918 IP appearing in a web access log
IOC Database Match	Critical	Source IP matches an active entry in your IOC database

You can tune the thresholds at the top of siem.py.

---

Supported log formats

The SIEM auto-detects the log format — you do not need to tell it what type of file it is.

• SSH auth logs (Linux /var/log/auth.log or macOS)
• Apache and Nginx access logs (combined log format)
• Firewall / iptables logs
• Windows Event Log text exports

---

Requirements

• Python 3.6+
• MySQL (optional but recommended)
• mysql-connector-python (optional, only needed if using MySQL)

---

Installation

1. Install MySQL if you have not already

brew install mysql
brew services start mysql
mysql_secure_installation

2. Install the Python connector

pip3 install mysql-connector-python

3. Set your database password

export DB_PASSWORD='your_mysql_password'

To make it permanent:

echo "export DB_PASSWORD='your_mysql_password'" >> ~/.zprofile
source ~/.zprofile

4. Create the SIEM database

python3 siem.py setup

---

Usage

Ingest a log file

python3 siem.py ingest --log /var/log/auth.log

python3 siem.py ingest --log /var/log/nginx/access.log

Ingest an entire directory of logs

python3 siem.py ingest --dir /var/log

View alerts

# all alerts
python3 siem.py alerts

# filter by severity
python3 siem.py alerts --severity high
python3 siem.py alerts --severity critical

# filter by status
python3 siem.py alerts --status new

# filter by rule name
python3 siem.py alerts --rule "Brute Force"

Terminal dashboard

python3 siem.py dashboard

Shows total alert counts by severity, top offending IPs, most triggered rules, and recent high/critical alerts.

List all detection rules

python3 siem.py rules

Export alerts to CSV

python3 siem.py export

python3 siem.py export --output my_report.csv

---

Using without MySQL

If you just want to run it against a log file without any database setup, it will still work. It parses the log, runs all the rules, and prints any alerts to the terminal. Nothing gets saved, but it is useful for quick ad-hoc analysis.

python3 siem.py ingest --log suspicious.log

---

IOC Database integration

If you have the IOC database tool set up on the same machine, the SIEM will automatically connect to it during ingestion and check every source IP against your known-bad IOC list. Any match fires a critical alert.

No configuration needed — it checks for the database automatically and skips the check silently if it is not available.

---

Tuning detection thresholds

Open siem.py and find this block near the top:

BRUTE_FORCE_THRESHOLD   = 5
BRUTE_FORCE_WINDOW      = 60
PORT_SCAN_THRESHOLD     = 10
WEB_SCAN_THRESHOLD      = 20
WEB_SCAN_WINDOW         = 60

Change the numbers to make rules more or less sensitive.

---

How the three tools fit together

Log files
    |
    v
siem.py ingest        <- parses logs, runs rules, fires alerts
    |
    +-- checks IOC database (ioc_db.py) for matching IPs
    |
    v
MySQL alerts table    <- stores everything
    |
    v
siem.py dashboard     <- terminal overview
siem.py export        <- CSV report for sharing

If you find a malicious IP through the SIEM, you can add it to the IOC database so future log ingestions will automatically flag it:

python3 ../ioc-database/ioc_db.py add --ioc 185.220.101.45 --type ip --source "siem brute force alert" --confidence high

---

Pushing to GitHub

git init
git add siem.py README.md .gitignore
git commit -m "Initial commit: Python SIEM"
git branch -M main
git remote add origin https://github.com/YOUR_USERNAME/python-siem.git
git push -u origin main

---

Suggested GitHub description

Lightweight command-line SIEM that parses SSH, web, firewall, and Windows logs, runs detection rules, stores alerts in MySQL, and integrates with a local IOC database.

---

License

MIT