03 Architecture

Architecture

Two paths that never block each other: the ban path is local and instant; the report path is scheduled and tolerant.

---

Diagram

flowchart LR
    A[fail2ban jail<br/>any filter] -->|actionban| B[nftables ban<br/>instant]
    A -->|actionban| C[abuseipdb-async<br/>enqueue]
    C --> D[(SQLite queue<br/>events + reports)]
    E[systemd timer / cron<br/>16 min] --> F[abuseipdb-report worker]
    D --> F
    F -->|read evidence| G[(fail2ban.sqlite3<br/>bips.data.matches)]
    F -->|POST /api/v2/report| H[AbuseIPDB API]
    F -->|retry / journal| D
    I[/etc/abuseipdb/.abuseipdb.key<br/>mode 600/] --> F
    J[updater.json<br/>GitHub] --> K[abuseipdb-upgrade] -->|install.sh| A

Loading

Components

Component	Responsibility
fail2ban/action.d/abuseipdb-async.conf	action norestored = 1; calls the queue helper with <ip>, <name>, <bantime>, <failures> only
bin/abuseipdb-enqueue	validates the address, applies ignore rules, writes one event (~0.1 s)
src/abuseipdb_reporter/db.py	SQLite schema, dedupe, state transitions, retention
src/abuseipdb_reporter/evidence.py	reads bips.data.matches from fail2ban's DB; classifies lines
src/abuseipdb_reporter/comments.py	builds the PII-free comment, categories, timestamp, attribution
src/abuseipdb_reporter/api.py	urllib client (no dependencies)
src/abuseipdb_reporter/reporter.py	run loop, budgets, throttles, error handling
src/abuseipdb_reporter/keys.py	key resolution, key file, first-run prompt, migration
src/abuseipdb_reporter/updater.py	manifest check, SHA-256 verify, safe extraction, installer call
bin/abuseipdb-report	worker CLI (--stats, --dry-run, --check-key, --setup, --key-status)
bin/abuseipdb-upgrade	self-update CLI (--check, --upgrade, --json, --dry-run)

Ban path

1. Fail2Ban matches a filter and creates a ban.
2. The ordinary firewall action applies the ban (nftables by default in the Debian packaging).
3. The extra abuseipdb-async action runs abuseipdb-enqueue, which:
   • parses the IP with ipaddress;
   • refuses private/loopback/link-local/documentation/reserved ranges (unless explicitly overridden for tests);
   • refuses jails listed in skip_jails (default recidive);
   • inserts (or updates, within a 5-minute dedupe window) one row in events.

No network access, no secrets, no attacker-controlled text on a command line.

Report path

Every 16 minutes (or on manual systemctl start abuseipdb-report):

1. /run lock — one worker at a time.
2. Select pending IPs, oldest first.
3. Skip an IP if its last successful report is younger than min_interval_ip (default 1200 s; AbuseIPDB itself enforces 900 s).
4. Stop when the daily budget (max_reports_per_day, 900) or per-run budget (max_reports_per_run, 25) is exhausted.
5. Read evidence for each jail from fail2ban's database.
6. Build the comment, categories and timestamp.
7. POST /api/v2/report.
8. Interpret the response (table below), journal the attempt, commit.
9. Delete finished events and old report rows older than retention_days.

Local queue

events(id, ip, jail, event_time, bantime, failures, created_at,
       reported, attempts, note)
reports(id, ip, jail, reported_at, status, categories, comment, response)

events.reported: 0 pending · 1 reported · 2 invalid report · 3 gave up.

Dedupe: repeated bans of the same IP+jail while the previous event is still pending (5-minute window) update the row instead of creating dozens of events. A persistent attacker therefore produces one report per throttle window.

Measured behaviour

Operation	Result
enqueue (one event, SQLite WAL)	≈0.1 s (10 calls in 1.06 s, Python start-up included)
worker, empty queue	well under 1 s; ~13 MB RSS
lock fallback	/run → /var/lock → $TMPDIR; if no path is writable the worker runs without the lock (so CI/unprivileged dry-runs do not silently skip)

Evidence, and why it is trustworthy

The comment is built from the log lines that fail2ban already matched. They live in fail2ban's own SQLite database:

SELECT data FROM bips WHERE ip=? AND jail=?;
-- data: {"matches": ["2026-... sshd[...]: Failed password for root ...", ...], "failures": 4}

Properties:

• exactly what triggered the ban — no re-parsing of 100 MB of web logs per run;
• no shell involved: text goes into a Python string, then into a form-encoded API call;
• the reporter strips timestamps and syslog hostnames for readability;
• for SSH/FTP/mail jails only generic patterns are published (root password attempts, invalid usernames tried), never the attacker-chosen usernames;
• for web jails the attacker's own URLs are published (GET /.env -> 404) — useful and safe;
• if fail2ban has no matches (e.g. a manual ban), the comment simply omits the evidence block.

Response policy

Status	Meaning	Action
200	accepted	mark events reported, journal the attempt
401	bad API key	stop the run, log an error; next tick retries after you fix the key
429 “once in 15 minutes”	per-IP limit	leave pending, retry on a later tick
429 other	daily/global limit	stop the run, keep everything pending
400 / 422	invalid report	mark invalid, never retry
5xx / timeout	transient	increment attempts, retry later; after 10 attempts mark “gave up”

Key file and updater

• The key is resolved at runtime by keys.py (CLI → env → key file → legacy config → prompt).
• It is stored in /etc/abuseipdb/.abuseipdb.key, never in reporter.conf, never in the tarball, .gitignore excludes *.key.
• updater.json is the release manifest. updater.py requires a SHA-256 checksum, refuses archives without one, rejects path traversal/symlinks/devices, snapshots the key file before running install.sh and restores it if the install fails.

Failure modes

Failure	Behaviour
AbuseIPDB down	events stay queued; retried on later ticks
API key revoked	401, run stops, queue intact, clear error
daily quota exhausted	run stops until the next day (local midnight of the host)
SQLite locked	WAL + 10 s busy timeout
disk full	enqueue logs an error; the ban still happens
reboot	queue on disk; Persistent=true catches up the missed tick
fail2ban restart	norestored = 1 — restored bans are not re-queued
worker crash	unfinished events stay pending; next tick retries
updater download corrupted	checksum mismatch aborts before extraction

Non-goals

No daemon, no web UI, no score-based blocking, no auto-update in the background, no storage of request bodies or mail content.

---

Next: Configuration Reference · Fail2Ban Fundamentals