Log Sources

Log Sources

A log source is one access log plus the format to parse it with. secwatch can watch a single log (the classic setup) or many at once — your edge reverse proxy and internal apps that don't sit behind it. All sources feed one detection engine, so bans, endpoint rules, and crowd intel apply across every app.

Supported formats

type	Format
traefik	Traefik JSON access log
nginx	nginx combined log
caddy	Caddy JSON log
regex	any format, via a custom pattern with named groups

For regex, supply a pattern with at least an ip named group; host, method, path, status, and ua are used if present:

log_source:
  type: regex
  regex: '(?P<ip>\S+) - "(?P<method>\S+) (?P<path>\S+)[^"]*" (?P<status>\d+) "(?P<ua>[^"]*)"'

Single source (classic)

paths:
  access_log: /var/log/traefik/access.json.log
log_source:
  type: traefik

Multiple sources

Set log_sources to a list. When present it takes precedence over the single log_source/paths.access_log. Each entry needs a name, path, and type (plus regex for the regex type):

log_sources:
  - { name: edge,     path: /var/log/traefik/access.json.log, type: traefik }
  - { name: gitea,    path: /var/log/nginx/gitea.access.log,  type: nginx }
  - { name: internal, path: /var/log/myapp/access.log,        type: regex,
      regex: '(?P<ip>\S+) - "(?P<method>\S+) (?P<path>\S+)[^"]*" (?P<status>\d+) "(?P<ua>[^"]*)"' }

Why watch internal apps? An app that isn't exposed to the internet can still be attacked — by a compromised device on your LAN, a malicious container, or an SSRF pivot. Watching its log means the same probe/scan/brute-force detection and auto-ban logic covers it too.

Add sources from the dashboard (no YAML, no restart)

The dashboard has a Log sources card. It lists every watched source with a live/idle indicator, line count, and last-activity time.

• + Add local app — enter a name, the log file path, and the format. secwatch validates the path (must be an existing, readable file), starts tailing it immediately, and remembers it across restarts.
• remove — stops watching a dashboard-added source right away.

YAML-defined and primary sources are shown but are read-only from the UI — edit secwatch.yaml to change those.

Dashboard-added sources are stored in data/log_sources.json and merged on top of whatever is in your YAML.

Auto-discover: "Scan for logs"

Click Scan for logs and secwatch walks common log locations (/var/log/nginx, /var/log/caddy, /var/log/apache2, /var/log/traefik, /srv/*/logs, /opt/*/logs, …), sniffs each file's format by parsing a sample of its lines, and returns the ones you aren't already watching as a review queue — each with a suggested name, detected format, and a sample line.

• Click Add on the ones you want (or Add all).
• Nothing is added automatically — you review first. Files that don't parse as a known access-log format, rotated/compressed copies, and empty files are skipped.

How resume works

secwatch remembers how far it has read in each log (by inode + offset), so a restart continues where it left off rather than replaying or skipping traffic. Log rotation is detected and handled automatically.