Parsers

Parsers

A parser tells SIEMLite how to read one source's lines: where the time, IPs, user and severity are, and which other values to keep. Choose how a source's lines are read on the Sources page:

• Automatic (the default): SIEMLite detects syslog and JSON, addresses, users and severity, as described under Log parsing.
• None: no detection. Each line is stored as it is, with the arrival time and Informational severity (parser_none in the API).
• A parser you built, as below.

Build one on the Parsers page: paste a few sample lines, describe them, and the preview shows how each line is read.

Or let SIEMLite build it: paste the samples and choose Draft a parser. It needs no AI model. SIEMLite works out the format (JSON, key=value, syslog or free text) and, for JSON and key=value lines, which keys hold what, from both their names and their values: timestamps become the time, IP addresses the source or destination (by names like dest_), detail or description the message, username or actor the user, a level field the severity. Without a level field, an action, outcome or status field decides it, by keywords: breach, attack and malware critical; disable, revok, unauthori, privilege, admin, grant and critical high; delete, remov, denied, deny, refus, forbidden, reset, password, role, permission, reject, lock, block and alert medium; error, fail and invalid low; the rest informational. A value with several keywords gets the most severe one's level. For HTTP status codes, 5xx is medium and 4xx low. When you choose a severity from a field's value, Reset next to Add rule puts these keyword rules back. Free text gets a pattern that matches every sample. Check the preview and the field mapping, adjust anything, and save. With AI Parsers on, Suggest with AI starts from the same draft and asks the model to improve it.

JSON can be pasted as it comes: pretty-printed objects, or an array of them, become one object per line.

• Text with a fixed layout: write one line out with {name} where values change, e.g. {src_ip} - {user} [{time}] "{method} {path} {protocol}" {status}. Spaces match any run of spaces, {} skips text, and {name:regex} sets exactly what to match ({status:\d+}).
• JSON objects and key=value pairs: every field is picked up; nested JSON keys read as client.ip.
• Fields: values named like time, src_ip, dst_port, user, host, message or level fill the event automatically; map any other name in the Fields table. Everything else is kept as an extra field, shown in the event's details.
• Syslog: tick "Lines start with a syslog header" and the pattern only needs to describe the message.
• Severity: detected from the line, always the same, or decided by rules on a field (e.g. status ^5 -> High).
• Templates for web server access logs, JSON application logs and key=value firewall logs.
• Upload and export parsers as small JSON files to share them or keep them in version control.
• A line that doesn't match its source's parser is still stored, parsed as Automatic would, with a parse_error field.

AI Parsers

With the AI Parsers integration on, the parser editor gets a Suggest a parser button. SIEMLite removes syslog headers and recognises JSON and key=value lines itself, drafts a pattern that matches every sample line, and asks the model to name the fields and pick severity and category. It keeps the model's answer only when it still reads every sample line.

Set it up on the Integrations page, choosing where the model runs:

• On this server, with Ollama. Nothing leaves your network. Choose a model size: the card shows this server's CPU cores and memory, recommends the largest model that will answer in reasonable time on them, and marks models that are too big. The chosen model downloads in the background (1 to 9 GB, once), with its progress on the card. With Docker, start Ollama by adding ai to COMPOSE_PROFILES in .env and running docker compose up -d; its address is http://ollama:11434. Without Docker, run Ollama yourself (http://localhost:11434). On a CPU, the medium model (qwen2.5-coder:3b) takes about 10-40 seconds per suggestion. Any Ollama model that can answer in a JSON shape can be used.

• In the cloud, with your API key for one of these services:

  • OpenRouter (keys at openrouter.ai/keys): hundreds of models from many companies, with prices shown. SIEMLite asks OpenRouter to use only companies that don't keep or train on what they're sent.
  • Anthropic (keys at console.anthropic.com): Claude.
  • Google Gemini (keys at aistudio.google.com). With a free key, Google may use what is sent to improve its products; a key on a paid (billing) project doesn't.
  • OpenAI (keys at platform.openai.com).
  • NVIDIA (keys at build.nvidia.com): NVIDIA's hosted open models, such as Llama and Nemotron.

  Choose the service and a model: the card reads the service's own model list with your key and recommends small, fast and inexpensive ones. Each service keeps its own key and model, so you can switch between them without entering them again. The sample lines you ask about are sent to the service, so:

  • IP, email and MAC addresses and host names are replaced before sending (on unless you turn it off). Every dotted name counts as a host name, including file and field names such as index.html or actor.ip, so nothing slips through on an unusual domain; this can make field mapping for JSON with dotted field names less precise. The stand-ins keep the lines' shape, and the answer is changed back, so the parser fits the real lines. Other text, such as user names, is sent as written.
  • Before anything is sent, SIEMLite checks the exact text: if an address, or part of one, is still there, nothing is sent and you are asked to remove it from the samples or use a model on this server.
  • The keys are encrypted like the email password and never shown again.

Turning the integration off hides the button; its settings are kept. Asking for a suggestion is recorded in the audit log (parser.suggest), and so are changes to the settings (system.ai.save). Before v0.19 this was set with SIEMLITE_AI_URL; that still works, as the starting settings, until the page saves its own.

Automatic parsing

/api/v1/logs converts each line into an OCSF event: syslog (RFC 3164 and 5424) headers give the time and hostname, the first two IPv4 addresses become source and destination, a user is picked up from patterns like for user alice, and severity comes from the syslog priority or keywords (error, failed, warning, ...). JSON objects that already contain category_uid are stored as OCSF; other JSON uses its message, level and timestamp fields. The detection is heuristic; use ?severity= to override it, or give the source a parser. The message stored with an event is the text after the syslog header (the whole line when there isn't one). A line that only starts with a timestamp, such as 2026-10-04T13:42:07Z ERROR payment failed, gets its time from it but no hostname.

This is the Automatic setting of a source. None skips detection: each line is stored as it is, with the line as its message, the time it arrived, Informational severity and no addresses, user or host. ?source= and ?severity= still apply, and threat intel still checks the line.