Skip to content

anomaly

Thomas Mangin edited this page Jul 13, 2026 · 1 revision

Pre-Alpha. This page describes behavior that may change.

Ze has a behavioral anomaly detector: a report-only subsystem that builds a per-source behavioral baseline, scores how far each source deviates from its own history and from its peers, and correlates weak signals into a single incident. It is a separate domain from the volumetric DDoS subsystem (which reacts to raw packet and bandwidth rates). Where DDoS asks "is this interface being flooded", anomaly asks "is this source behaving unlike itself and unlike its neighbors".

For the volumetric subsystem, see DDoS detection and mitigation.

Traffic-feature facts layer

Both detectors read from a neutral facts layer. trafficfeature is a second, independent consumer of the observation feed that derives domain-neutral per-source feature signals. These are facts (measurable numbers), never verdicts: the detection plugins apply judgment on top, and this layer never decides "attack".

Feature Meaning
fan-out Count of distinct destinations a source talked to in the window (scanning / spread signal)
out-in-ratio Out-bytes divided by in-bytes for the source (exfiltration signal); a pure sender with no inbound bytes is surfaced as inf
port-entropy Shannon entropy (bits) of the source's destination-port byte distribution; near 0 for a single port, higher for a scan
new-peer The source was first observed within the recent new-peer window (never-seen-before)
rare-port The source's dominant destination port is outside the well-known allowlist
beaconing Interval-regularity score in [0,1] over the source's flow arrival times; higher is more clock-like (C2 beacon signal)

The service is lazy and consumer-refcounted (it costs nothing until a consumer attaches), aggregates on a 1s tick, and bounds memory with per-source cardinality and idle-eviction caps. Operators can read it directly:

show traffic-feature
show traffic-feature 203.0.113.7   // one source

The output lists the top source IPs by activity with their feature vector and a degraded flag when no data has arrived yet.

How detection scores a source

Each tick the detector folds one traffic-feature snapshot into per-source, per-feature baselines and scores every source two ways:

  • Self-deviation. A z-score of the current value above the source's own running mean, in units of standard deviation, from an EWMA baseline. Below-baseline values score zero (only excess is anomalous). Each feature has a standard-deviation floor so a near-constant feature does not produce a divide-by-tiny-variance spike.
  • Cohort rarity. How far the value sits above the source's cohort (other sources in the same source-prefix bucket), computed leave-one-out so an outlier is never compared against a baseline it dominates. Rarity is only scored once the cohort has at least min-cohort-size other members.

A continuous feature fires at the larger of its self-deviation and its cohort rarity when that reaches deviation-threshold. The binary features (new-peer, rare-port) each contribute exactly one unit of evidence, enough to satisfy the correlation gate but never enough to dominate a continuous signal.

Correlation into one incident

A source is anomalous this tick only when at least min-features-to-correlate features fire together. The fired z-scores are then fused into one incident score: the strongest at full weight plus a discounted sum of the rest (corroboration-weight), capped so a benign multi-feature move cannot saturate into a critical incident. The discount prevents correlated, co-moving features from double-counting. The incident carries a graded severity derived from the score.

Confirm / clear lifecycle

An incident is confirmed only after confirm-duration consecutive anomalous ticks, and cleared only after clear-consecutive consecutive normal ticks. This drives an AnomalyDetected / AnomalyOngoing / AnomalyCleared event stream and a bounded recent-incident ring.

Two guards against poisoning and false positives

Freeze-learn. The per-source baseline is updated with the current tick only when the source is not anomalous this tick (or while it is still warming up). A sustained attack therefore cannot drift the source's own baseline upward until it looks normal again, so an attacker cannot slowly train the baseline to accept the attack.

No first-sight false positive. A newly-seen source has no baseline to deviate from. Self-deviation is suppressed until the baseline has accumulated a short warmup of samples: the early values seed the baseline first, so a never-seen source is not flagged against an empty baseline the moment it appears.

Viewing anomaly incidents

show anomaly detect

Reports whether the detector is enabled and lists the recent-incident ring: per incident the entity, its cohort, the fused score and severity, the timestamp, and the fired features with their z-scores.

Shadow-first responder

The anomaly-shape responder can act on confirmed incidents, but it is shadow-first: by default it only logs the action it would take and installs nothing. It becomes live only when explicitly set to armed.

anomaly {
    shape {
        mode shadow;        // default; armed installs live per-source rules
        action limit;       // rate-limit (default) or drop
        limit-rate 1000;
        limit-unit second;
        auto-revert-ttl 300;
        blast-radius-cap 16;
        allowlist [ 192.0.2.0/24 ];
    }
}

When armed, it installs a surgical per-source firewall rule (a rate-limit by default, or a drop) for the anomalous source. Its safety properties:

  • Per-entity arming. Each source is armed independently as a whole owner set; re-arming an already-armed source refreshes its timer without reinstalling or double-counting.
  • Timed auto-revert. Every armed rule carries an auto-revert-ttl. The rule is withdrawn when the TTL elapses regardless of any clear event, and an ongoing incident extends the deadline. A cleared incident withdraws the rule early.
  • Blast-radius cap. At most blast-radius-cap sources are armed at once; beyond the cap, arming is refused and logged rather than expanding without bound.
  • Kill switch. Engaging the kill switch reverts every armed rule and forces the responder back to shadow.
  • Allowlist. Sources in the allowlist are never armed.

Inspect the responder state:

show anomaly shape

It reports the mode, the configured action, whether the kill switch is engaged, the armed count, and the list of armed sources.

Configuration reference

anomaly detect

Parameter Default Range Description
enabled false bool Enable the detector
deviation-threshold 3.0 1.0-100.0 Sigma at or above which a feature fires
min-features-to-correlate 2 1-6 Features that must fire together to form an incident
min-cohort-size 4 2-1024 Minimum other cohort members before rarity is scored
corroboration-weight 0.5 0.0-1.0 Discount applied to corroborating features in the fused score
confirm-duration 3 1-3600 Consecutive anomalous ticks to confirm an incident
clear-consecutive 10 1-100 Consecutive normal ticks to clear an incident
baseline-window 300 10-86400 Baseline horizon in ticks (derives the EWMA alpha)
cohort-prefix-len-v4 24 8-32 Source-prefix bucket length for v4 cohorts
cohort-prefix-len-v6 48 16-64 Source-prefix bucket length for v6 cohorts

anomaly shape

Parameter Default Range Description
mode shadow shadow, armed shadow logs the would-be action; armed installs live rules
action limit limit, drop limit rate-limits the armed source; drop drops it
limit-rate 1000 1+ Rate for the limit action
limit-unit second second, minute, hour, day Rate unit
limit-burst 0 uint Burst allowance
auto-revert-ttl 300 5-3600 s Safety ceiling; a rule is reverted this long after the last signal
blast-radius-cap 16 1-1024 Maximum concurrently-armed live actions
kill-switch false bool Revert all armed rules and force shadow
allowlist (none) list of prefixes Protected sources never armed

See also

  • DDoS detection and mitigation for the volumetric subsystem that shares the traffic-feature facts layer.
  • Firewall for the nftables backend the shadow-first responder installs rate-limits and drops onto.
  • Traffic Control for per-interface queueing and scheduling.

Adapted from the in-tree source: internal/plugins/anomaly/ and internal/component/trafficfeature/ .

Home

About

First Steps

Configuration

Operation

Interfaces

Plugins

Plugin Development

Chaos Testing

Blueprints

Development

Reference

Clone this wiki locally