Skip to content

Detection Engine

Jump to bottom
Mohhudib edited this page May 29, 2026 · 10 revisions

Detection Engine

Hybrid R-Sentry uses five detection layers that work together to produce a high-confidence threat score. Each layer catches a different aspect of ransomware behavior.

Layer 1 — Canary Files

What it does: Places strategically named bait files (AAA_*.txt) across the monitored directory. Any process that reads, modifies, moves, or deletes a canary file triggers an immediate CRITICAL alert.

Why it works: Ransomware encrypts files indiscriminately — it will inevitably touch the canary files. Legitimate processes have no reason to touch files named AAA_.

Key details:

  • 15 canary files placed by default (configurable via CANARY_COUNT)
  • Placement strategy: BFS across directory tree for maximum coverage
  • Any canary touch, move, or deletion → CRITICAL → auto-containment fires immediately
  • AAA_*.txt is excluded from .gitignore so they can never be accidentally committed
  • Markov repositioner is blocked from placing canaries into .git/, /proc/, /sys/, /dev/, /run/
  • Agent validates WATCH_PATH at startup and exits if it's inside a git repo

Layer 2 — Shannon Entropy Analysis

What it does: Monitors the Shannon entropy (randomness) of files as they are modified. Ransomware encrypts files, which dramatically increases their entropy. The engine tracks a rolling window of entropy values per file and fires when the delta exceeds the threshold.

Why it works: Encrypted data is indistinguishable from random noise — it always has very high entropy (close to 8 bits). Normal file modifications do not cause sudden entropy spikes.

Key details:

  • Threshold: 3.5 bits delta → MEDIUM alert
  • Absolute threshold: 7.2 bits current entropy → HIGH alert
  • Rolling window: last 30 samples per file
  • Spike window: delta must occur within 10 seconds
  • Memory cap: 5 000 files with LRU eviction — prevents OOM on large watch paths
  • Partial reads: only the first 65 KB of each file is sampled

Severity mapping:

Condition Severity
Delta >= 5.0 bits HIGH
Absolute entropy >= 7.2 bits HIGH
Delta >= 3.5 bits MEDIUM

Layer 3 — Process Lineage Scoring

What it does: When a file event is triggered by an active process (PID > 0), the engine walks the process ancestry chain and scores the process on multiple suspicion signals.

Scoring factors:

Signal Weight Description
Suspicious parent process +30 Parent is netcat, ncat, etc.
Suspicious spawn path +25 Process executable in /tmp/, /dev/shm/
Deep ancestry chain +15 More than 5 ancestor processes
Unreadable executable +10 Cannot read the process binary
No controlling TTY +5 Background/hidden process
Rapid spawn +5 Process age < 2 seconds
Process exited rapidly +40 Process exits before scoring completes (baseline score)
Benign parent -15 Parent is systemd, bash, python3, etc.

dpkg hash verification: The process binary SHA-256 is verified against a database of 416 K dpkg hashes. Verdict is one of:

  • MATCH — binary matches a known legitimate package
  • MISMATCH — binary is present but hash differs (tampered)
  • UNKNOWN — binary not in dpkg database (unpackaged or custom binary)

A 512-entry LRU cache avoids repeated SHA-256 computation on the same binary.

Score range: 0–100. A score >= 40 triggers an alert.

Layer 4 — Ransomware Extension Detection

What it does: Watches for file rename events where the destination extension matches known ransomware patterns. Also detects newly created files with ransomware extensions.

Trigger conditions:

Condition Severity
Document file renamed to .enc, .locked, .wcry, .crypted, .crypt, .encrypted etc. CRITICAL
Non-document file renamed to a ransomware extension HIGH
New file created with a ransomware extension HIGH

Why it works: Ransomware often renames files in-place after encrypting them, appending its signature extension. This layer catches that behavior even before entropy thresholds are reached.

Layer 5 — Markov Chain Adaptive Repositioning

What it does: Tracks which directories are accessed most frequently and automatically moves canary files to predicted high-risk locations.

Why it works: Ransomware typically starts in one area and spreads. The Markov chain models directory access transitions and predicts where the attacker will go next, pre-positioning canary files there.

Key details:

  • Repositioning triggers when transition probability >= 70%
  • Minimum 10 observations required before repositioning
  • Repositioning is logged as MARKOV_REPOSITION event (not treated as a threat)
  • Runs every 300 seconds by default
  • _is_safe_target() blocks all system paths — never repositions into .git/, /proc/, /sys/, /dev/, /run/

Combined Threat Score

When both entropy and lineage signals fire, they are combined into a single weighted score:

combined_score = lineage_score × 0.6 + (entropy_delta / 8.0 × 100) × 0.4
Combined Score Event Type Severity
>= 70 COMBINED_ALERT CRITICAL
40–69 COMBINED_ALERT HIGH
Entropy only ENTROPY_SPIKE MEDIUM
Lineage only (>= 40) PROCESS_ANOMALY HIGH

False Positive Suppression

The system includes an exception whitelist (agent/exceptions.py) that suppresses alerts for known-safe activity:

  • Whitelisted processes: apt, git, pip, docker, firefox, gpg, celery, uvicorn, etc.
  • Whitelisted paths: ~/.cache/, /var/cache/apt/, browser profile directories
  • Whitelisted extensions: .zip, .gpg, .pdf, .jpg, .mp4, .pyc, .deb, and more
  • Smart /tmp filter: files in /tmp/ with document extensions (.docx, .pdf, etc.) bypass the whitelist — prevents ransomware staging in /tmp/ being silently ignored

Canary hits are never whitelisted — they always trigger regardless of process or path.

Clone this wiki locally