Detection Engine

Detection Engine

Hybrid R-Sentry uses six detection layers that work together to produce a high-confidence threat score. Each layer catches a different aspect of ransomware behavior. The first five layers run in the inotify-based agent (monitor.py); Layer 6 is the optional eBPF kernel sensor (monitor_ebpf.py) that provides deeper, lower-latency visibility.

---

Layer 1 — Canary Files

What it does: Places strategically named bait files (AAA_*.txt) across the monitored directory. Any process that reads, modifies, moves, or deletes a canary file triggers an immediate CRITICAL alert.

Why it works: Ransomware encrypts files indiscriminately — it will inevitably touch the canary files. Legitimate processes have no reason to touch files named AAA_.

Key details:

• 30 canary files placed by default (configurable via CANARY_COUNT)
• 4 naming prefixes: AAA_, aaa_, ZZZ_, zzz_ — 4× the surface area vs. a single prefix
• Placement strategy: BFS across directory tree, 5 canaries per directory spread across prefixes
• Any canary touch, move, or deletion → CRITICAL → auto-containment fires immediately
• All 4 prefix variants are excluded from .gitignore to prevent accidental commits
• In eBPF mode, canary inodes are registered in the canary_inodes BPF map at startup — renames are blocked at the kernel level (-EPERM) before any file is overwritten
• Markov repositioner (inotify mode only) is blocked from placing canaries into .git/, /proc/, /sys/, /dev/, /run/
• Agent validates WATCH_PATH at startup and exits if it's inside a git repo

---

Layer 2 — Shannon Entropy Analysis

What it does: Monitors the Shannon entropy (randomness) of files as they are modified. Ransomware encrypts files, which dramatically increases their entropy. The engine tracks a rolling window of entropy values per file and fires when the delta exceeds the threshold.

Why it works: Encrypted data is indistinguishable from random noise — it always has very high entropy (close to 8 bits). Normal file modifications do not cause sudden entropy spikes.

Key details:

• Threshold: 3.5 bits delta → MEDIUM alert
• Absolute threshold: 7.2 bits current entropy → HIGH alert
• Rolling window: last 30 samples per file
• Spike window: delta must occur within 10 seconds
• Memory cap: 5 000 files with LRU eviction — prevents OOM on large watch paths
• Partial reads: only the first 65 KB of each file is sampled

Severity mapping:

Condition	Severity
Delta >= 5.0 bits	HIGH
Absolute entropy >= 7.2 bits	HIGH
Delta >= 3.5 bits	MEDIUM

---

Layer 3 — Process Lineage Scoring

What it does: When a file event is triggered by an active process (PID > 0), the engine walks the process ancestry chain and scores the process on multiple suspicion signals.

Scoring factors:

Signal	Weight	Description
Suspicious parent process	+30	Parent is netcat, ncat, etc.
Suspicious spawn path	+25	Process executable in /tmp/, /dev/shm/
Deep ancestry chain	+15	More than 5 ancestor processes
Unreadable executable	+10	Cannot read the process binary
No controlling TTY	+5	Background/hidden process
Rapid spawn	+5	Process age < 2 seconds
Process exited rapidly	+40	Process exits before scoring completes (baseline score)
Benign parent	-15	Parent is systemd, bash, python3, etc.

dpkg hash verification: The process binary SHA-256 is verified against a database of 416 K dpkg hashes. Verdict is one of:

• MATCH — binary matches a known legitimate package
• MISMATCH — binary is present but hash differs (tampered)
• UNKNOWN — binary not in dpkg database (unpackaged or custom binary)

A 512-entry LRU cache avoids repeated SHA-256 computation on the same binary.

Score range: 0–100. A score >= 40 triggers an alert.

---

Layer 4 — Ransomware Extension Detection

What it does: Watches for file rename events where the destination extension matches known ransomware patterns. Also detects newly created files with ransomware extensions.

Trigger conditions:

Condition	Severity
Document file renamed to .enc, .locked, .wcry, .crypted, .crypt, .encrypted etc.	CRITICAL
Non-document file renamed to a ransomware extension	HIGH
New file created with a ransomware extension	HIGH

Why it works: Ransomware often renames files in-place after encrypting them, appending its signature extension. This layer catches that behavior even before entropy thresholds are reached.

---

Layer 5 — Markov Chain Adaptive Repositioning

What it does: Tracks which directories are accessed most frequently and automatically moves canary files to predicted high-risk locations.

Why it works: Ransomware typically starts in one area and spreads. The Markov chain models directory access transitions and predicts where the attacker will go next, pre-positioning canary files there.

Key details:

• Repositioning triggers when transition probability >= 70%
• Minimum 10 observations required before repositioning
• Repositioning is logged as MARKOV_REPOSITION event (not treated as a threat)
• Runs every 300 seconds by default
• _is_safe_target() blocks all system paths — never repositions into .git/, /proc/, /sys/, /dev/, /run/

---

Layer 6 — eBPF Kernel Sensor

What it does: A kernel-level sensor (agent/monitor_ebpf.py) that attaches to multiple syscall probes and (optionally) a BPF LSM hook to detect and block ransomware behavior at the kernel boundary — below the userspace filesystem layer. Provides faster detection, richer behavioral context, and the ability to block canary file renames in nanoseconds.

Requirements: Linux kernel ≥ 6.19 with BCC 0.35 installed (sudo apt install python3-bpfcc bpfcc-tools -y). Falls back to inotify automatically if BCC is not available (SENSOR_BACKEND=inotify in .env).

Key capabilities:

Feature	Description
5-syscall coverage	Monitors openat (file access), vfs_write (binary writes), unlink (file deletion), rename (velocity tracking), execve (child spawning) simultaneously via kprobes/tracepoints
proc_profile BPF map	Per-process behavioral profile (10 000 entries): files_opened, files_written, files_deleted, files_renamed, child_procs, write_bytes, score (0–100), alerted flag
Behavioral scoring (6 signals)	+35: rapid unlink+write (≥2 del/sec); +10: same + deleted>5; +25: rename velocity ≥3; +15: total_ops>15 && deleted>3; +10: child spawning + file ops; +15: hyper-fast burst ≥20 del/sec + 5+ files (Signal 6 — closes LockBit/Qilin speed gap)
Kernel unlink blocking	__handle_unlink now calls blocked_pids.update() when score ≥ SCORE_BLOCK (70) — no userspace round-trip needed for ultra-fast families
Entropy verification	On behavior_event: if entropy ≥ 6.5 or score ≥ 70 → BLOCK + SIGSTOP; if < 6.5 and score < 70 → ALERT only. The score bypass handles families that delete files faster than the 1 ms perf poll
BPF LSM canary blocking	When lsm=bpf is active: LSM_PROBE(path_rename) + canary_inodes BPF map blocks canary renames with -EPERM in nanoseconds — no userspace roundtrip needed
blocked_pids BPF map	Velocity burst sets a kernel-level block on the PID — all subsequent renames by that process are denied by the LSM hook
Ransomware family profiling	Recognizes LockBit 5.0 (16-char extension, two-pass), Akira (.akiracrypt, intermittent), ESXi-targeting (.vmdk/.vmx/.vmem)
IGNORE_COMMS suppression	Filters out 40+ known benign processes (Docker, Redis, git, apt, npm, uvicorn, Celery, Firefox, etc.) before any scoring
Run modes	--mode audit (alert only, no SIGSTOP); --selftest (unit-tests the engine); --seed-canaries (place canaries without running sensor); --print-bpf (print compiled BPF program)

Sensor backend selection: Set SENSOR_BACKEND=ebpf (default) or SENSOR_BACKEND=inotify in .env.

Markov repositioning: Disabled when using eBPF backend — the eBPF sensor tracks file access patterns via proc_profile directly, making Markov repositioning redundant.

BPF LSM activation: The canary_inodes kernel-level blocking requires the lsm=bpf kernel boot parameter. Without it, the sensor falls back to SIGSTOP via the userspace contain() callback. To enable:

# Add to /etc/default/grub:
GRUB_CMDLINE_LINUX="lsm=bpf"
sudo update-grub && sudo reboot

Why eBPF? Userspace inotify events can be delayed or coalesced by the kernel. eBPF probes fire inside the kernel syscall path — sub-millisecond latency and no way for ransomware to evade by timing userspace polling.

---

Combined Threat Score

When both entropy and lineage signals fire, they are combined into a single weighted score:

combined_score = lineage_score × 0.6 + (entropy_delta / 8.0 × 100) × 0.4

Combined Score	Event Type	Severity
>= 70	COMBINED_ALERT	CRITICAL
40–69	COMBINED_ALERT	HIGH
Entropy only	ENTROPY_SPIKE	MEDIUM
Lineage only (>= 40)	PROCESS_ANOMALY	HIGH

---

False Positive Suppression

The system includes an exception whitelist (agent/exceptions.py) that suppresses alerts for known-safe activity:

• Whitelisted processes: apt, git, pip, docker, firefox, gpg, celery, uvicorn, etc.
• Whitelisted paths: ~/.cache/, /var/cache/apt/, browser profile directories
• Whitelisted extensions: .zip, .gpg, .pdf, .jpg, .mp4, .pyc, .deb, and more
• Smart /tmp filter: files in /tmp/ with document extensions (.docx, .pdf, etc.) bypass the whitelist — prevents ransomware staging in /tmp/ being silently ignored

Canary hits are never whitelisted — they always trigger regardless of process or path.

---

Attack Simulations

Three simulation scripts in simulations/ are provided to validate detection performance without needing real ransomware:

Script	Ransomware	Pattern	Detection path
sim_lockbit.py	LockBit 5.0	Two-pass: rename all → encrypt all; 16-char random extension; priority .vmdk/.vmx	Canary → extension detection → behavioral Score 6 (rename velocity)
sim_akira.py	Akira	Intermittent: encrypts ~60% of corpus, .akiranew extension, non-sequential write offsets	SILENT_ENCRYPTION (write-offset tracker)
sim_qilin.py	Qilin	Percent-encryption (40%), 7-char random extension, bulk unlink+create	Signal 6 (hyper-fast unlink burst) → kernel unlink block
sim_dfs.py	Generic DFS	Depth-first traversal with canary hits	Canary touch → CRITICAL
sim_random.py	Generic random	Random order encryption	Entropy spike

How to run (with agent and backend running):

cd ~/hybrid-rsentry
source venv/bin/activate
python simulations/sim_lockbit.py

# Validate containment fired:
python simulations/sim_lockbit.py --validate-defense

All simulations create a sandboxed corpus under /tmp/rsentry_sim_*/, run the attack, then restore originals — non-destructive.

Full simulation test results (v2.2.0, live eBPF stack):

Family	Files	Duration	Detection path	Contained
WannaCry (generic)	1 392	~0.6 s	Canary → CRITICAL	✅
LockBit 5.0	1 392	~0.6 s	Signal 6 + kernel unlink block	✅
Akira	~835	~1.2 s	SILENT_ENCRYPTION (write-offset)	✅
DarkSide (generic)	1 392	~0.6 s	Canary → CRITICAL	✅
Qilin	~557	~0.3 s	Signal 6 + kernel unlink block	✅
DFS Canary	—	instant	Canary inode touch → CRITICAL	✅

Validation targets (LockBit 5.0 evaluation — tests/test_lockbit.py):

Metric	Target	Result
Files encrypted before detection	< 3	✅ Met
Detection latency	< 500 ms	✅ Met
False positive rate	0%	✅ Met
Coverage	100%	✅ Met