Repository navigation
Lab SIEM EDR and Hardware Verification Matrix
Opened: November 11, 2026 ยท This is an evidence ledger and reproducible acceptance plan, not proof that these deployments work in a current environment.
Static only means syntax, documentation or a specific isolated fixture may be checked; lab blocked means no real device/SIEM/provider outcome has been provided; external provider required means an authorized API tenant, version and credentials are required. Do not copy a successful source parser status onto network coverage, a physical radio, a live endpoint agent or a customer device.
This matrix inventories high-use canonical guides and separates observed, supported evidence from still-required experiments. Dates below refer to inventory review, not runtime verification. For a live test to count, use the record template below and link the sanitized evidence only if it contains no private deployment details. Keep client names, internal IPs, raw alerts and credentials in secure engagement records.
The Offline Canary Detection Evidence Validator now provides a JSONL input contract, positive/benign-negative fixtures, and machine-readable true-positive/true-negative/false-positive/false-negative/unassessable outcomes. It checks case IDs, source, timezones, duplicate/missing telemetry, rule IDs and latency. The Python tool never contacts any SIEM, and its real_siem_validated flag remains false even when user-supplied exports are accepted. It is a building block for future authorized hardware/manager evidence, not a substitute for that test.
The Wireless IDS CI workflow and Guide Fixture CI workflow exercise offline Python unit tests, synthetic event normalization, a localhost-only HTTP decoy/webhook, and structural/regex-only Wazuh rule contracts on an ephemeral GitHub-hosted Ubuntu runner. The full PR #195 CI results are the record for the actual commit, runner and success/failure outcomes; do not transfer these claims to a different Wazuh version or device.
The audit identified and corrected malformed Wazuh XML: rule 100340 had been placed inside the grouping label of rule 100300, and 100402 inside the grouping label of 100400. The new offline tests assert root-level rule structure, unique IDs, local ancestry, and synthetic positive and benign-negative field patterns; they explicitly do not run /var/ossec/bin/wazuh-logtest or simulate Wazuh frequency/timeframe/grouping logic.
| Scope | Positive Synthetic Expectation | Benign/Negative Expectation | Actual Evidence Available |
|---|---|---|---|
| Kismet normalizer + Wazuh candidate rules |
wids.integration=kismet, DEAUTHFLOOD => structural candidate 100301; correlation => 100340
|
CHANCHANGE must not match the deauth child; other integration must not match |
GitHub Python fixture result only โ no actual manager alert |
| nzyme webhook + candidate rules |
disconnect_anomaly => structural candidate 100352
|
system_event must not match 100352; instead 100354
|
Loopback webhook status/JSONL and regex fixture tests; actual nzyme provider API not exercised |
| DIY HTTP honeypot + candidate rules |
GET => 100400; POST => 100400 + 100401; ESP32 serial => 100402
|
GET must not match elevated write-method 100401
|
Localhost-only HTTP and schema fixtures; actual Pi/ESP32 and Wazuh ingestion not exercised |
Observed detection score: Not available. A passing code fixture demonstrates test-contract consistency, not actual sensor visibility, analyst detection, low false-positive rates, or an approved deployment. Log the real manager's match/level and non-match results with wazuh-logtest on a separately authorized Wazuh 4.x environment; record the exact installed release and ruleset commit, then check event ingestion and dashboard alert correlation. Do not deploy candidate rules directly to production from CI.
| Canonical Procedure | Evidence Available | Unverified Condition / Acceptance Experiment | Current Status |
|---|---|---|---|
| OSINT Case Investigator | Isolated shell fixtures in hardening tests cover named behavior | Current registry/API network response, rate limits, actual report quality and consent | Static fixture only |
| Kismet โ Wazuh Wireless Alerts | Source/configuration, synthetic bridge fixtures and XML group/regex contract tests exist | Kismet release, capture chipset, firmware, monitor mode, real 802.11 frames, end-to-end Wazuh ingestion and rule firing | Lab blocked |
| nzyme โ Wazuh | Guide and local adapter/config content | nzyme version, webhook semantics, signature, TLS, observed source/destination timestamps, false-positive handling | External provider / lab blocked |
| Suricata & Zeek | Documentation and example detection configuration | Verified TAP/SPAN visibility, sensor interface, IDS ruleset revision, PCAP replay and actual alert + Zeek log correlation | Lab blocked |
| Wi-Fi Disconnect Detection | Defensive detection guidance and filter examples | 802.11 frame capture, PMF state, negative benign disconnect fixtures, severity fidelity in SIEM | Lab blocked |
| Graylog Deployment | Historical Docker example (Graylog 5.2; OpenSearch 2.12.0) | Supported version/migration, TLS, secrets, inputs, index writes, alert-to-notification chain | Historical; not deployment-verified |
| Elastic SIEM/ELK | Example compose/health check procedure | Current image compatibility, secure account setup, ingestion, query/rule results and recovery after restart | Lab blocked |
| Snort/IDS Integration | Architectural guides | Actual mirrored traffic reaching monitored interface; rule and SIEM event chain with timestamped capture | Lab blocked |
| DIY Sensor Honeypot | Local-only demo, loopback HTTP fixture and repaired XML rule group with synthetic positive/negative contract tests | Isolated canary health-check, synthetic hit, JSONL/event schema validation, Wazuh intake, cleanup | Lab blocked |
| Raspberry Pi Honeypot | Hardware setup procedure | Specific Pi model, OS kernel, network segmentation, capture, watchdog and SIEM correlation | Hardware blocked |
| NanoVNA | Fixture test runner for lab files | Actual model, calibration standards, firmware, connection, measured sweep against known reference and RF safe handling | Fixtures only; hardware blocked |
| ESP32 Security Firmware | Catalog/source metadata only | Exact board revision, bootloader/flasher release, backup/recovery, passive receive test, approved RF scope | Hardware blocked |
| M5StickC Plus2 | Source and board guidance | Actual display/IR/RF peripherals, battery, firmware compatibility and reset/recovery | Hardware blocked |
| PowerShell Port Scanner | Source parser checks; bounds corrected | Owned loopback listener, refused/timeout paths, DNS errors, timeouts, outbound-scope control and observed PowerShell version | Static syntax only |
| AnythingLLM CVE Lookup | Reviewed reference and risk boundaries | Actual NVD/CISA/FIRST provider responses, valid JSON parsing, stale/unknown values, no invented remediation | External provider required |
| Data Engineering Fixtures | CI calls offline runner | External broker/store and real telemetry behavior, ingestion latency, security of credentials and rollback | Static/fixture only |
- Approve scope: record named operator, authorization window, written owner approval, isolated lab network, allowed simulated activity, stop conditions, and emergency contact. A fixture is not approval for external or production targeting.
- Baseline: record OS/kernel, device model and revision, firmware/build, SIEM/EDR release, ruleset ID/revision, time sync, listening interfaces, monitoring points, target host, and expected sensor visibility. Use artificial IDs and RFC 5737 documentation addresses in public examples.
- Positive test: feed an approved PCAP or synthetic event into the pipeline; document event/alert time, log fields, source component, collector, rule identifier, rule version, and exact expected/observed detection. Do not automatically execute LLM-generated exploit code.
- Negative/benign test: supply similar harmless activity and empty/invalid input; measure false positives, dropped events, parse errors, crash/retry behavior and missing telemetry. A positive alert with zero negatives is insufficient to rate the rule effective.
-
Score: score separately for sensor visibility, telemetry completeness, rule trigger, correlation/context, latency, and false-positive behavior. Use
observed/failed/not-testedfor every dimension instead of manufacturing a composite detection percentage. - Cleanup & sign-off: record final state, disable test accounts, stop temporary listeners, delete synthetic artifacts per retention policy, roll back agent/rule changes, and have a human approve deployment or mark it blocked.
| Field | Operator Entry |
|---|---|
| Lab/engagement authorization and scope | Required before executing; store sensitive details privately |
| Guide path and exact Git commit | Not recorded โ supply commit under test |
| Test ID, operator and UTC date | Not tested โ fill on execution |
| Device/VM model, OS, kernel and firmware | Not tested โ identify exact versions |
| SIEM/EDR and detection rule/version | Not tested โ name product/build and rule ID |
| Positive synthetic fixture and checksum | Not recorded โ use approved non-sensitive fixture |
| Benign/negative fixture and checksum | Not recorded โ must be independently reviewed |
| Expected alert source, fields and severity | Not recorded โ derive from tested rule, not a guess |
| Observed source/collector/alert timestamps | Not tested โ preserve redacted evidence privately |
| Detection/false positive/false negative results | Not tested โ specify sample sizes |
| Telemetry gaps and vendor/API limitations | Not tested โ document failures too |
| Rollback, data retention and cleanup | Not performed โ document exact completion |
| Approval and next action | Awaiting authorized operator review |
Only anonymized fixture outputs, SHA-256 checksums of synthetic data, public documentation references, scoped test commands and non-sensitive summarized outcomes belong in this repository. Do not publish client log streams, telemetry containing personal data, deployment-specific addresses, tokens, proprietary SIEM rules or detailed exploitation output.
This is the starting work product for issue #190. The issue is not closed until specific hardware/vendor/SIEM tests have been performed and reviewed by an authorized operator.
Documentation Index ยท Verification Protocol ยท Maintenance Operations
๐ Source: Documentation/LAB-VERIFICATION-MATRIX.md | โ๏ธ Suggest an edit | โฌ
๏ธ Back to Technical Guides & Cheat Sheets
- ๐บ๏ธ Browse by Topic
- ๐ All Repository Files
- ๐งญ Learning Paths
- ๐ Master Guide
- ๐ค Glossary
- โ๏ธ Legal & Disclaimer
๐ Foundations & Learning
๐ก๏ธ Defense & Investigations
- Incident Response
- Open Source Intelligence (OSINT)
- Reverse Engineering
- Operations Security (OPSEC)
- Compliance
โ๏ธ Platforms & Infrastructure
๐ก Hardware, Wireless & Field Projects
- PNWC Cyberdeck
- Hardware Testing
- Hardware Hacking
- Software-Defined Radio
- Wardriving
- uConsole
- Space Security
๐งฐ Assessment & Field Reference
๐ฆ Downloads & Project Resources
๐๏ธ Additional Topics