Skip to content

scalytics/kafSIEM

Folders and files

NameName
Last commit message
Last commit date

Latest commit

 

History

510 Commits
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 

Repository files navigation

kafSIEM

Edge-ready operations intelligence with an evidence-linked entity graph.

kafSIEM watches Kafka agent traffic and OSINT feeds, materializes entities and relationships in SQLite, and serves analyst workflows through a web desk and typed OpenAPI. Deploy with Docker on a single host. No cluster database required.

License: Apache-2.0 OpenAPI Docker

Keywords: entity graph, Kafka SIEM, OSINT dashboard, drone fleet operations, SCADA security, edge deployment, ontology packs, provenance, SQLite analyst API.

Screenshots

kafSIEM OSINT globe and alert feed kafSIEM Operations desk with drones ontology graph kafSIEM Operations desk with SCADA ontology graph

OSINT | Operations / Drones | Operations / SCADA

Try in 30 seconds

No Kafka, API keys, or Docker required.

git clone https://github.com/scalytics/kafSIEM.git && cd kafSIEM
make help          # list all targets
make demo-osint    # public OSINT relations demo (fixtures, no Docker)
make demo-ontology # Operations ontology desk
make demo-fusion   # Fusion desk

make demo-osint opens the OSINT console with bundled alert and incident fixtures — cyber, malware, ICS/energy, and terror clusters in the Relations panel. No Kafka or Docker required.

What you get

  • Entity graph stored in SQLite: platforms, faults, devices, agents, tasks. Graph edges can cite the source Kafka record.
  • Analyst desk: flow queue, topology graph, map, replay studio, entity profiles, provenance drawer.
  • Domain packs: YAML ontology for drones and SCADA. Detectors, queries, views, map layers, report templates. No pack executable code.
  • Typed API: /api/v1/... with generated OpenAPI. Integrates with existing enterprise platforms.
  • Edge deployment: collector writes, API reads, shared volume. Docker Compose or Helm.

Who it is for

Unmanned systems teams tracking readiness, sorties, EW events, software lineage, and signoff evidence across a fleet.

SCADA and critical infrastructure teams connecting plants, devices, change audit, alarms, firmware, vulnerabilities, and engineer sessions.

Pack Example questions
Drones What changed since last signoff? Is this fault fleet-wide? Which platforms run software version X?
SCADA Which changes lack a work order? Is firmware drifting on safety PLCs? What CVEs affect live devices?

How it works

KafClaw agents
      |
      v
KafScale / Kafka
      |
      v
kafsiem-collector  --->  /data/agentops.db (+ WAL/SHM)
      |                           |
      | legacy /api/*             v
      +---------------->  kafsiem-api  --->  Caddy + web UI

The collector ingests OSINT sources and, when enabled, consumes Kafka traffic. It is the sole writer for observed operational state. kafsiem-api serves read-only analyst resources and enqueues replay requests. See Architecture.

Operating modes

Product name UI_MODE Purpose
OSINT OSINT Globe-first intelligence, alert feeds, attack-relation clusters
Operations AGENTOPS Kafka agent flow tracking and ontology desk
Fusion HYBRID Operations plus heuristic OSINT correlation

Set mode via UI_MODE and mounted policy files. User-facing copy uses OSINT, Operations, and Fusion. Runtime env values stay AGENTOPS and HYBRID for compatibility.

Quick start (Docker)

cp .env.example .env
docker compose up --build

UI: http://localhost:8080

Make targets for local HTTP dev:

make dev-start
make dev-stop
make dev-restart
make dev-logs

Remote install

wget -qO- https://raw.githubusercontent.com/scalytics/kafSIEM/main/deploy/install.sh | bash

The installer checks Docker, clones or updates the repo, prompts for profile (OSINT, Operations, or Fusion), site address, and profile-specific settings, then starts GHCR images.

API example

curl -s http://localhost:8080/api/v1/ontology/packs | jq .
curl -s http://localhost:8080/api/v1/flows | jq .
curl -s http://localhost:8080/api/v1/entities/platform/auv-7 | jq .

Full contract: api/openapi.yaml. Client notes: docs/api-clients.md.

Stack context

Component Role
KafScale External Kafka transport. kafSIEM consumes from it.
KafClaw External agent plane. Emits group envelopes on Kafka topics.
kafSIEM Observer, graph store, API, UI, packs (this repository).

kafSIEM complements enterprise intelligence platforms. It exposes typed entity and graph APIs, pack-defined ontology, and provenance back to source records.

Documentation

Topic Document
Full doc index docs/README.md
Architecture docs/architecture.md
Operations / Docker / VM docs/operations.md
Operations Kafka config docs/agentops-operator-guide.md
API clients docs/api-clients.md
Drones pack docs/packs/drones.md
SCADA pack docs/packs/scada.md
OSINT user guide docs/userguide.md
Advanced env tuning docs/advanced-config.md
Minimal env template .env.example

Development

make check
make ci
make docker-build

Node 25.8.1 and npm 11.11.0 are pinned in package.json, .nvmrc, and .node-version. The Go collector powers scheduled feed refresh, Docker runtime, and local collection commands.

Local workflows (make help for the full list):

Goal Command
OSINT relations demo (fixtures) make demo-osint
Full local stack (live collector + API) make dev-starthttp://localhost:8080
Go collector tests make go-test
Full CI gate make ci or make commit-check

Browse corroborated clusters: header Relations → filter by attack type (cyber, terror, maritime, conflict). See docs/userguide.md.

License

Apache-2.0. See LICENSE.

About

Entity graph platform for air-gapped defense and critical infrastructure. Kafka-native OSINT, OT fusion, and immutable provenance on every edge.

Topics

Resources

License

Stars

60 stars

Watchers

0 watching

Forks

Packages

 
 
 

Contributors