Skip to content

ocsf: add OCSF security event analytics demo app#41

Merged
gangtao merged 14 commits into
mainfrom
feature/39-ocsf-demo
May 29, 2026
Merged

ocsf: add OCSF security event analytics demo app#41
gangtao merged 14 commits into
mainfrom
feature/39-ocsf-demo

Conversation

@gangtao
Copy link
Copy Markdown
Contributor

@gangtao gangtao commented May 29, 2026

Summary

Adds a new apps/ocsf/ Timeplus app that streams synthetic OCSF (Open Cybersecurity Schema Framework) security events from the timeplus-ocsf-simulator PyPI package and surfaces real-time threat-detection analytics across five dashboards.

What's in the box

Pipeline (7 resources): a Python streaming external_stream calling ocsf_simulator.stream_ocsf_events → MV → persistent ocsf_events(raw, class_uid) stream with configurable TTL → four flatten views projecting nested JSON into typed columns for each OCSF class:

  • v_ocsf_authentication_flatten (3002)
  • v_ocsf_network_activity_flatten (4001)
  • v_ocsf_process_activity_flatten (1007)
  • v_ocsf_security_finding_flatten (2001)

Flatten view bodies copied verbatim from demos/cases/ocsf/extraction.sql with three mechanical edits per file: {{ .DB }} templating, switch WHERE json_value(raw, '$.class_uid') = 'NNNN' → typed class_uid = NNNN, and IF NOT EXISTS.

Five dashboards (each ≤6 live streaming queries):

  • OCSF Overview — 4 singleValue counters + About markdown
  • OCSF — Authentication — Auth by status/min, Top failed sources, Brute-force candidates
  • OCSF — Network Activity — Bytes by protocol/min, High-severity events, Scanning candidates
  • OCSF — Process Activity — Process creates/min, Suspicious cmd lines, Privilege-mismatch
  • OCSF — Security Findings — Findings by severity/min, Malware by classification, Top finding titles

Config knobs: event_classes, interval_seconds, ocsf_version, retention_hours.

Lessons captured (memory updated)

Two non-obvious dashboard gotchas surfaced during build and are now recorded so future apps avoid them:

  • chartType: "singleValue" (camelCase)single_value is not registered in axion and silently renders "Unable to render chart"
  • 12-column position grid, not 24 — putting x=12 or x=18 wraps panels into a stacked column on the right edge

Test plan

  • make build install APP=ocsf from a fresh checkout succeeds against a local Timeplus
  • Wait ~60s, then SHOW STREAMS FROM ocsf lists ocsf_events_source, ocsf_events, the 4 flatten views, and mv_ocsf_events_ingest
  • SELECT class_uid, count() FROM table(ocsf.ocsf_events) GROUP BY class_uid returns positive counts for all of {1007, 2001, 3002, 4001}
  • All 5 dashboards open in the UI and every panel renders (no "Unable to render chart", no stuck-loading)
  • Multi-series line charts (Auth by status, Bytes by protocol, Findings by severity) show one line per series, not collapsed
  • Overview lays out 4 metric panels in a single row above the About markdown
  • make build install APP=ocsf over an existing install is a clean upgrade (idempotent CREATEs)

🤖 Generated with Claude Code

gangtao and others added 14 commits May 28, 2026 11:59
Approved design for a new .tpapp wrapping the timeplus-ocsf-simulator
PyPI package with four flatten views (auth, network, process, security
finding) and a single dashboard. Lean v1 — no alerts, no sequence
analysis, no cross-schema joins.

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
17 panels across Authentication, Network Activity, Process Activity, and
Security Finding event classes: 4 metric headers, 4 multi-series line charts
(with color by series column), 8 tables, and 1 markdown about panel.

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
Each new dashboard stays under 6 live streaming queries. The overview
dashboard holds the 4 headline counters and About markdown; each event
class (auth, network, process, findings) gets its own 3-panel dashboard.
The 4 header metric panels were using chartType:'single_value' with a
config of {updateMode, updateKey} — neither matches axion's chart
registry. The actual chart type is 'singleValue' (camelCase) with a
config containing the column name to display plus sparkline/delta/unit/
font controls. Per-panel 'value' wired to each SELECT's alias; counts
use fractionDigits=0 and events_per_sec uses fractionDigits=2.
All existing apps use a 12-column position grid (per skill/references/
dashboard-spec.md line 44 and every dashboard in apps/). The OCSF
dashboards were authored against an assumed 24-column grid, which made
the 3rd and 4th overview metric panels (x=12, x=18) wrap into a vertical
stack on the right column. Halving x and w on every panel: overview
metrics now fit 4 across, per-class line charts span the full row, and
the two-table rows split 50/50.

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
Records the executed task-by-task plan: pre-flight simulator signature
check, 9 implementation tasks, and final verification. Matches what
shipped (singleValue chart fix and 12-column grid fix retroactively
applied after the user spotted the bugs in the browser).

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
@gangtao gangtao linked an issue May 29, 2026 that may be closed by this pull request
@gangtao gangtao merged commit 77c157a into main May 29, 2026
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

OCSF demo

1 participant