Dual Timeline

Dual Timeline

Overview

IRIS-NG adds a working timeline panel alongside the existing master timeline. The working timeline is a staging area: import bulk forensic tool output, review events individually, and promote the ones worth keeping to the master timeline.

Architecture

┌─────────────────────────────────────────────────┐
│  Case timeline page                             │
│                                                 │
│  [Master timeline]  │  [AI analysis panel]      │
│  (permanent record) │  (always-visible prose)   │
│                     │                           │
│  ─────────────────────────────────────────────  │
│                                                 │
│  [Working timeline panel] (right side)          │
│  import → review → promote / reject             │
└─────────────────────────────────────────────────┘

Master timeline events are permanent. Working timeline events are ephemeral staging entries in case_working_event.

Importing events

Hayabusa (JSONL)

POST /api/v2/cases/<cid>/working-timeline/import/hayabusa
Content-Type: multipart/form-data

file=<hayabusa-results.jsonl>
[begin_date=YYYY-MM-DD]   # optional, inclusive
[end_date=YYYY-MM-DD]     # optional, inclusive
csrf_token=<token>

Hayabusa's sigma-rule fan-out (1 EVTX event → N matched rules) is collapsed on (Timestamp, Computer, Channel, EventID, RecordID) so the panel shows one reviewable event per source event, not one per matched rule.

The EVTX filename in each card shows the basename only (full path preserved in event_raw.evtx_file). Collection paths from Windows tools use backslashes; always split on [\\/] — not os.path.basename() — in parsers running inside Linux containers.

EZ Tools / KAPE CSV (11 sub-formats)

POST /api/v2/cases/<cid>/working-timeline/import/eztools
Content-Type: multipart/form-data

file=<EZTools-output.csv>
[begin_date=YYYY-MM-DD]
[end_date=YYYY-MM-DD]
csrf_token=<token>

Sub-formats are auto-detected by column-header signature:

Sub-format	Source tool
evtxecmd	EvtxECmd
mft-usn	MFTECmd (USN journal)
mft-full	MFTECmd (full MFT)
prefetch-timeline	PECmd Timeline CSV
prefetch-full	PECmd Output CSV
appcompat	AppCompatCacheParser
amcache-unassoc	AmcacheParser (unassociated)
amcache-prog	AmcacheParser (programs)
recycle-bin	RBCmd
jumplist	JLECmd
lnk	LECmd

JumpList must be detected before LNK — the JumpList CSV is a superset of LECmd columns and would be mis-classified if the order were reversed.

csv.field_size_limit is set to sys.maxsize at module load — PECmd's Files / Directories cells routinely exceed Python's 128 KB default limit.

AppCompatCache forensic note: shimcache indicates the file existed, not that it executed. LastModifiedTimeUTC is MFT $STANDARD_INFORMATION ModifiedTimeUTC, not a last-run time. The parser emits <basename> present on disk titles (no "executed" suffix), severity stays None. Use Prefetch / Security 4688 / Sysmon 1 for execution confirmation.

PECmd prefetch folding: prefetch-full (PECmd_Output.csv) emits one event per prefetch keyed on LastRun, with previous run timestamps listed in the body. This avoids flooding the panel with 9 events per prefetch entry. Use prefetch-timeline if you want one event per run.

Master-timeline CSV → working timeline

The "Upload CSV of events" modal now offers a Master timeline / Working timeline radio. Choosing Working timeline routes to:

POST /api/v2/cases/<cid>/working-timeline/import/master-csv
Content-Type: multipart/form-data

file=<iris-export.csv>
[begin_date=YYYY-MM-DD]
[end_date=YYYY-MM-DD]
csrf_token=<token>

The canonical 10-column export format is accepted: event_date,event_tz,event_title,event_category,event_content,event_raw,event_source,event_assets,event_iocs,event_tags

Assets and IOCs are not auto-linked at import — promote individually to trigger resolution.

Date-range filter

All three import endpoints accept optional begin_date / end_date (YYYY-MM-DD). Filtering is server-side with inclusive UTC day bounds (begin → 00:00:00, end → 23:59:59.999999).

Rows with unparseable timestamps (event_date is None) are always kept — a filter must never silently swallow unplaceable data.

Reviewing events

Each working-timeline card shows:

• Event timestamp, title, sigma rule match (Hayabusa), channel/EventID
• ✨ Explain pill — LLM 3-paragraph explanation (what it detects / what happened / triage hint)
• Promote / Reject / Reset buttons

Explain pill 3-state toggle:

1. First click — fetches cached explanation (or generates if not cached)
2. Second click — hides (artifact stays cached)
3. Third+ click — re-renders from in-memory cache, no API call

The list endpoint bulk-joins cached explanations so they remain visible after promote/reject without N+1 follow-up GETs.

Promoting events

POST /api/v2/cases/<cid>/working-timeline/events/<eid>/promote

At promote time (not import time):

1. Asset materialization — asset_resolver.py auto-creates CaseAssets rows for the host computer and subject/target users
   • Windows Computer/Server/DC heuristic by hostname
   • Windows Account AD/Local heuristic by domain presence
   • asset_domain populated from FQDN (bare hostnames and IPv4-prefixed strings → None)
2. AI IOC extraction — ioc_resolver.py calls the IOC extractor (confidence ≥ 0.70, noise-flagged candidates dropped), find-or-creates on (case_id, type_id, value)
3. Cross-links — update_event_assets(..., sync_iocs_assets=True) creates case_events_assets and IocAssetLink rows

Promoted events arrive on the master timeline flagged (event_is_flagged=True) — the flag is a "needs review" marker. Unflag once reviewed.

ISO timestamp serialization

Working timeline events store event_date as a naive datetime representing UTC.

The serializer appends Z (_iso_utc() in working_timeline.py) so browsers parse it as UTC rather than local time. Without the Z, new Date(iso) interprets the string as local time, and .getUTCHours() adds the offset on display.

This same latent bug exists on master-timeline cases_events.event_date — watch for it if you add new ingest sources that store naive non-UTC timestamps.

Master-timeline flag-by-default

New master-timeline events (manual add or promote from working) arrive with event_is_flagged = True. The flag means "needs review"; analysts uncheck it once verified. The running AI analysis panel treats:

• is_flagged: false → HIGH confidence (reviewed, treat as fact)
• is_flagged: true → MEDIUM confidence (provisional, hedge in narrative)

Event history

A History item in the event modal's ⋮ dropdown shows the event's full lifecycle from CasesEvent.modification_history JSONB. Lifecycle points recorded:

• created (manual)
• promoted from working timeline (<source>)
• flagged / unflagged

Adding a new ingest source

1. Create source/app/iris_engine/working_timeline/<source>_parser.py
   • Return a list of CaseWorkingEvent objects
   • Split Windows paths on [\\/] not os.path.basename()
   • Fold repeated-execution records (one card + secondaries in body, not N cards)
2. Add an IMPORT_SOURCES JS entry and a dropdown item in the import modal
3. Add a new /import/<source> endpoint in working_timeline.py
   • Reuse _read_date_window / _filter_rows_by_window for the date-range filter
   • Set working_event.source to a descriptive discriminator string

The case_working_event.source column already carries the discriminator for filtering and display.