fix(auto-triage): paginated fetch prevents queue stall on high-score …

Packs/soc-optimization-unified/DC_QUICKSTART.md

@@ -0,0 +1,107 @@
+# SOC Framework PoV — DC Quick Start
+
+> End-to-end setup in under 30 minutes. Follow in order.
+
+---
+
+## Phase 1 — Install SOC Framework
+
+- [ ] Install `soc-optimization-unified` pack (Foundation, Universal Command, Lists, Dashboards)
+- [ ] Install `soc-framework-nist-ir` pack (NIST IR lifecycle playbooks)
+- [ ] Install vendor pack(s) for customer data sources:
+  - `SocFrameworkCrowdstrikeFalcon` — if customer has CrowdStrike
+  - `SocFrameworkProofPointTap` — if customer has Proofpoint TAP
+- [ ] Run `!SOCFWPackManager action=apply` to configure lists, lookups, and integration instances
+- [ ] Verify correlation rules are enabled: **Detection Rules → Correlation**
+- [ ] Verify shadow mode is ON: `SOCFrameworkActions_V3` list — all actions have `shadow_mode: true`
+
+---
+
+## Phase 2 — Install PoV Test Pack
+
+- [ ] In XSIAM: **Settings → Data Sources → Add Data Source → HTTP Collector**
+  - Create **CrowdStrike** collector: Vendor = `CrowdStrike`, Product = `Falcon_Event`
+  - Create **Proofpoint TAP** collector: Vendor = `Proofpoint`, Product = `TAP`
+  - Copy the **URL** and **API Key** for each — you need them in the next step
+- [ ] Install `soc-framework-pov-test` pack via SDK upload
+- [ ] **Settings → Integrations → SOCFWPoVSender** — configure both instances:
+  - `socfw_pov_crowdstrike_sender` → paste CrowdStrike collector URL + API Key
+  - `socfw_pov_tap_sender` → paste TAP collector URL + API Key
+  - Click **Test** on each — must return `ok`
+- [ ] **Automation → Jobs → POV Teardown Reminder V1** → set schedule to last day of PoV
+
+---
+
+## Phase 3 — Run the Attack Scenario
+
+- [ ] Open the playground or any case war room
+- [ ] Run CrowdStrike (138 endpoint detection events):
+  ```
+  !SOCFWPoVSend list_name="SOCFWPoVData_CrowdStrike_TurlaCarbon_V1"
+    instance_name="socfw_pov_crowdstrike_sender"
+    source_name="crowdstrike"
+    global_min="2025-12-02T13:00:00Z"
+    global_max="2025-12-04T12:01:07Z"
+  ```
+- [ ] Run Proofpoint TAP (2 email threat events):
+  ```
+  !SOCFWPoVSend list_name="SOCFWPoVData_TAP_TurlaCarbon_V1"
+    instance_name="socfw_pov_tap_sender"
+    source_name="proofpoint"
+    global_min="2025-12-02T13:00:00Z"
+    global_max="2025-12-04T12:01:07Z"
+  ```
+- [ ] Watch **Cases & Issues → Cases** — cases appear within seconds
+- [ ] Confirm cross-source grouping: one case showing both email delivery + endpoint detections
+- [ ] Confirm SOC Framework ran: case war room shows Foundation → Analysis → AI narrative
+
+---
+
+## Phase 4 — Show the Value Dashboard
+
+- [ ] Navigate to **Dashboards → XSIAM SOC Value Metrics**
+- [ ] Walk the customer through the 4 Value Drivers:
+  - **VD1** — MTTD, MTTC, MTTR (detection and response speed)
+  - **VD2** — Shadow mode actions (what automation would have done)
+  - **VD3** — Analyst time saved, automation percentage
+  - **VD4** — Attack surface risk reduction, recovery validation
+- [ ] Show the AI narrative in the case — cross-vendor correlation across TAP + CrowdStrike
+- [ ] Show the NIST IR lifecycle running in shadow mode — containment, eradication, recovery logged but not executed
+- [ ] Explain the 1-flip production path: `shadow_mode = false` per action in `SOCFrameworkActions_V3`
+
+---
+
+## Phase 5 — Teardown (Before PS Handoff)
+
+- [ ] Run `!SOCFWPackManager action=apply` on the production tenant (Skynet) with PS
+- [ ] On the PoV tenant, uninstall `soc-framework-pov-test` from Marketplace
+- [ ] Delete both HTTP Collectors: **Settings → Data Sources → socfw_pov_crowdstrike / socfw_pov_tap → Delete**
+- [ ] Set `shadow_mode = false` per action in `SOCFrameworkActions_V3` to enable live execution
+- [ ] Help PS onboard the customer's real data sources ASAP:
+  - CrowdStrike Falcon integration (native Marketplace pack)
+  - Proofpoint TAP integration (native Marketplace pack)
+  - Any additional sources per customer environment
+- [ ] Verify correlation rules fire on real data before leaving the customer
+- [ ] Close the POV Teardown Reminder case
+
+---
+
+## Replay anytime
+
+To re-run the scenario (suppression IDs rotate automatically on each run):
+
+```
+!SOCFWPoVSend list_name="SOCFWPoVData_CrowdStrike_TurlaCarbon_V1"
+  instance_name="socfw_pov_crowdstrike_sender"
+  source_name="crowdstrike"
+  global_min="2025-12-02T13:00:00Z"
+  global_max="2025-12-04T12:01:07Z"
+```
+
+```
+!SOCFWPoVSend list_name="SOCFWPoVData_TAP_TurlaCarbon_V1"
+  instance_name="socfw_pov_tap_sender"
+  source_name="proofpoint"
+  global_min="2025-12-02T13:00:00Z"
+  global_max="2025-12-04T12:01:07Z"
+```

Packs/soc-optimization-unified/PRE_CONFIG_README.md

@@ -0,0 +1,78 @@
+# Pre-Configuration — SOC Framework PoV Test Pack
+
+Complete these steps **before** installing this pack.
+
+---
+
+## Step 1 — Verify SOC Framework is installed
+
+Confirm the following are installed and correlation rules are enabled:
+
+- `soc-optimization-unified` — Foundation layer, lists, dashboards
+- `soc-framework-nist-ir` — NIST IR lifecycle playbooks
+- `SocFrameworkCrowdstrikeFalcon` — CrowdStrike correlation rule
+- `SocFrameworkProofPointTap` — Proofpoint TAP correlation rule
+
+In **Detection Rules → Correlation** confirm both rules show **Enabled**.
+
+---
+
+## Step 2 — Create HTTP Collectors
+
+In XSIAM: **Settings → Data Sources → Add Data Source → HTTP Collector**
+
+Create one collector per data source. Use these exact values — vendor and product
+determine which dataset events are written to.
+
+### Collector 1 — CrowdStrike Falcon
+
+| Field | Value |
+|---|---|
+| Name | `socfw_pov_crowdstrike` |
+| Vendor | `CrowdStrike` |
+| Product | `Falcon_Event` |
+| Dataset | `crowdstrike_falcon_event_raw` (auto-created on first event) |
+
+### Collector 2 — Proofpoint TAP
+
+| Field | Value |
+|---|---|
+| Name | `socfw_pov_tap` |
+| Vendor | `Proofpoint` |
+| Product | `TAP` |
+| Dataset | `proofpoint_tap_v2_generic_alert_raw` (auto-created on first event) |
+
+After saving each collector: copy the **Endpoint URL** and **API Key**.
+You will paste these into the integration instances after pack install.
+
+---
+
+## Step 3 — Install the pack
+
+Upload `soc-framework-pov-test` via SDK:
+
+```bash
+bash tools/upload_package.sh Packs/soc-framework-pov-test
+```
+
+---
+
+## Step 4 — Configure integration instances
+
+Go to **Settings → Integrations → SOCFWPoVSender**.
+
+Two instances are created automatically. Configure each:
+
+**`socfw_pov_crowdstrike_sender`**
+- HTTP Collector URL → paste CrowdStrike collector URL
+- API Key → paste CrowdStrike collector API key
+- Source Name → `crowdstrike` (pre-filled — do not change)
+- Click **Test** → must return `ok`
+
+**`socfw_pov_tap_sender`**
+- HTTP Collector URL → paste TAP collector URL
+- API Key → paste TAP collector API key
+- Source Name → `proofpoint` (pre-filled — do not change)
+- Click **Test** → must return `ok`
+
+See `POST_CONFIG_README.md` to run the scenario.

Packs/soc-optimization-unified/Playbooks/JOB_-_Triage_Alerts_V3.yml → Packs/soc-optimization-unified/Playbooks/JOB_-_Auto_Triage_V3.yml

@@ -1,8 +1,9 @@
+adopted: true
 fromversion: 5.0.0
 id: JOB - Auto Triage V3
-version: 7
+version: -1
 contentitemexportablefields:
-  contentitemfields:
+  '13':
     packID: soc-optimization-unified
     packName: SOC Framework Unified
     itemVersion: 3.7.3
@@ -12,7 +13,6 @@ contentitemexportablefields:
     prevname: ""
     isoverridable: false
     supportedModules: []
-vcShouldKeepItemLegacyProdMachine: false
 name: JOB - Auto Triage V3
 tags:
   - SOC
@@ -31,61 +31,16 @@ tasks:
       brand: ""
       playbooktaskmissingcomponent: null
       istaskmissingcomponenterrordismissed: false
-    nexttasks:
-      '#none#':
-        - "1"
-    separatecontext: false
-    continueonerrortype: ""
-    view: |-
-      {
-        "position": {
-          "x": 50,
-          "y": 50
-        }
-      }
-    note: false
-    timertriggers: []
-    ignoreworker: false
-    skipunavailable: false
-    quietmode: 0
-    isoversize: false
-    isautoswitchedtoquietmode: false
-  "1":
-    id: "1"
-    taskid: 993570fa-2548-4e9c-af63-478189b6c22b
-    type: regular
-    task:
-      id: 993570fa-2548-4e9c-af63-478189b6c22b
-      version: -1
-      name: Get Unstarred Open Cases
-      description: Queries get_incidents — starred=false, status new, sorted oldest-first.
-        Fetches up to 100 cases per run (API maximum). Age window and score filtering
-        handled in SOCAutoTriageScoreFilter. Run this JOB frequently (e.g., every 15m)
-        to drain backlogs exceeding 100 cases across successive executions.
-      script: '|||core-api-post'
-      type: regular
-      iscommand: true
-      brand: ""
-      playbooktaskmissingcomponent: null
-      istaskmissingcomponenterrordismissed: false
     nexttasks:
       '#none#':
         - "12"
-    scriptarguments:
-      body:
-        simple: '{"request_data":{"filters":[{"field":"status","operator":"eq","value":"new"},{"field":"starred","operator":"eq","value":false}],"fields":["incident_id","aggregated_score","creation_time","status","starred"],"sort":{"field":"creation_time","keyword":"asc"},"search_from":0,"search_to":100}}'
-      extend-context:
-        simple: Found=.
-      uri:
-        simple: /public_api/v1/incidents/get_incidents
     separatecontext: false
-    continueonerror: true
     continueonerrortype: ""
     view: |-
       {
         "position": {
           "x": 50,
-          "y": 220
+          "y": 50
         }
       }
     note: false
@@ -130,7 +85,7 @@ tasks:
       {
         "position": {
           "x": 50,
-          "y": 775
+          "y": 420
         }
       }
     note: false
@@ -159,7 +114,7 @@ tasks:
       {
         "position": {
           "x": 50,
-          "y": 1515
+          "y": 1160
         }
       }
     note: false
@@ -205,7 +160,7 @@ tasks:
       {
         "position": {
           "x": 162.5,
-          "y": 960
+          "y": 610
         }
       }
     note: false
@@ -239,7 +194,7 @@ tasks:
       {
         "position": {
           "x": 50,
-          "y": 1330
+          "y": 980
         }
       }
     note: false
@@ -256,11 +211,13 @@ tasks:
     task:
       id: ba55380b-2e2c-44ef-8fcf-d828bfbfa09a
       version: -1
-      name: Filter Cases by Score Threshold
-      description: Filters API response by aggregated_score <= TriageScoreThreshold
-        and manual_score is null. Cases above threshold or analyst-touched are skipped.
-        Passes only eligible cases to the close loop.
-      scriptName: SOCAutoTriageScoreFilter
+      name: Fetch and Filter Cases by Score Threshold
+      description: Fetches unstarred new cases via core-api-post in paginated batches
+        of 100, sorted by creation_time asc. Skips cases with aggregated_score above
+        TriageScoreThreshold. Applies age window filter (TriageWindowHours) and skips
+        analyst-touched cases. Stops when eligible cases are found or max_batches
+        reached. Passes only eligible cases to the close loop.
+      script: SOCAutoTriageScoreFilter
       type: regular
       iscommand: false
       brand: ""
@@ -270,10 +227,6 @@ tasks:
       '#none#':
         - "5"
     scriptarguments:
-      incidents:
-        complex:
-          root: Found.response.reply
-          accessor: incidents
       score_threshold:
         complex:
           root: lists
@@ -314,14 +267,16 @@ tasks:
                 field:
                   value:
                     simple: TriageWindowHours
+      max_batches:
+        simple: "5"
     separatecontext: false
     continueonerror: true
     continueonerrortype: ""
     view: |-
       {
         "position": {
           "x": 50,
-          "y": 590
+          "y": 220
         }
       }
     note: false
@@ -339,7 +294,7 @@ view: |-
     },
     "paper": {
       "dimensions": {
-        "height": 1525,
+        "height": 1170,
         "width": 492.5,
         "x": 50,
         "y": 50
@@ -348,5 +303,3 @@ view: |-
   }
 inputs: []
 outputs: []
-dirtyInputs: true
-adopted: true