[Darktrace] Initial Release for the Darktrace

.github/CODEOWNERS

@@ -54,6 +54,7 @@
 /packages/cyberark @elastic/security-external-integrations
 /packages/cyberarkpas @elastic/security-external-integrations
 /packages/cylance @elastic/security-external-integrations
+/packages/darktrace @elastic/security-external-integrations
 /packages/dga @elastic/ml-ui
 /packages/docker @elastic/obs-cloudnative-monitoring
 /packages/elastic_agent @elastic/elastic-agent-control-plane

packages/darktrace/_dev/build/build.yml

@@ -0,0 +1,3 @@
+dependencies:
+  ecs:
+    reference: git@v8.4.0-rc1

packages/darktrace/_dev/build/docs/README.md

@@ -0,0 +1,138 @@
+# Darktrace
+
+## Overview
+
+The [Darktrace](https://darktrace.com/) integration allows you to monitor Alert Logs. Darktrace is a network solution for detecting and investigating emerging cyber-threats that evade traditional security tools. It is powered by Enterprise Immune System technology, which uses machine learning and mathematics to monitor behaviors and detect anomalies in your organization’s network.
+
+Use the Darktrace integration to collect and parse data from the REST APIs or via Syslog. Then visualise that data in Kibana.
+
+For example, you could use the data from this integration to know which model is breached and analyse model breaches, and also know about system health, changes in monitored traffic, and any errors experienced by Darktrace Security Modules or probe instances.
+
+## Data streams
+
+The Darktrace integration collects logs for three types of events: AI Analyst Alert, Model Breach Alert and System Status Alert.
+
+**AI Analyst Alert** is generated by investigates, analyzes, and reports upon threats seen within your Darktrace environment; as a starting point, it reviews and investigates all Model Breaches that occur on the system. If behavior which would be of interest to a cyber analyst is detected, an event is created. See Example Schema [here](https://customerportal.darktrace.com/product-guides/main/syslog-aia-json-schema).
+
+**Model Breach Alert** is generated when a model breach is triggered. A model is used to define a set of conditions which, when met, will alert the system to the occurrence of a particular event or chain of anomalous behavior. Darktrace models are focused on pattern-of-life anomaly detection, potentially malicious behavior, and compliance issues. See Example Schema [here](https://customerportal.darktrace.com/product-guides/main/syslog-json-schema).
+
+**System Status Alert** keep Darktrace operators informed of system health, changes in monitored traffic, and any errors experienced by Darktrace Security Modules or probe instances. System Status Alerts include details of the originating host, the severity of the event, and links that may be helpful to investigate or resolve the issue. Notifications are sent for active system events and (optionally) on event resolution. See Example Schema [here](https://customerportal.darktrace.com/product-guides/main/syslog-json-system-schema).
+
+## Requirements
+
+You need Elasticsearch for storing and searching your data and Kibana for visualizing and managing it. You can use our hosted Elasticsearch Service on Elastic Cloud, which is recommended, or self-manage the Elastic Stack on your own hardware.
+
+Firewall exceptions to allow communication from the Darktrace master instance to the Syslog server.
+
+This module has been tested against **Darktrace Threat Visualizer v5.2**.
+
+## Setup
+
+### To collect data from Darktrace REST APIs, follow the below steps:
+
+1. Hostname URL will be your <appliance-ip>. (Threat Visualizer Console Hostname)
+2. Public and Private Token will be generated by following this [Link](https://customerportal.darktrace.com/product-guides/main/api-tokens).
+
+**Note:** System Status Alert are not supported by REST API.
+
+### To collect data from Darktrace via Syslog, follow the below steps:
+
+The user needs to create a different Syslog Forwarder with different ports for each data stream.
+
+The process for configuring syslog-format alerts is identical for AI Analyst Alerts, Model Breach Alerts and System Status Alerts. Generic configuration guidance is provided below:
+
+1. Open the Darktrace Threat Visualizer Dashboard and navigate to the **System Config** page. (**Main menu › Admin**).
+2. From the left-side menu, select **Modules**, then navigate to the **Workflow Integrations** section and choose **Syslog**.
+3. Select **Syslog JSON** tab and click **New** to set up new Syslog Forwarder.
+4. Enter the **IP Address**  and **Port** of the Elastic Agent that is running the integration in the **Server** and **Server Port** field respectively.
+
+For more details, see [Documentation](https://customerportal.darktrace.com/product-guides/main/json-alerts).
+
+**Note:**
+  - It is recommended to turn on **Full Timestamps** toggle in **Show Advanced Options** to get the full timestamp instead of the RFC3164-formatted timestamp.
+  - It is also recommended to turn off **Reduced Message Size** toggle in **Show Advanced Options** to get more information about alerts.
+
+### After following generic guidance steps, below are the steps for collecting individual logs for all three data streams.
+
+#### For AI Analyst Alert, below are the suggested configurations to collect all the events of AI Analyst Alert:
+
+- Configure the following settings in **Show Advanced Options**:
+
+| Field Name                              | Value                               |
+| --------------------------------------- | ----------------------------------- |
+| Send AI Analyst Alerts                  | ON                                  |
+| Send AI Analyst Alerts Immediately      | ON                                  |
+| AI Analyst Behavior Filter              | Critical, Suspicious and Compliance |
+| Minimum AI Analyst Incident Event Score | 0                                   |
+| Minimum AI Analyst Incident Score       | 0                                   |
+| Legacy AI Analyst Alerts                | OFF                                 |
+
+#### For Model Breach Alert, below are the suggested configurations to collect all the events of Model Breach Alert:
+
+- Configure the following settings in **Show Advanced Options**:
+
+| Field Name                   | Value                                              |
+| ---------------------------- | -------------------------------------------------- |
+| Send Model Breach Alerts     | ON                                                 |
+| Model Breach Behavior Filter | Critical, Suspicious, Compliance and Informational |
+| Minimum Breach Score         | 0                                                  |
+| Minimum Breach Priority      | 0                                                  |
+| Model Expression             | N/A                                                |
+| Model Tags Expression        | N/A                                                |
+| Device IP Addresses          | N/A                                                |
+| Device Tags Addresses        | N/A                                                |
+
+#### For System Status Alert, below are the suggested configurations to collect all the events of System Status Alert:
+
+- Configure the following settings in **Show Advanced Options**:
+
+| Field Name                         | Value         |
+| ---------------------------------- | ------------- |
+| Send System Status Alerts          | ON            |
+| Send Resolved System Status Alerts | ON            |
+| Minimum System Status Priority     | Informational |
+
+### See more about [Syslog Filters and Optional Settings](https://customerportal.darktrace.com/product-guides/main/syslog-json-alert-settings)
+
+**Note** : A Fully Qualified Domain Name (FQDN) must be configured for the Darktrace instance in order for links to be included in external alerts.
+  - An FQDN can be configured from the **System** subsection on the **Settings** tab of the Darktrace **System Config** page.
+
+### Enabling the integration in Elastic
+
+1. In Kibana go to **Management > Integrations**.
+2. In the "Search for integrations" search bar, type **Darktrace**.
+3. Click on **Darktrace** integration from the search results.
+4. Click on **Add Darktrace** button to add Darktrace integration.
+5. Enable the Integration with either via API or TCP or UDP input.
+
+## Logs reference
+
+### ai_analyst_alert
+
+This is the `ai_analyst_alert` dataset.
+
+#### Example
+
+{{event "ai_analyst_alert"}}
+
+{{fields "ai_analyst_alert"}}
+
+### model_breach_alert
+
+This is the `model_breach_alert` dataset.
+
+#### Example
+
+{{event "model_breach_alert"}}
+
+{{fields "model_breach_alert"}}
+
+### system_status_alert
+
+This is the `system_status_alert` dataset.
+
+#### Example
+
+{{event "system_status_alert"}}
+
+{{fields "system_status_alert"}}

packages/darktrace/_dev/deploy/docker/docker-compose.yml

@@ -0,0 +1,60 @@
+version: '2.3'
+services:
+  darktrace-ai_analyst_alert-tls:
+    image: docker.elastic.co/observability/stream:v0.7.0
+    volumes:
+      - ./sample_logs:/sample_logs:ro
+    command: log --start-signal=SIGHUP --delay=5s --addr elastic-agent:9571 -p=tls --insecure /sample_logs/ai_analyst_alert.log
+  darktrace-ai_analyst_alert-tcp:
+    image: docker.elastic.co/observability/stream:v0.7.0
+    volumes:
+      - ./sample_logs:/sample_logs:ro
+    command: log --start-signal=SIGHUP --delay=5s --addr elastic-agent:9571 -p=tcp /sample_logs/ai_analyst_alert.log
+  darktrace-ai_analyst_alert-udp:
+    image: docker.elastic.co/observability/stream:v0.7.0
+    volumes:
+      - ./sample_logs:/sample_logs:ro
+    command: log --start-signal=SIGHUP --delay=5s --addr elastic-agent:9574 -p=udp /sample_logs/ai_analyst_alert.log
+  darktrace-model_breach_alert-tls:
+    image: docker.elastic.co/observability/stream:v0.7.0
+    volumes:
+      - ./sample_logs:/sample_logs:ro
+    command: log --start-signal=SIGHUP --delay=5s --addr elastic-agent:9572 -p=tls --insecure /sample_logs/model_breach_alert.log
+  darktrace-model_breach_alert-tcp:
+    image: docker.elastic.co/observability/stream:v0.7.0
+    volumes:
+      - ./sample_logs:/sample_logs:ro
+    command: log --start-signal=SIGHUP --delay=5s --addr elastic-agent:9572 -p=tcp /sample_logs/model_breach_alert.log
+  darktrace-model_breach_alert-udp:
+    image: docker.elastic.co/observability/stream:v0.7.0
+    volumes:
+      - ./sample_logs:/sample_logs:ro
+    command: log --start-signal=SIGHUP --delay=5s --addr elastic-agent:9575 -p=udp /sample_logs/model_breach_alert.log
+  darktrace-system_status_alert-tls:
+    image: docker.elastic.co/observability/stream:v0.7.0
+    volumes:
+      - ./sample_logs:/sample_logs:ro
+    command: log --start-signal=SIGHUP --delay=5s --addr elastic-agent:9573 -p=tls --insecure /sample_logs/system_status_alert.log
+  darktrace-system_status_alert-tcp:
+    image: docker.elastic.co/observability/stream:v0.7.0
+    volumes:
+      - ./sample_logs:/sample_logs:ro
+    command: log --start-signal=SIGHUP --delay=5s --addr elastic-agent:9573 -p=tcp /sample_logs/system_status_alert.log
+  darktrace-system_status_alert-udp:
+    image: docker.elastic.co/observability/stream:v0.7.0
+    volumes:
+      - ./sample_logs:/sample_logs:ro
+    command: log --start-signal=SIGHUP --delay=5s --addr elastic-agent:9576 -p=udp /sample_logs/system_status_alert.log
+  darktrace:
+    image: docker.elastic.co/observability/stream:v0.7.0
+    hostname: darktrace
+    ports:
+      - 8080
+    volumes:
+      - ./files:/files:ro
+    environment:
+      PORT: "8080"
+    command:
+      - http-server
+      - --addr=:8080
+      - --config=/files/config.yml