Skip to content
Original file line number Diff line number Diff line change
@@ -1,5 +1,5 @@
- Data collection endpoint (DCE)
- The DCE endpoint URL is shown as the **Logs Ingestion Endpoint** or **Data Collection Endpoint** on the DCR Overview page. An example URL: `https://<DCE-ID>.ingest.monitor.azure.com/dataCollectionRules/<DCR-Immutable-ID>/streams/<Stream-Name>?api-version=2023-01-01`.
- The DCE endpoint URL is shown as the **Logs Ingestion Endpoint** or **Data Collection Endpoint** on the DCR Overview page. An example URL: `https://<DCE-ID>.ingest.monitor.azure.com`.
- Stored as the environment variable `DD_OP_DESTINATION_MICROSOFT_SENTINEL_DCE_URI`
- Client secret
- This is the Azure AD application's client secret, such as `550e8400-e29b-41d4-a716-446655440000`.
Expand Down
Original file line number Diff line number Diff line change
@@ -1,38 +1,60 @@
#### Prerequisites

To set up the Microsoft Sentinel destination, you need the following information:
To set up the Microsoft Sentinel destination, you need to create a Workspace in Azure if you haven't already. In that workspace:
1. [Add Microsoft Sentinel][10163] to the workspace.
1. [Create a Data Collection Endpoint (DCE)][10164].
1. [Create a Log Analytics Workspace][10165] in the workspace if you haven't already.
1. In the Log Analytics Workspace, navigate to **Settings** > **Tables**.
1. Click **+ Create**.
1. Define a custom table (for example, `Custom-MyLogs_CL`).
- **Notes**:<br>- For custom tables, the table name must start with `Custom-`. `CL` is automatically appended to the end of the table name. You need the table name to set up the Observability Pipelines Microsoft Sentinel destination.<br>- You can also use an Azure Table instead of a custom table.
1. Select **New Custom Log (DCR-based)**.
1. Click **Create a new data collection rule** and select the DCE you created earlier.
1. Click **Next**.
1. Upload a sample JSON Log. For this example, the following JSON is used for the **Schema and Transformation**, where `TimeGenerated` is required:
```json
{
"TimeGenerated": "2024-07-22T11:47:51Z",
"event": {}
}
```
1. Click **Create**.
1. In Azure, navigate to **Microsoft Entra ID**.
1. Click **Add** > **App Registration**.
1. Click **Create**.
1. On the overview page, click **Client credentials: Add a certificate or secret**.
1. Click **New client secret**.
1. Enter a name for the secret and click **Add**. **Note**: Make sure to take note of the client secret, which gets obfuscated after 10 minutes.
1. Also take note of the **Tenant ID** and **Client ID**. You need this information, along with the client secret, when you [set up the Observability Pipelines Microsoft Sentinel destination](#set-up-the-destination-in-observability-pipelines).
1. In Azure Portal's [Data Collection Rules][10166] page, search for and select the DCR you created earlier.
1. Click **Access Control (IAM)** in the left nav.
1. Click **Add** and select **Add role assignment**.
1. Add the **Monitoring Metrics Publisher** role.
1. On the Members page, select **User, group, or service principal**.
1. Click **Select Members** and search for the application you created in the app registration step.
1. Click **Review + Assign**. **Note**: It can take up to 10 minutes for the IAM change to take effect.

The table below summarizes the Azure and Microsoft Sentinel information you need when you [set up the Observability Pipelines Microsoft Sentinel destination](#set-up-the-destination-in-observability-pipelines):

| Name | Description |
|------------------------------------|---------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------|
| Application (client) ID | The Azure Active Directory (AD) application's client ID. See [Register an application in Microsoft Entra ID][10161] for information on creating a new application.<br>**Example**: `550e8400-e29b-41d4-a716-446655440000` |
| Directory (tenant) ID | The Azure AD tenant ID. See [Register an application in Microsoft Entra ID][10161] for information on creating a new application.<br>**Example**: `72f988bf-86f1-41af-91ab-2d7cd011db47` |
| Table (Stream) Name | The name of the stream which matches the table chosen when configuring the Data Collection Rule (DCR).<br>**Example**: `Custom-MyLogs_CL` |
| Application (client) ID | The Azure Active Directory (AD) application's client ID. See [Register an application in Microsoft Entra ID][10161] for more information.<br>**Example**: `550e8400-e29b-41d4-a716-446655440000` |
| Directory (tenant) ID | The Azure AD tenant ID. See [Register an application in Microsoft Entra ID][10161] for more information.<br>**Example**: `72f988bf-86f1-41af-91ab-2d7cd011db47` |
| Table (Stream) Name | The name of the stream which matches the table chosen when configuring the Data Collection Rule (DCR). **Note**: The table name must start with `Custom-`. `CL` is automatically appended to the end of the table name.<br>**Example**: `Custom-MyLogs_CL` |
| Data Collection Rule (DCR) immutable ID | This is the immutable ID of the DCR where logging routes are defined. It is the **Immutable ID** shown on the DCR Overview page.<br>**Note**: Ensure the Monitoring Metrics Publisher role is assigned in the DCR IAM settings.<br>**Example**: `dcr-000a00a000a00000a000000aa000a0aa`<br>See [Data collection rules (DCRs) in Azure Monitor][10162] to learn more about creating or viewing DCRs. |

Do the following to get that information:

1. Create or identify a Data Collection Rule (DCR).
1. In the Azure Portal, navigate to **Azure Monitor** → **Data Collection Rules**.
1. Create a DCR or use an existing one for custom logs. See [Data collection rules (DCRs) in Azure Monitor][10162] to learn more about creating or viewing DCRs.
1. Take note of the DCR Immutable ID and, if you are using private links, the DCR's Data Collection Endpoint (DCE). You need this information when you set up the Microsoft Sentinel destination.
1. Define a custom table (for example, `Custom-MyLogs_CL`) in the DCR, which is where Observability Pipelines sends logs to.
1. Get the ingestion URL.
1. In the DCR, locate the **Logs Ingestion API endpoint**. The endpoint has the format: `https://<DCE-ID>.ingest.monitor.azure.com/dataCollectionRules/<DCR-Immutable-ID>/streams/<Stream-Name>?api-version=2023-01-01`, where the `<Stream-Name>` typically matches your custom table (for example, `Custom-MyLogs_CL`).
1. The ingestion URL is needed when you set up you Microsoft Sentinel destination's environment variable.
1. To authenticate the Observability Pipelines Worker with Microsoft Sentinel:
1. In the Azure Portal, navigate to **Azure AD** > **App Registrations** and register an Azure Active Directory (AD) application. See [Register an application in Microsoft Entra ID][10161] for information on creating a new application.
1. Generate a **Client Secret**.
1. Assign it the **Monitoring Metrics Publisher** role on the Log Analytics workspace
1. Take note of the **Tenant ID**, **Client ID**, and **Client Secret**. You need this information when you set up the Microsoft Sentinel destination.

#### Set up destination in Observability Pipelines
#### Set up the destination in Observability Pipelines

To set up the Microsoft Sentinel destination in Observability Pipelines:

1. Enter the client ID for your application, such as `550e8400-e29b-41d4-a716-446655440000`.
1. Enter the directory ID for your tenant, such as `72f988bf-86f1-41af-91ab-2d7cd011db47`. This is the Azure AD tenant ID.
1. Enter the name of the table, such as `Custom-MyLogs`, to which you are sending logs.
1. Enter the name of the table to which you are sending logs. An example table name: `Custom-MyLogs_CL`.
1. Enter the Data Collection Rule (DCR) immutable ID, such as `dcr-000a00a000a00000a000000aa000a0aa`.

[10161]: https://learn.microsoft.com/en-us/entra/identity-platform/quickstart-register-app?tabs=certificate%2Cexpose-a-web-api
[10162]: https://learn.microsoft.com/en-us/azure/azure-monitor/essentials/data-collection-rule-overview
[10162]: https://learn.microsoft.com/en-us/azure/azure-monitor/essentials/data-collection-rule-overview
[10163]: https://portal.azure.com/#browse/microsoft.securityinsightsarg%2Fsentinel
[10164]: https://portal.azure.com/#view/HubsExtension/BrowseResource.ReactView/resourceType/microsoft.insights%2Fdatacollectionendpoints
[10165]: https://portal.azure.com/#create/Microsoft.LogAnalyticsOMS
[10166]: https://portal.azure.com/#view/HubsExtension/BrowseResource.ReactView/resourceType/microsoft.insights%2Fdatacollectionrules
Loading