Deploy Function App for getting Office 365 Management API data into Azure Sentinel

This function app will poll O365 Activity Managment API every 5 mins for logs. It is designed to get Audit.General and DLP.All events.

How to Ingest Office 365 Audit.General and DLP.All Activity Logs into Azure Sentinel

The Office 365 data connector in Azure Sentinel supports ongoing user and admin activity logs for Microsoft 365 workloads, Exchange Online, SharePoint Online and Microsoft Teams. The activity logs include details of action such as file downloads, access request send, change to group event, mailbox operations. Once the activity logs are ingested into Azure Sentinel, it can be used for custom analytics rules, hunting, visualization as well as for investigation process.

The Azure Sentinel data connector for Office 365 uses the Office 365 Activity Management API. Below is a summary of which content types are part of the Office 365 Activity Management API and their mapping with Azure Sentinel.

Content Type	Description	Azure Sentinel Mapping
Audit.AzureActiveDirectory	Azure Active Directory logs that’s relates to Office 365 only	Supported with the default connector for Office 365 in Azure Sentinel
Audit.Exchange	User and Admin Activities in Exchange Online	Supported with the default connector for Office 365 in Azure Sentinel
Audit.SharePoint	User and Admin Activities in SharePoint Online	Supported with the default connector for Office 365 in Azure Sentinel
Audit.General	Includes all other workloads not included in the previous content types	Not supported with the default connector for Office 365 in Azure Sentinel
DLP.All	DLP events only for all workloads	Not supported with the default connector for Office 365 in Azure Sentinel

Specifically, Audit.General activity logs could be of interest in SIEM if there is a need for correlation with alerts from Defender for Office 365 and alerts from Security and Compliance Center. As follow most asked use cases are:

• Usage of Security and Compliance Center alerts
• Alerts generated by Defender for Office 365
  • Safe Links time-of-block and block override
  • Phishing and malware alerts for files in SharePoint Online, OneDrive for Business, and Microsoft Teams
  • Usage of Phishing and malware events

This document covers the required steps to ingest Audit.General and DLP.All activity logs from the Office 365 Management Activity API into Azure Sentinel and how to use the ingested alerts. For the ingestion of activity logs I will use an Azure Function App connector.

The Azure Function App uses a PowerShell script to collect Office 365 Audit.General and DLP.All Activity logs and ingests into a custom table in Azure Sentinel (custom tables end with _CL when created in Log Analytics). The secrets for the required connections are stored in Azure Key Vault.

Let’s get started with the configuration!

Preparation

The following tasks describe the necessary preparation and configurations steps.

• Register an application in Azure AD
• Create an Office 365 Management Activity API Subscription
• Deploy the Azure Function App
• Post Configuration Steps for the Function App and Key Vault
• How to Use the Activity Logs in Azure Sentinel

Register an application in Azure AD

The Azure AD app is later required to use it as service principle for the Azure Funtion App app.

1. Go to Azure Active Directory / App Registrations
2. Create New Registration
   
   
3. Call it "O365APItoAzureSentinel". Click Register.
4. Click API Permissions Blade.
5. Click Add a Permission.
6. Click Office 365 Management APIs.
7. Click Appplication Permissions
8. Check ActivityFeed.Read and ActivityFeed.ReadDlp. Click Add permissions.
   
   
9. Click Grant admin consent for ....
   
   
10. Click Certificates and Secrets blade.
11. Click New Client Secret.
12. Enter a description, select never. Click Add.
    
    
13. IMPORTANT. Click copy next to the new secret and paste it somewhere temporaily. You can not come back to get the secret once you leave the blade.
14. Copy the client Id from the application properties and paste it somewhere.
15. Also copy the tenant Id from the AAD directory properties blade.

For the deployment of Azure Funtion App, make a note of following settings:

• The Azure AD Application ID
• The Azure AD Application Secret
• The Tenant ID
• The Tenant Domain

Create an Office 365 Management Activity API Subscription

After successfully creating the service principles, run the following PowerShell script to register the API subscription.

1. Open a PowerShell terminal.
2. Run the following, replacing variables with strings from the previous steps.

$ClientID = "<GUID> from AAD App Registration"
$ClientSecret = "<clientSecret> from AAD App Registrtion"
$loginURL = "https://login.microsoftonline.[com][us]/"
$tenantdomain = "<domain>.onmicrosoft.[com][us]"
$TenantGUID = "<tenantguid> from AAD"
$resource = "https://manage.office.[com][us]"
$body = @{grant_type="client_credentials";resource=$resource;client_id=$ClientID;client_secret=$ClientSecret}
$oauth = Invoke-RestMethod -Method Post -Uri $loginURL/$tenantdomain/oauth2/token?api-version=1.0 -Body $body
$headerParams = @{'Authorization'="$($oauth.token_type) $($oauth.access_token)"} 
$publisher = "<randomGuid>" Get a guid from https://guidgenerator.com/

3. Run this command to enable Audit.General Subscription.

Invoke-WebRequest -Method Post -Headers $headerParams -Uri "https://manage.office.[com][us]/api/v1.0/$tenantGuid/activity/feed/subscriptions/start?contentType=Audit.General&PublisherIdentifier=$Publisher"

4. Run this command to enable DLP.ALL subscription

Invoke-WebRequest -Method Post -Headers $headerParams -Uri "https://manage.office.[com][us]/api/v1.0/$tenantGuid/activity/feed/subscriptions/start?contentType=DLP.ALL&PublisherIdentifier=$Publisher"

5. A successful output looks like as below.
   
   

Deploy the Azure Function App

Thanks to the published ARM template the deployment of the Azure Funtion App is done with just a few clicks.

1. Click to Deploy the template / Deploy to Azure below.

2. Now it is time to use the noted details from previous steps.

• Select the right Subscription, Resource Group and Region where you what to deploy the Azure Funtion App.
• Fill the Instance Details Client ID, Client Secret, Tenant Domain, Publisher Guid.
• There is also a need of Workspace ID and Workspace Key from where Azure Sentinel is deployed.
• The Content Types you can leave as default with Audit.General, or you can also add DLP.All as well. Or use only DLP.All.

3. Click to Review + create, review the configuration and click Create.
4. Now the deployment of ARM template is completed.

How to use the Activity Logs in Azure Sentinel

Once the Azure Function App is functional you can query the General.Audit and DLP.All activity logs. The activity will reside in a Custom Table as configured in the Azure Function App above. The following table includes sample Kusto Language Queries (KQL). You can see these are using the Custom Logs (Custom log tables always end in “_CL”) and the values we mentioned earlier.

Note: Custom Logs are a billable data source. The record types that are important have been added below, as simple starting queries.

Member Name	Kusto Language Query (KQL)
ThreatIntelligence	O365_CL | where RecordType_d == "28"
ThreatIntelligenceUrl	O365_CL | where RecordType_d == "41"
ThreatIntelligenceAtpContent	O365_CL | where RecordType_d == "47"
SecurityComplianceAlerts	O365_CL | where RecordType_d == "40"
PowerBI Events	O365_CL | where RecordType_d == "20"
Power Automate	O365_CL | where RecordType_d == "30"
PowerApps	O365_CL | where RecordType_d == "45"

An example results for the Defender for Office Safe Attachment block detection alert.

O365_CL 
| where RecordType_d == "28" 

RecordTypes

Here is a list of Record IDs which may be in the audit timeline if that event has been generated
1 – Indicates a record from the Exchange admin audit log.
2 – Indicates a record from the Exchange mailbox audit log for an operation performed on a singled mailbox item.
3 – Also indicates a record from the Exchange mailbox audit log. This record type indicates that the operation was performed on multiple items in the source mailbox (such as moving multiple items to the Deleted Items folder or permanently deleting multiple items).
4 – Indicates a site admin operation in SharePoint, such as an administrator or user assigning permissions to a site.
6 – Indicates a file or folder-related operation in SharePoint, such as a user viewing or modifying a file.
8 – Indicates an admin operation performed in Azure Active Directory.
9 – Indicates OrgId logon events in Azure Active Directory. This record type is being deprecated.
10 – Indicates security cmdlet events that were performed by Microsoft personnel in the data center.
11 – Indicates Data loss protection (DLP) events in SharePoint.
12 – Indicates Sway events.
13 – Indicates DLP events in Exchange, when configured with a unified a DLP policy. DLP events based on Exchange mail flow rules (also known as transport rules) aren’t supported.
14 – Indicates sharing events in SharePoint.
15 – Indicates Secure Token Service (STS) logon events in Azure Active Directory.
18 – Indicates Security & Compliance Center events.
19 – Indicates aggregated Exchange mailbox operations for repetitive activity that occurs within a very short duration.
20 – Indicates Power BI events.
21- Indicates Dynamics 365 events.
22 – Indicates Yammer events.
23 – Indicates Skype for Business events.
24 – Indicates eDiscovery events. This record type indicates activities that were performed by running content searches and managing eDiscovery cases in the security and compliance center. For more information, see Search for eDiscovery activities in the Office 365 audit log.
25, 26, or 27 – Indicates Microsoft Teams events.
28 – Indicates phishing and malware events from Exchange Online Protection and Office 365 Advanced Threat Protection.
29 – Indicates submission events from Exchange Online Protection and Office 365 Advanced Threat Protection.
30 – Indicates Microsoft Power Automate (formerly called Microsoft Flow) events.
31 – Indicates Advanced eDiscovery events.
32 – Indicates Microsoft Stream events.
33 – Indicates events related to DLP classification in SharePoint.
35 – Indicates Microsoft Project events.
36 – Indicates SharePoint list events.
37 – Indicates events related to SharePoint comments.
38 – Indicates events related to retention policies and retention labels in the security and compliance center.
40 – Indicates events that results from security and compliance alert signals.
41 – Indicates safe links time-of-block and block override events in Office 365 Advanced Threat Protection
42 – Indicates events related to insights and reports in the Office 365 security and compliance center.
44 – Indicates Workplace Analytics events.
45 – Indicates Power Apps events.
47 – Indicates phishing and malware events from Office 365 Advanced Threat Protection for files in SharePoint, OneDrive, and Microsoft Teams.
49 – Indicates Patients application events in Microsoft Teams for Healthcare.
50 – Indicates events related to the MailItemsAccessed mailbox audit action.
52 – Indicates events related to the Data Insights REST API.
53 – Indicates events related to the application of information barrier policies. For more information, see Define policies for information barriers.
54 – Indicates SharePoint list item events.
55 – Indicates SharePoint content type events.
56 – Indicates SharePoint list field events.
62 – Indicates events related to email attack campaigns. For more information, see Campaign Views in Office 365 ATP.
64 – Indicates automated investigation and response events. For information, see automated investigation and response (AIR) in Office 365 66 – Indicates Microsoft Forms events.
68 – Indicates Communication compliance events in Exchange. For more informat ion, see Communication compliance in Microsoft 365.
69 – Indicates events related Customer Key Encryption. For more information, see Service encryption with Customer Key in Office 365.

Summary

In this document I have shown how you can onboard Office 365 Management Activity API General.Audit and DLP.All activity logs, and some basics queries for you to start to build out your use cases with Defender for Office and Security and Compliance Center alerts. This solution helps you extend, correlate and enrich the data you have with the existing O365 connector, giving you more insights.