Copilot

Copilot Analytics

This page describes what Microsoft 365 Copilot data the analytics engine tracks, how it is collected, and when.

Data Source

Copilot events are collected from the Office 365 Management Activity API using the Audit.General content type. The workload identifier is Copilot.

Reference: https://learn.microsoft.com/en-us/office/office-365-management-api/copilot-schema

For Copilot events to be imported, Audit.General must be included in the ContentTypesListAsString configuration key (it is by default). Additionally, the ActivityLog import task must be enabled in ImportJobSettings.

Import Schedule

Copilot events are imported as part of the Office365ActivityImporter web-job. This runs continuously:

• Each import cycle scans the Activity API for events from the last several days (configured by DaysBeforeNowToDownload, default 6 days).
• After each cycle completes, the web-job waits 10 minutes before starting the next cycle.
• Events already processed are skipped (tracked via the ignored_audit_events table).

There is typically a 2–3 day delay between a Copilot interaction occurring and the event appearing in the Activity API.

What is Tracked

Every Copilot interaction generates one audit event. The engine categorises each event into one of three types based on the interaction context:

1. Chat-Only Interactions

When a user interacts with Copilot without any file or meeting context (e.g. a general question in Teams chat or Bing).

Field	Description
Date/time	When the interaction occurred (UTC).
User	The user who initiated the interaction.
App host	Which Copilot surface was used (e.g. Teams, Word, Excel, Outlook, Bing, M365App, etc).
Agent	The Copilot agent involved, if any. Includes whether it is a custom agent (extracted from AppIdentity).
Accessed resources	Documents and data sources Copilot referenced to generate the response (see below).
Response messages	Message IDs of Copilot responses (user prompts are not tracked).
AI models used	Names of AI models involved (e.g. DEEP_LEO for deep reasoning).
Credit estimate	Estimated Copilot Credits consumed, with a breakdown by category.

2. File Interactions

When Copilot is used in the context of a specific file (e.g. summarising a Word document, generating a slide in PowerPoint).

All fields from chat-only, plus:

Field	Description
File name	Name of the file.
File extension	File type (docx, pptx, xlsx, etc).
URL	Full URL of the file.
Site	SharePoint site collection URL where the file resides.

File metadata is resolved via the Microsoft Graph API using the context ID from the audit event.

3. Meeting Interactions

When Copilot is used during a Teams meeting (e.g. generating meeting notes or a summary).

All fields from chat-only, plus:

Field	Description
Meeting name	Subject of the online meeting.
Meeting ID	Graph identifier for the meeting.
Meeting created date	When the meeting was originally created (UTC).

Meeting metadata is resolved via the Microsoft Graph API (/users/{id}/onlineMeetings).

Accessed Resources

For any interaction type, Copilot may reference documents and data sources to generate a response. Each accessed resource is tracked with:

Field	Description
Resource ID	Identifier of the accessed resource.
Resource name	Display name of the resource.
Site URL	SharePoint site URL of the resource (if applicable).
Resource type	Type of resource (e.g. file, email, chat message).
Sensitivity label	Microsoft Purview sensitivity label ID, if one is applied to the resource.

Credit Estimation

For interactions involving a Copilot agent (standard or custom), the engine estimates the number of Copilot Credits consumed based on the Microsoft Copilot Studio billing model.

Billing Category	Credits	How Detected
Generative answer	2 per response	Response messages that are not prompts.
Tenant Graph grounding	10 per grounded answer	Accessed resources from SharePoint, OneDrive, Email, or Teams.
Agent action (e.g. deep reasoning)	5 per action	DEEP_LEO model detected in ModelTransparencyDetails.

The credit estimate is stored both as a total and as a JSON breakdown per event.

Reference: https://learn.microsoft.com/en-us/microsoft-copilot-studio/requirements-messages-management

App Hosts

The AppHost value indicates which Copilot surface the user was interacting with. Known values include:

Assist365, Bing, BashTool, DevUI, Excel, Loop, M365AdminCenter, M365App, Office, OneNote, Outlook, Planner, PowerPoint, SharePoint, Stream, Teams, VivaCopilot, VivaEngage, VivaGoals, Whiteboard, Word.

Custom Agent Detection

If the AgentName and AgentId fields are not set in the audit event but AppIdentity contains an organisation ID, the engine extracts the agent name from the AppIdentity string (format: Copilot.Studio.Default-{OrgId}-{AgentName}). These are flagged as custom agents via the is_custom_agent column.

Copilot usage reports

Added in Stable build 1791. Config flag: GraphCopilotUsageReports. Off by default.

Separate from everything above. The audit-feed import describes interactions we observed; this one pulls Microsoft's own official Copilot usage reporting from Graph — the same figures the Microsoft 365 admin centre shows, which is what customers will compare our numbers against.

Three reports are imported into three tables:

Report	Table
Copilot user-count summary	copilot_user_count_log
Copilot user-count trend	copilot_user_count_log
Per-user Copilot usage detail	copilot_usage_user_activity_log

Plus copilot_usage_report_import_log, one row per import run, so the Health page can show whether the last run succeeded.

• Permission: Reports.Read.All, admin-consented.
• Global cloud only. Outside the global cloud these endpoints return 404, which the importer tolerates and records as the reason.
• A missing or unconsented grant now fails the import. Previously a 403 was swallowed and recorded as an empty day, which read as "this tenant has no Copilot licences" and suppressed retry for 24 hours. It now surfaces the HTTP status on the Health page instead. A genuinely empty report is still a clean success.
• Retention: these tables had no age bound before Stable build 1791 and grew forever. Clean Old Data Data.sql now purges them in 10,000-row batches. That script is not deployed by the installer — see Updating the solution.

Copilot AI interaction history (prompt history)

Added in Stable build 1791. Config flag: CopilotInteractionHistory. Off by default, and deliberately harder to enable than the others.

Imports Copilot prompt history from /copilot/users/{id}/interactionHistory/getAllEnterpriseInteractions.

What is stored — and what is not

Never stored: prompt text, response text.

Stored: derived statistics per interaction — body character and word counts, attachment / link / mention / context counts, response latency, app class, conversation type, locale, device.

Additionally stored when Cognitive Services is configured: a detected language, a positive-sentiment confidence score (0.0–1.0), and extracted keywords.

This is a new egress path for prompt content. Enabling cognitive enrichment sends prompt text to Azure AI Language for analysis. The extracted keywords that come back are stored, and a keyword can amount to a whole short prompt. Decide this deliberately — see Security Model → Copilot prompt content.

Three prerequisites, all required

1. AiEnterpriseInteraction.Read.All — application permission, not requested or granted by the installer. An Entra administrator must add it to the runtime app registration and grant admin consent explicitly.
2. The M365_COPILOT_BUSINESS_CHAT service plan on each user. Users without it return nothing.
3. A scope. The endpoint is one HTTP call per user. At the ~200,000-user design target an unscoped run means 200,000 Graph calls per cycle — point it at a pilot group with UserGroupsFilter. UserGroupsFilter is an optional narrowing rather than a precondition, so leaving it unset runs tenant-wide. The import is also capped per cycle as a backstop, and resumes where it left off.

What it powers

The Copilot tab in Reports gains three visuals when Cognitive Services is configured: a keyword word cloud, weekly prompt sentiment, and a prompt-language breakdown. Without it the three are hidden and the page explains why.

Tables

copilot_interactions, copilot_interaction_sessions, copilot_interaction_keywords, copilot_interaction_user_watermarks, plus lookups (copilot_interaction_types, copilot_interaction_app_classes, copilot_interaction_conversation_types, copilot_interaction_locales, copilot_interaction_devices) and copilot_interaction_import_log.

copilot_interaction_user_watermarks records how far the import got per user and is deliberately not cleaned by the retention job — deleting it would make the next run re-scan every user's whole backfill window.

Database Tables

All Copilot data is stored in the tables documented in the Database Schema page. The key tables are:

Table	Purpose
copilot_chats	Core interaction record: app host, agent, credit estimate. Links to audit_events.
copilot_event_files	File context: name, extension, URL, site.
copilot_event_meetings	Meeting context: links to online_meetings.
copilot_agents	Agent definitions: name, agent ID, custom flag.
copilot_event_accessed_resources	Resources Copilot referenced, with type and sensitivity label.
copilot_event_messages	Response message IDs.
copilot_ai_models / copilot_event_ai_models	AI models used per interaction.
sensitivity_labels	Shared sensitivity label lookup.

Required Permissions

Copilot audit events are delivered through the Office 365 Management Activity API. The runtime application registration requires:

• Office 365 Management APIs – ActivityFeed.Read (application permission).
• Microsoft Graph – OnlineMeetings.Read.All and Sites.Read.All (application permissions) for resolving meeting and file metadata.

The two Graph-based Copilot imports need their own permissions on top:

• Reports.Read.All — for Copilot usage reports.
• AiEnterpriseInteraction.Read.All — for Copilot AI interaction history. Not granted by the installer; needs explicit admin consent.

Meeting enrichment also needs a Teams application access policy

OnlineMeetings.Read.All on its own is not enough to read meeting details. Microsoft Graph additionally requires the tenant to grant the runtime application an application access policy for the users whose meetings should be readable. Without it, every meeting lookup is refused with:

403 Forbidden - No application access policy found for this app <app-id> on the user

This is a tenant configuration step, not a product fault. When it's missing, Copilot events still import normally — only the meeting name / meeting created date are blank, and the importer logs a single warning per import run explaining the fix.

To grant it, a Teams administrator runs (in the Microsoft Teams PowerShell module):

Connect-MicrosoftTeams

# Create a policy that allows the runtime app to read online meetings
New-CsApplicationAccessPolicy -Identity M365AnalyticsMeetingRead `
    -AppIds "<runtime-app-client-id>" `
    -Description "Microsoft 365 Analytics Insights - Copilot meeting enrichment"

# Grant it tenant-wide...
Grant-CsApplicationAccessPolicy -PolicyName M365AnalyticsMeetingRead -Global

# ...or to specific users only
Grant-CsApplicationAccessPolicy -PolicyName M365AnalyticsMeetingRead -Identity user@contoso.com

Policy changes can take a few hours to take effect. See Configure application access policy for the full reference.