Copilot Backfill

Copilot → Copilot Backfill

Backfilling older Copilot history (Preview)

The live import does all the routine work: every importer cycle it fetches recent Copilot interactions, and nobody has to start it. The Copilot audit backfill is for one problem only: Copilot interactions from more than about a week ago that were never imported. The live import can never reach those, however long you wait. It reads the Office 365 Management Activity API, which only offers content up to about seven days old, and only content created after your tenant's subscription to the feed started. The interactions are still in the Microsoft 365 unified audit log, which keeps them for 180 days with standard auditing. The backfill searches that log through Microsoft Graph and imports what the live import missed.

Not to be confused with "Copilot interactions awaiting backfill" on the Copilot Adoption page. That is an automatic upgrade of interactions already in the database, and needs nothing from you.

Live import or backfill?

	Live audit import	Copilot audit backfill (Preview)
Use it for	Keeping Copilot data up to date	Filling a gap in older history, once
Runs	Automatically, every importer cycle	Only when an administrator starts it. It then works a few minutes at a time, after the live import in each importer cycle, until it has searched every day
Source	Office 365 Management Activity API (Audit.General)	Microsoft Graph Audit Search (/security/auditLog/queries) over the unified audit log
Reaches back	The last DaysBeforeNowToDownload days (6 by default). The API only offers content up to about 7 days old	Up to 180 days before you start it, or from a later date you choose
Imports	Every workload you have switched on	Copilot interactions only (CopilotInteraction, record type 261)
Permission	ActivityFeed.Read (Office 365 Management APIs), one of the standard permissions	AuditLogsQuery.Read.All (Microsoft Graph, application). Optional, and not granted by the installer

As a rule of thumb: if the missing history is less than about six days old, you don't need the backfill, because the live import catches up by itself once the importer is running again. If it's older than that, and less than 180 days old, use the backfill.

When to use it, and when not

Use it after:

• a new install, to bring in Copilot history from before the install;
• switching the Copilot import on, to bring in history from before you did;
• an importer outage of more than about a week, for example after a client secret expired or a deployment broke.

Don't use it:

• routinely, or on a schedule. The live import keeps the data current.
• to complete interactions already imported, for example to fill in the fields captured since build 1807. It never changes a row that already exists.
• for other workloads. It imports Copilot interactions only.
• for history older than 180 days. It can't reach it, even if your tenant keeps audit records for longer.

Before you start a backfill

1. Grant AuditLogsQuery.Read.All. On the runtime app registration, add the Microsoft Graph application permission AuditLogsQuery.Read.All and grant admin consent. Consenting to a Microsoft Graph application permission needs a Privileged Role Administrator (or Global Administrator). See Backfill privacy for why you may want to remove it again afterwards.
2. Give it an importer cycle. The importer checks the permission on the token it uses for each cycle, so a backfill picked up by a cycle that began before the consent fails with a missing-permission error. Wait until Service health → Import liveness shows a cycle that finished after you granted consent. If a backfill does fail this way, just start a new one.
3. Check that the Copilot import is on, and that the web app and web job have a Storage connection string. The backfill keeps its progress in the AnalyticsState Azure table (partition CopilotAuditBackfill), where both can see it. If either is missing, the page says so and won't start a backfill.

Starting, following and cancelling a backfill

Open Administration → Copilot audit backfill (#/admin/copilot-audit-backfill). It needs the Administration permission only, not See PII.

1. Optionally set Earliest date. It is a UTC date. Leave it empty to go back the full 180 days; an earlier date is moved forward to 180 days ago. There is no end date: a backfill always runs up to the moment you start it.
2. Select Start backfill. It shows Queued until the importer's next cycle picks it up, then Running.

Only one backfill runs at a time, and the page shows only the latest. While one is queued or running, the page refreshes itself every few seconds.

Row	What it tells you
Permission status	Whether the runtime app's token carries AuditLogsQuery.Read.All. It says Not checked yet until the importer first picks the backfill up.
Slices completed / total	A slice is one search, normally one UTC day. The total changes as days are split into smaller searches or recorded as gaps.
Queries in flight	Searches submitted and not yet fully imported, at most four. Current slice is the oldest of them.
Records imported / seen	Seen is what the searches returned, and imported is the interactions written, including partly saved ones it completed. The difference is the records already present, people outside UserGroupsFilter, and any record that couldn't be read.
Records already present	Interactions already in the database with their Copilot details, which are skipped. Expect plenty for the last week, which the live import has already covered.
Completed days (most recent 12), Failed days, Incomplete days	Whole UTC days searched, and the days with gaps. See When a backfill finishes.
Last error	The most recent problem. Not every error stops the backfill: see Backfill errors.

Cancel stops the backfill during the importer's current or next cycle. Whatever it has already imported stays. Searches it has already submitted aren't withdrawn: Microsoft finishes them, and the backfill ignores the results.

Duration and the tenant's Audit Search allowance

Audit Search is shared by everything in your tenant that uses it, including Microsoft Purview's own audit search, SIEM connectors and other tools. Microsoft gives each tenant a baseline of at least 200 search submissions in any rolling 24 hours (see Microsoft Graph service limits). The backfill is deliberately gentle with it:

• It budgets at most 100 successfully accounted searches in a rolling 24 hours, leaving room within the baseline for other callers. When the recorded budget reaches 100, it waits for its oldest submissions to drop out of the 24 hours.
• It searches one UTC day per search, newest first, with up to four searches running at once. A day with more records than one search can return (at least a million) is split into four six-hour searches, and a six-hour search that is still too big is split into hours. Since Stable 1856 (PR #768): an hour that is still too big is split again into quarter-hours, and the records a quarter-hour search still returns are imported rather than discarded.
• If Microsoft throttles a submission (429), it stops submitting for as long as Microsoft asks, and for at least 15 minutes. It keeps collecting the searches already running, and the throttled search doesn't count as a failed attempt.
• If a submission fails for another reason, such as a Microsoft outage, a network error or a timeout, it stops submitting for 5 minutes, then 10, 20, 40 and at most 60 if submissions keep failing, and goes back to normal once one succeeds. The failed search counts as one of its three attempts. A backfill only finishes when every day has been searched or recorded as a gap, so an outage makes it slower, never shorter.
• It uses an eight-minute soft budget in each importer cycle, after the live import. A single HTTP request or SQL commit can exceed that budget and delay the next normal cycle; it is neither a hard deadline nor a zero-impact guarantee. Between cycles, Microsoft carries on running the searches already submitted.

The 100-a-day limit and any pause apply to the backfill feature as a whole, not to one backfill: they're kept in one shared record in the AnalyticsState table, so cancelling a backfill and starting another doesn't reset them.

Preview accounting limit: a remote submission and local durable state writes are not one transaction. Exactly-once quota accounting through a crash immediately after the remote POST, and distributed multi-worker atomicity, have not been proven. The local budget is not a guarantee that no unrecorded request reached Microsoft.

So a full 180 days, about 181 searches, can't finish in less than a day: after its first 100 searches the backfill has to wait for the 24 hours to roll on before it submits the rest. Allow about two days. Slow searches, long importer cycles and days that have to be split all make it longer.

---

Return to Copilot · All topics in this guide

Next: Copilot Backfill Recovery