Importer Tuning Reference

Configuration Reference → Importer Tuning Reference

Tuning & advanced

These are rarely changed. Most are set by the installer or left at their defaults.

Setting	Default	Purpose
MetadataRefreshMinutes	1440 minutes (24 hours)	SharePoint page-metadata refresh/cache cadence; the setting's unit is minutes. See Product notes.
ChunkSize	1.00:00:00 (1 day)	Time window per Activity API download chunk.
DaysBeforeNowToDownload	6	How many days back the activity importer downloads audit-log (Office 365 Management Activity API) content on each run. Builds before 1856 also used it for the Microsoft 365 usage reports; from Stable 1856 they have their own look-back, UsageReportsDaysBack.
UsageReportsDaysBack	25	Since Stable 1856 (PR #768). How many days back the daily Microsoft 365 usage reports (Teams, Outlook, OneDrive, SharePoint, Viva Engage, apps and platforms) are requested. Microsoft Graph keeps only about 28 days of usage-report detail, so a day that fell out of the old 6-day window before its import succeeded could never be fetched again. A whole number of days, effectively from 3 to 28: a missing, invalid, zero or negative value uses 25, 1 or 2 is raised to 3, and anything above 28 is treated as 28. Days already stored that Graph no longer changes are skipped, so the larger window mostly costs extra downloads only on a report's first run, or after a save that failed part-way. The first run after upgrading can download, and fill in, days that earlier builds missed. Optional; not written by the installer, and not part of the installer's saved configuration.
CleanUpConcealedIdentityUsers	on	Since Stable 1856 (PR #768). Whether the activity importer deletes the placeholder users that builds before 1856 created from concealed (hashed) usage-report identities, together with their usage-report rows. It works through them a bounded slice per import cycle until none are left. Set it to false before upgrading to keep those rows; deleted rows can only be restored from a backup. false turns it off, and true, or a missing or empty value, leaves it on; letter case doesn't matter. Any other value, such as 0, no, off or a typo, also turns it off, because the delete can't be undone, and the importer logs an error naming the value once after each start: set exactly true or false. A runtime setting only; not written by the installer, and not part of the installer's saved configuration. See Usage reports and email troubleshooting → Placeholder users from concealed usage reports.
ReadHitsDaysBeforeToday	(unset)	How many days back to start reading web-traffic hits from Application Insights. Can also be passed via the -readHitsDaysBeforeToday command-line argument. While it's set, every cycle re-reads whole days from that many days ago and leaves the import's ingestion cursor where it is, so remove it once the re-read has finished. See Web tracking troubleshooting → A day of web traffic failed to save.
TimeChunkOverlapMinutes	5	Overlap between time chunks to avoid missing events at boundaries.
MaxSummaryFetchConcurrency	8	Maximum number of simultaneous content-summary requests to the Office 365 Management Activity API (the audit log). Lower it if the Activity API throttles the importer.
ForceUsageReportsImport	off	Forces a usage-reports import even if one ran recently (diagnostics).
ContentTypesListAsString	Audit.SharePoint	Which Office 365 Management Activity API feeds to subscribe to, ;-separated. The installer writes it on every run from the import flags (ActivityLog → Audit.SharePoint; Copilot, ImportPowerPlatform → Audit.General; ImportDlp → DLP.All), so don't edit it on its own. See Troubleshooting → Enabling/disabling imports.
ResolveCopilotResourceMetadata	True	Resolve Copilot file/meeting metadata (file name/site, meeting name/date) from the Graph. Set False to keep only Copilot agent metadata (agent id/name/type, cost, messages, accessed resources) and skip the per-event Graph resolution — this removes serial, network-bound calls from the save path when only Copilot agent-level reporting is needed. Optional; not written by the installer. Since Stable 1856 (PR #768): False also suspends durable enrichment retries without deleting pending work. No additional setting is needed.
AUDIT_PERBATCH_DEDUP_CACHE	off	Safety valve. The audit importer builds its "already-processed" dedup cache once per cycle by default. Set true to revert to the older per-batch rebuild (much slower and higher memory on large tenants). Optional; not written by the installer. Since Stable 1856 (PR #768): the default once-per-cycle cache no longer loads every event ID of the download window (about seven days) up front, which took several GB on a tenant with millions of events a day; each batch looks up its own event IDs instead. This setting still restores the per-batch rebuild.
UserGroupsFilter	(unset)	Restrict every import to the direct members of named Entra ID groups. See Restricting imports to Entra groups.
WebAppURL	(set by installer)	Public URL of the admin/website app service.
AppInsightsConnectionString	(set by installer)	Application Insights connection string used for the solution's own logging/telemetry.
AppInsightsContainerName	(set by installer)	The Application Insights resource the AITracker web-traffic data is read from.

Restricting imports to Entra groups (UserGroupsFilter)

Set UserGroupsFilter to import data only about the members of particular Entra ID groups - a Copilot pilot, a region, a business unit. Leave it unset, or set it to *, to import everyone.

Value. One or more group names or group object IDs, separated by ;. A * inside a name is a wildcard, and names match case-insensitively.

Example	Selects
Copilot pilot	Members of the group whose display name is exactly Copilot pilot.
00000000-0000-0000-0000-000000000000	Members of the group with that object ID.
Pilot*;Contoso Sales	Members of every group whose name starts with Pilot, and of Contoso Sales.

Who counts as a member. Direct user members of the matched groups only. Members of a nested group are not included, and neither are devices, service principals or other non-user members.

What it applies to. Every import that records something about a person:

Import	Under a filter
User directory (user details and licences)	Only members are added to or updated in the users table. A manager who is not a member is not recorded. Since Stable 1856 (PR #768): an existing row whose Entra account is disabled or deleted is marked account_enabled = 0 whether or not the person is a member, because a leaver has usually left the groups too. For someone outside the filter, nothing else is written. See Troubleshooting → Leavers still show as enabled.
Audit log (SharePoint, Copilot, Power Platform, DLP)	Events by anyone else are skipped.
Microsoft 365 usage reports, and the Copilot per-user usage report	Rows for anyone else are skipped. Concealed (de-identified) rows can't be matched to anyone, so they are skipped too.
Copilot AI interaction history	Only members are read.
Sent emails	Only members' mailboxes are read.
Teams calls	A call is kept when anyone on it is a member. Everyone else on it is stored as the anonymous Unknown User, and none of their feedback is kept. A call with no members on it is not stored.
Teams crawl	Only members are recorded as team owners and members. Channel messages and reactions by anyone else are left out of the channel statistics.
Web traffic (Application Insights)	Page views, clicks, searches and page exits by anyone else are not stored, and neither are their page comments and likes. The page itself is still recorded.
Copilot Studio per-user credits	Rows for anyone else are not stored. The per-agent and capacity totals stay tenant-wide.

Tenant-wide reports with no per-person rows - the Copilot summary and trend reports, Yammer group activity, SharePoint site usage - are not affected.

How the groups are resolved. Each web job reads the groups' members from Microsoft Graph, which needs Group.Read.All (see Prerequisites). It does so when it starts and then about once an hour, and every import in that web job uses the same list. Each cycle logs how many people are in scope.

• A filter that matches no group means nobody is in scope: no per-person data is imported, and a warning is logged every cycle. Check the group names.
• If the groups can't be read - a missing permission, or a failed or throttled request - the imports carry on with the last list that did resolve. If none has resolved since the web job started, they import everyone and log an error every cycle until the groups can be read: a failure never stops the imports. The Copilot AI interaction history import is the exception, and imports nobody instead.

Changing the filter. Changing the setting restarts the web jobs and the web app. The next user-directory import then reads the whole directory, so the people the new filter lets in are added. People who join a group later are added by a full re-read of the directory, at most once a day. The restart also stops a purge that is running; start it again to purge against the new filter.

Not retroactive. Data imported before the filter was set, or about people who have since left its groups, stays in the database: setting the filter doesn't delete it. To remove it, use Administration → User scope in the web portal - see The web portal → Purging people outside the scope.

Reducing importer CPU usage (aggressiveness)

The two importer WebJobs (Office365ActivityImporter and AppInsightsImporter) run continuously on the same App Service plan as the website (Basic B1 = 1 vCPU by default, B2 in production). Each WebJob wakes on a fixed cycle and, by default, fans work out across many threads — so on a small plan you can see short bursts of 100% CPU, which may trip tenant CPU alerts.

These settings let you trade a little import speed for lower, steadier CPU without losing freshness on the data that needs it. Audit events and web-traffic hits keep their normal cycle cadence at every level — only the burst concurrency (and, at Gentle, the cycle pause) change. The other Graph imports (user metadata and the Teams crawl) change very little intraday, so at Balanced and Gentle they are gated to run once a day instead of every cycle. At the default High they still run every cycle.

All of these are optional — they are not written by the installer. To change them, add the key on App Service → Configuration → Application settings and restart the WebJobs.

ImportAggressiveness — one-setting preset

Value	Audit-load threads	Cycle pause	Non-fresh Graph imports*	Effect
High (default)	20	10 min	every cycle	Fastest, highest peak CPU — full pre-feature behaviour.
Balanced	8	10 min	daily (24h)	Modestly eased; a good first step if the plan runs hot.
Gentle	3	20 min	daily (24h)	Lowest CPU; imports take longer but stay well below 100%.

*Non-fresh Graph imports = user metadata and the Teams crawl. Audit events and web-traffic hits keep their normal cycle cadence at every level.

The default is High, which is the original behaviour — upgrading does not quietly slow your imports down. If the App Service plan is running hot, set ImportAggressiveness=Balanced (or Gentle for the quietest footprint); nothing changes until you do.

Individual overrides

Any of these override the value the preset would otherwise supply:

Setting	Default	Purpose
MaxAuditReportLoadConcurrency	preset (High 20 / Balanced 8 / Gentle 3)	Max parallel threads used to full-load audit reports from the Activity API. The main per-cycle CPU lever.
ImportCyclePauseMinutes	preset (High/Balanced 10 / Gentle 20)	Minutes a WebJob waits between import cycles.
GraphMetadataImportIntervalHours	preset (High 0 / Balanced 24 / Gentle 24)	Minimum hours between the static non-fresh imports (user metadata). 0 = run every cycle.
GraphTeamsImportIntervalHours	preset (High 0 / Balanced 24 / Gentle 24)	Minimum hours between Teams crawls. Separate from the above because Teams analytics (channel messages/reactions) is fresher — lower it to keep Teams current without un-gating the static imports. 0 = run every cycle.
ForceGraphMetadataImport	off	One-off: bypass the gate for all of the non-fresh Graph imports on the next cycle(s). See Forcing a re-import.
ImportStartStaggerMinutes	0 (disabled)	One-off delay before the hits WebJob's first cycle, so the two WebJobs don't peak on the shared plan at the same time. A good starting point is half the cycle pause.
MaxSqlCommitConcurrency	preset (High 20 / Balanced 8 / Gentle 3)	Max parallel SQL commit operations. Lower this first if the database is the bottleneck rather than the App Service.

The gate's "last run" timestamps are stored in AnalyticsState (PartitionKey = ImportSchedule), so the gate survives WebJob restarts. Storage read/write failures are fail-open — a storage blip never skips an import.

Teams channel-work budget

Released in 1854 through PR #760. This opt-in budget is persisted in installer schema 3.4.0; zero retains the legacy crawl.

TeamsMaxChannelsPerTeamPerCycle is an optional advanced installer JSON root property and App Service application setting. 0 or an absent setting preserves the existing all-channel crawl. A positive integer limits deep message reads to that many channels per Team per Teams-import cycle. The wizard preserves this JSON setting but has no separate visual control. Malformed, negative or overflowing non-blank runtime values are configuration errors, not an instruction to turn the budget off.

The importer stops between channels, saves SQL, commits completed channel delta tokens, and then saves round-robin progress in AnalyticsState / TeamsChannels. Deferred channels retain their Graph tokens and receive later slots, including after restart or channel enumeration changes. A normal budget stop does not split a channel or routinely replay its additive statistics.

This is not a hard time, page-count or peak-memory bound. One noisy channel can still reach the existing root/reply paging caps; membership, tabs and metadata work are not separately capped. An incomplete channel or a genuine SQL/checkpoint failure retains the pre-existing at-least-once/additive-replay risk. The budget does not make those operations exactly once.

Before a pilot, balance load against freshness: a full sweep takes about ceil(channel count / limit) × actual Teams-import interval. With a fictional 1,000 channels, limits of 8 and 32 need approximately 125 and 32 cycles respectively; these are illustrative pilot values, not approved defaults. At a 24-hour Teams-import interval that trade-off can be unacceptable. Longer sweeps also increase Graph delta-expiry risk.

Use one importer and an authorised synthetic pilot before choosing a production value. Watch TeamsChannelBudgetExhausted (work deferred), TeamsChannelReadIncomplete (old checkpoint retained), TeamsChannelGraphFailure (actual request failure) and TeamsChannelBudgetProgressSaved (persistence boundary). Restart/recycle the importer after changing application settings. Set the value back to 0 to restore the prior crawl. No SQL migration is required.

Copilot import pacing

The two optional Graph-based Copilot imports are paced separately from ImportAggressiveness, because both are expensive per user and neither needs to be fresh by the hour.

Setting	Default	Purpose
GraphCopilotUsageReportsIntervalHours	24	Minimum hours between Copilot usage report imports. 0 runs every cycle. Microsoft publishes these daily, so there is nothing to gain from going faster.
CopilotInteractionHistoryIntervalHours	24	Minimum hours between Copilot prompt-history imports. 0 runs every cycle.
CopilotInteractionHistoryMaxUsersPerCycle	500	The main safety valve. The interaction-history endpoint is one HTTP call per user, so an unscoped tenant-wide run at the 200,000-user design target would be 200,000 Graph calls per cycle. The import stops at this many users and resumes where it left off next cycle. Scope it properly with UserGroupsFilter as well.
CopilotInteractionHistoryMaxDaysBackOnFirstRun	30	How far back to reach for a user the importer has never seen. Bounds the initial backfill.
CopilotInteractionHistoryEmptyUserBackOffHours	72	How long to wait before re-checking a user who returned nothing — typically someone without the M365_COPILOT_BUSINESS_CHAT service plan. Stops the import spending its per-cycle budget on users who will never have data.
SentEmailNoMailboxRetryHours	24	How long to wait before retrying a user whose mailbox could not be read by the optional sent-email import.

Environment-variable switches

Two further controls are read from the environment rather than from the settings above (AUDIT_PERBATCH_DEDUP_CACHE in the previous table is the third). On App Service, application settings are surfaced as environment variables, so you still set them under Configuration → Application settings.

Variable	Default	Purpose
AUDIT_MAX_CONCURRENT_SAVES	1 (strictly serial)	How many audit batches may commit in parallel. 1 is the original serial path against the single shared staging table; above 1 each save builds its own sharded staging table and only the shared-table writes (the merge and the metadata pass) stay serialised. Anything that is not a whole number greater than 1 — unset, blank, non-numeric, zero or negative — leaves the serial default. Validate in a non-production environment before using it.
AI_USAGE_REPORT_SAVE_DIAGNOSTICS	off	Opt-in, privacy-safe per-stage diagnostics for the daily usage-report save path, emitted to Application Insights. Accepts a truthy value (e.g. true/1). Off by default and a no-op when unset, so the hot path keeps its normal cost.

---

Return to Configuration Reference · All topics in this guide

Previous: Runtime State Reference · Next: Optional Services Configuration