Runtime State Reference

Configuration Reference → Runtime State Reference

Runtime state in Azure Table storage

Runtime state is stored in the existing storage account named by the Storage connection string. If you are upgrading from an older cache-backed release, see Upgrading from a release that used Redis. The table is AnalyticsState and is created automatically on first use. It is reached the same way as the audit-import blob checkpoint table, and as Redis was. The credentials in the Storage connection string (the storage account key the installer writes, or a SAS) are used as they are. If the storage account denies them, for example because shared-key access is disabled or the key has been rotated, the runtime service principal is used instead (RBAC). If the connection string has no credentials, the service principal is used directly. The runtime principal needs Storage Table Data Contributor on the storage account for that; the installer grants it, and Storage Blob Data Contributor does not grant table access.

On private-endpoint deployments, this uses the storage account's table private endpoint and the privatelink.table.core.windows.net DNS zone. The installer has created both since Stable build 1716; a private deployment last installed with an earlier build gets them when the installer is run again (see Manual or binaries-only deployments if you upgrade without it). No separate cache, role assignment, firewall rule or private DNS zone is required.

The table uses one partition per state type. Most partitions use one entity per value; the enrichment queue (since Stable 1856) uses an atomic pair of entities:

PartitionKey	What it contains	Row keys / examples
ImportSchedule	Last-run and retry stamps for interval-gated imports.	GraphUsersMetadataLastImported, GraphTeamsLastImported, GraphCopilotUsageReportsLastImported, GraphCopilotUsageReportsLastImported:UserCountTrend, GraphCopilotUsageReportsLastImported:UserCountSummary, GraphCopilotUsageReportsLastImported:UsageUserDetail, CopilotInteractionHistoryLastImported, CopilotStudioCreditsLastImported, AzureCostLastImported, Agent365PackageCatalogLastImported, UserActivityLastImported, UserActivityReportLastImported:<ReportLoaderName>, UserActivityReportNextAttemptUtc:<ReportLoaderName>, statsLastUploaded, telemetryAdoptionStatsLastRun. The retired GraphCopilotUsageReportsLastImported:CoworkUsageUserDetail key is not used by current stable; see Known issues. Since Stable 1856 (PR #768): UserActivityReportNextAttemptUtc:SharePointSitesWeeklyUsageReportLoader now holds the weekly SharePoint site usage report's own retry time, and UserActivityReportNextAttemptUtc:ConcealedIdentityRefusal records, for one day, that usage reports were refused because identities are concealed.
UserImport	Microsoft Graph /users/delta checkpoint, plus the UserGroupsFilter and group membership it was taken under, so that a change to either makes the next import read the whole directory.	UserDeltaCode-<tenant-id>-v2, UserDeltaCodeScope-<tenant-id>-v2, UserDeltaCodeScopeMembers-<tenant-id>-v2, each with a suffix when Entra ID organisation types are configured. Administration → User import shows the checkpoint's exact row key.
UserIdentityQuarantine (since Stable 1856)	Users the user import couldn't safely match to a users row (for example, their new sign-in name is held by another person's row, or Graph listed two names for them), with the latest Graph profile of each, so they're checked again every cycle while the /users/delta checkpoint moves on. Rewritten only when the set changes. Holds UPNs, so it's personal data, protected like the rest of this table. See User import troubleshooting → Renamed users.	IdentityQuarantine-<tenant-id> (header), then pages IdentityQuarantine-<tenant-id>:0, :1, … (or :b:0, :b:1, … in the second of two slots).
SentEmails	One delta token per mailbox, plus the list of users with no mailbox: a header row (when the list was last rebuilt and how many pages hold it) and pages of up to 10,000 UPNs each. Since Stable 1856 (PR #768): also where a mailbox read that reached its per-cycle limit stopped, so the next cycle carries on from there; see Usage reports and email troubleshooting → Sent emails.	SentEmails-<UPN>; SentEmailNoMailboxUsers (header), SentEmailNoMailboxUsers:0, SentEmailNoMailboxUsers:1, … (pages); SentEmailsNextPage-<UPN> (since Stable 1856).
TeamsChannels	Per-channel Teams message delta tokens; builds with the optional channel budget also keep a separate scheduling cursor.	<teamId>|<channelId>; budget|<teamId> is fairness progress, not a Graph token.
TeamsAuth	Delegated refresh token for each Team authorised for Teams deep analytics. These are secrets protected by the storage account's access control.	Team id.
CallsWebhook (since Stable 1856)	Which Teams calls subscription the importer created: its Graph subscription ID, when it was created, and a SHA-256 fingerprint of its clientState, never the value. That's how the importer notices that the client secret has been rotated, and replaces the subscription. Added by PR #768; see Teams and calls troubleshooting → How the calls webhook authenticates notifications.	The subscription's notification URL, https://<web app>/api/CallRecordWebhook?cs=v1 (escaped as described below).
CallDeadLetters (since Stable build 1860, PR #870)	An administrator's latest request, from Administration → Service configuration, to reprocess or clear the Teams calls dead-letter queue, and its result once the importer has carried it out: times, the action, counts and codes only, never a message, call ID or user name. Deleting the row forgets the request. See Reprocessing or clearing from the portal.	LatestRequest.
CognitiveCache	Azure AI Language results for Teams messages. Entries live for 24 hours; expired rows are ignored and purged.	SHA-256 hash of the analysed text.
UserScopePurge	The record of each Administration → User scope purge: its progress, counts and who started it. A finished purge's record is kept for 90 days.	LatestJob, Job:<number>, Job:<number>:Cancel.
CopilotAuditBackfill	The record of each Administration → Copilot audit backfill: its window, progress, gaps and who started it, kept for 180 days after it last changed (the page shows only the latest). Plus one shared submission ledger, the backfills' Audit Search submissions in the last 24 hours and any pause, which every backfill counts against; it expires two days after its last update.	LatestJob, Job:<number>, Job:<number>:Cancel, SubmissionLedger.
AgentCostDelegatedAuth (1852)	The encrypted shared Copilot Studio billing connection/token cache. Treat the partition and backups as credentials. No in-memory credential fallback.	Connection, Cache_<generation>, Error_<generation>; see billing connection.
CopilotAdoptionSettings	33 governed settings in 1854, version and up to 50 administrator changes; older documents receive existing defaults for added fields.	Current. Use portal reset, not history deletion; activation windows are not stored tenant controls.
CopilotInvestmentAssumptions (since Stable 1857)	The assumptions the Copilot investment report models money with (currency, prices, hourly cost, three minutes and three realisation values, an optional note), their version, who last saved them and when, and the last 50 changes with each administrator's sign-in name.	Current. Change it on Copilot Investment Assumptions, not in the table: an unreadable row makes the report withhold money rather than replace it, and deleting the row returns the report to Never saved and loses the history. See where the assumptions are stored.
UserRefresh (1854)	Durable one-person profile/licence refresh requests, outcomes and requester identity, plus confirmation evidence. Personal operational data, not public telemetry.	Managed by User Licence Diagnostics; requests expire after 24 hours if unclaimed and records are retained for 30 days. Do not edit queue rows to bypass scope/locking.
LeadershipCohort (1852)	Settings, durable refresh request, latest outcome and bounded pages of internal SQL member IDs; no member UPN/name list.	Settings, RefreshRequest, Snapshot, Members:a:000 / Members:b:000, …
PromptCategories (1852)	Durable processing opt-in/taxonomy versions and the newest ten operational run counters. No prompt/model text.	current, taxonomy-<version>, runs. Category facts themselves are in SQL.
CopilotEnrichment (since Stable 1856, PR #768)	Durable retries for optional Copilot file/meeting metadata. Event IDs, attempts, due times and expiring leases only; no audit payload, user name or URL.	event-<event-id> anchor plus due-<UTC-ticks>-<event-id> index, changed together by Table transactions. Do not delete either row to force a retry; see enrichment recovery.
DlpHistoricalArchive (since Stable 1856, PR #768)	Immutable copies of incorrectly classified historical DLP workload metadata, including resolved file/message context, archived before correcting SQL activity counts. Contains tenant data and has no automatic expiry.	<event-id>-<SHA256-of-metadata>. Different versions keep different keys. See historical DLP repair.
AppInsightsImport (since Stable 1856)	The web-traffic import's ingestion-time cursor: everything Application Insights ingested before it has been saved. Deleting it makes the next cycle start again from the newest page view stored. See Web tracking troubleshooting → A day of web traffic failed to save.	IngestionCursor:<Application Insights application id>.

Key/value rows store the logical key in Key and the value in Value as plain text; timestamps use round-trip ISO 8601. Values longer than 32,000 characters, such as a page of the no-mailbox list, are gzip-compressed into ValueGzipChunks plus ValueGzip0, ValueGzip1 and so on, still in one row. A row holds at most about 1 MB, which is why the no-mailbox list, the one value that can grow with the size of the tenant, is split into pages rather than kept in one row. Cached rows can have ExpiresUtc. Row keys are normally the same as the logical key, except Azure Tables' forbidden characters (/, \, #, ?, control characters) and % are escaped as %XX — for example a guest UPN containing #EXT# becomes %23EXT%23. Keys over 255 characters are stored as %% plus the SHA-256 hash. The enrichment queue instead has typed entity properties and its explicit anchor/due keys.

Archive governance. DlpHistoricalArchive is evidence, not disposable cache state. Its document names, URLs and message identifiers have the same access-control requirements as SQL audit data. SQL retention and User scope > Purge do not remove this separate archive; include it in your approved retention, erasure and backup procedures. Do not publish its values in support issues or logs.

The installer always configures the Storage connection string. Without one (missing or empty; normally only a development setup), the web app and WebJobs still start, and keep ordinary cadence/checkpoint state in memory instead, just as they did when no Redis connection string was configured. In-memory state resets whenever the process restarts, so every restart brings a full user read and re-runs the interval-gated imports. The audit-import blob checkpoint is also held in memory, Teams deep analytics cannot be enabled (the Teams permissions page reports that Azure Storage is not configured), and a Copilot audit backfill can't be started. The WebJob log says so at startup: No Storage connection string configured - using in-memory import cadence gating (resets when the WebJob restarts). The Copilot enrichment queue and DLP archive (both since Stable 1856) require durable Storage; they do not share this fallback. The investment assumptions (since Stable 1857) cannot be saved without it either.

Forcing a re-import or retry by clearing state

Open the storage account in Azure Storage Explorer or the Azure portal's Storage browser, then go to Tables → AnalyticsState, filter by PartitionKey, and delete the row you want the importer to rebuild. Restarting the WebJob does not clear these rows; it only clears the in-memory developer fallback.

Goal	Delete this row	Safer portal option
Run the Graph user metadata import on the next cycle	PartitionKey = ImportSchedule, RowKey = GraphUsersMetadataLastImported	None needed.
Force a full read of all users	PartitionKey = UserImport, RowKey = UserDeltaCode-<tenant-id>-v2	Administration → User import → Clear checkpoint.
Retry Copilot Studio credits sooner	In 1852 with a current/previous connection: ImportSchedule / CopilotStudioCreditsLastImported_<current-generation>. The unsuffixed key is for no connection history only.	Reconnect when indicated; a new connection permits a next-cycle retry.
Retry Azure cost import sooner	PartitionKey = ImportSchedule, RowKey = AzureCostLastImported	None.
Re-run one usage-report loader	PartitionKey = ImportSchedule, rows UserActivityReportLastImported:<ReportLoaderName> and, if present, UserActivityReportNextAttemptUtc:<ReportLoaderName>	None.

The User import page now shows the table, partition and row key it clears, so administrators can audit or perform the same action manually.

Fail-open and fail-closed are feature-specific

A Table outage is not universally a harmless cache miss:

State/control	Behavior when unavailable
Ordinary cadence stamps	Fail open: an import can run again rather than being silently skipped
User delta checkpoint	A process without its last committed checkpoint defers the user import instead of starting an uncontrolled full crawl; inspect the checkpoint error
Copilot enrichment retry (since Stable 1856)	No memory-only substitute. Failure to enqueue keeps the source batch uncheckpointed; a retry-pass failure leaves existing work durable for a later cycle
Historical DLP archive (since Stable 1856)	Storage is mandatory. Archive write/read verification must succeed before any SQL metadata is removed; failure leaves the SQL evidence unchanged
Delegated Teams/billing credentials	Do not substitute anonymous/app-only consumption credentials or pretend a durable connection exists
Copilot Adoption score settings	Mandatory: fail closed with copilotAdoptionSettingsUnavailable / 503, not changed/default scores
Leadership state (1852)	Optional: one ten-second deadline for the complete storage read. Timeout/read failure withholds only the comparison (unavailable / stateUnavailable); a completed tenant summary/workbook remains available. No last-good or empty cohort presented as current
Prompt processing opt-in	Fail closed: unreadable/invalid or memory-only config cannot authorise model calls
Prompt run counters	Fail open for the main import; missing durable diagnostic history is not successful SQL fact confirmation
Global report filter	Stored in SQL, not AnalyticsState; unreadable/evaluation failure refuses scoped reports, not an unfiltered fallback

The general no-Storage developer fallback is not a production governance substitute. In particular it cannot save prompt opt-in, tenant score settings, leadership configuration or billing credentials.

Forcing a re-import (clearing the last-run flags)

The non-fresh Graph imports run only once per their interval. To force them to run again now, use one of these options:

1. Config flag (recommended for user metadata and Teams). Set ForceGraphMetadataImport=true on App Service → Configuration, let one import cycle run, then set it back to false (or remove it). While true it bypasses the gate every cycle.
2. Delete the relevant AnalyticsState row. Open the solution storage account in Azure Storage Explorer or the Azure portal Storage browser, go to Tables → AnalyticsState, filter for PartitionKey = ImportSchedule, and delete the row you want to re-run, such as GraphUsersMetadataLastImported, GraphTeamsLastImported or UserActivityLastImported. See Runtime state in Azure Table storage for the full row list.

Restarting the WebJob does not clear persisted table state.

If CPU is still a concern at Gentle, the other lever is to scale the App Service plan up (e.g. B2/B3 or a Standard tier with more vCPUs) — see Architecture and costs.

---

Return to Configuration Reference · All topics in this guide

Next: Importer Tuning Reference