Skip to content

Copilot Adoption Leadership Comparison

Sam Betts edited this page Nov 5, 2026 · 10 revisions

This page is for administrators who want to add the optional leadership comparison to Copilot Adoption, and for programme leads who want to know what its card means. It needs Stable 1852 or later.

The Copilot Adoption Analyst view can compare the Copilot adoption of one named leadership group with the whole tenant: adoption rate, habit rate and average adoption score, each with the gap in points. It answers the question "are the people leading the organisation using Copilot as much as everyone else?" without naming any of them.

The feature is off until an administrator turns it on, and it only ever shows totals.

Contents

What the report shows

The Leadership group compared with the tenant card on Copilot Adoption → Analyst view, and the Leadership comparison sheet of the Excel export, show:

Measure Leaders Tenant Gap
Adoption rate % of licensed leaders active in the window % of all licensed users active leaders minus tenant, in percentage points
Habit rate % of licensed leaders with a habitual band the same for all licensed users percentage points
Average adoption score mean score of licensed leaders mean score of all licensed users score points

It also shows how many licensed leaders there are, how many of them are active, and when the membership was last read. Both sides use the same scoring rules, window and denominator (every scored licensed user) as the rest of the page. The tenant figures include the leaders: the card compares the leaders with everyone, not with everyone else.

Only group members who hold a Copilot licence count. A member without a licence is not in the leaders' figures and doesn't count towards the minimum.

This is different from manager modelling, which derives people managers from the current direct-manager relationship. A leadership group is explicitly chosen by an administrator.

The comparison stays on its own Leadership comparison workbook sheet. It is not part of Snapshot facts, and the workbook-comparison script does not compare it. Record the membership-read time when comparing leadership sheets manually.

Turn it on

You need the portal's Administration permission (Portal permissions). See PII is not needed, and it doesn't change what the report shows.

  1. In Entra ID, create a security or Microsoft 365 group (or choose an existing one) whose direct members are the leadership team. Copy its Object ID.
  2. In the portal, go to Administration → Copilot & reporting → Leadership group (#/admin/leadership-cohort).
  3. Paste the object ID and select Save and request membership read. This saves the setting durably; it does not wait for Graph or SQL. Keep the Office 365 activity importer running.
  4. After the importer's next available cycle, select Check status. This retrieves the outcome without requesting another read: the group's name, the number of direct members, how many matched users the solution has imported, and the time of the read.
  5. Open Copilot Adoption → Analyst view. The card appears for every report reader.

To turn it off, clear the group ID and save. The card then says no group is configured.

Prerequisite: the solution's storage account. The setting is kept in Azure Table storage (see below). If the deployment has no Storage connection string, the page refuses to save, because the importer and every web instance must see the same setting.

Permissions it needs

The runtime app registration reads the group with application permissions. GET /groups/{id}/members accepts any of these (Microsoft Graph reference):

Permission Notes
Group.Read.All Already in the documented runtime permission set. A deployment that follows it needs nothing new.
GroupMember.Read.All The narrower option, if your registration doesn't have Group.Read.All. The installer does not add it. Add it by hand under API permissions → Add a permission → Microsoft Graph → Application permissions, then select Grant admin consent.

Without either, the admin page shows Permission missing: grant Group.Read.All or GroupMember.Read.All. The report card says the application is missing permission to read group members. A newly granted permission can take a few minutes to reach the token. Select Request membership read to queue a retry rather than waiting for the next scheduled read, then Check status after the importer has run.

How membership is read

  • Direct members only. The solution reads /groups/{id}/members/microsoft.graph.user. It does not expand nested groups, and it does not follow the manager hierarchy. Put the leaders in the group itself.
  • When. The activity importer reads the membership in its next available cycle when it is due: 6 hours after a successful read, 1 hour after a failed one, or after the group is changed. An administrator can also request a read from the admin page. Neither admin HTTP requests nor report requests call Graph or run the SQL membership lookup. Graph retries can delay the worker.
  • Pending requests. A manual request is durable and survives restarts. Until the worker saves an outcome acknowledging that request, the comparison withholds the previous figures and reports pending. A failed attempt acknowledges the request too, but shows the failure rather than old figures. A newer request made while the worker is writing stays pending for another cycle. The web app caches settings, read status and the request for 60 seconds, so another web instance can briefly show its previously cached comparison; a save/request invalidates the cache immediately in the instance handling it.
  • Capped. At most 10,000 direct members, about eleven Graph calls whatever the tenant's size. A larger group is refused with too large rather than partially read. It is a leadership group, not a department.
  • Matched by object ID. Each member's Entra object ID is matched to the users the solution has already imported (dbo.users.azure_ad_id). A member the user import hasn't reached yet isn't counted until it has.
  • Current membership, even for past periods. The solution keeps no membership history. When you look at an earlier reporting period, the comparison uses today's members, not the people in the group at that time. Keep that in mind before reading a trend into it across a reorganisation.
  • Bounded portal state read. One ten-second storage deadline covers the complete optional read of settings, snapshot/refresh-request headers and all membership pages, including lazy Table opening and SDK retries. It is not a deadline for the whole tenant analysis or for the importer's Graph membership refresh.

When the comparison is hidden

The card always explains why it isn't showing figures. Nothing below is relaxed for readers with See PII.

Optional leadership does not block a completed tenant report. If its storage deadline expires or state cannot be read, the leadership result is Unavailable (unavailable / stateUnavailable), with its counts, rates and scores withheld as null, not zero or last-good values. Provided the tenant analysis completed and the request was not cancelled, the summary and Excel workbook still return HTTP 200, with leadership's unavailable status. This differs from mandatory Copilot Adoption score settings: unreadable scoring rules refuse the report with HTTP 503 / copilotAdoptionSettingsUnavailable.

Cancelling a summary or export request cancels that request only: it isn't recorded as a storage failure, and it doesn't cancel the shared tenant analysis. A read that finishes after its deadline can't publish membership, or overwrite a newer save or membership request.

The card says Why What to do
No leadership group is configured The default. Turn it on.
Fewer than 10 members of the leadership group hold a Copilot licence The cohort is too small to report without singling people out. The minimum is fixed at 10 and can't be lowered. That is twice the default department minimum (MinSeatsPerSegment, 5), because a leadership team is small, senior and well known. If you raise MinSeatsPerSegment above 10, this minimum rises with it. Not even the count is shown. Nothing. This is intended.
Fewer than 10 licensed people are outside the leadership group The complement is too small. The tenant's figures sit next to the leaders', so tenant minus leaders would describe everyone outside the group. If that is 1–9 people, the subtraction would single them out. A group that contains every licensed user is still compared, because there is no one else to expose. Nothing. This is typical of a small pilot where almost everyone licensed is in the group.
Shown only for the whole tenant The page is narrowed by an email domain, a filter or the global user filter. Inside a narrowed view, a handful of leaders could be singled out. Clear the filters.
Members haven't been read yet The group was just set or changed, or a manual membership request has not yet received a saved worker outcome. Previous figures are withheld once the web app rereads the state. Keep the activity importer running, then select Check status after its next cycle. Request another read only if a retry is needed.
Unavailable: group not found / missing permission / too many members / couldn't be read The last read failed. The previous good membership is not used as a fallback: a failure hides the figures rather than quietly comparing against an old list. See Troubleshooting.
Last read successfully on date, too long ago The last successful read is more than 72 hours old, so people may have joined or left. Check the importer is running (Monitoring).
Settings storage could not be reached The web app couldn't read/parse settings, the snapshot, the durable request or membership pages from Azure Table storage, or the complete read reached its deadline. Only the comparison reports "unavailable", never "not configured" or previous figures. Failed reads are cached for 15 seconds before retrying. Check Table endpoint/DNS/connectivity and the runtime identity's Storage Table Data Contributor role, then retry after the failed-read cache expires.

When the licensed-user detail hits its row cap, some leaders may not have been scored. The card and the export then say the figures are incomplete.

Privacy and where the data is stored

  • Report readers get totals only. The report API, the card and the export never contain the group's name, its ID or any member. They carry status and reason keys that the portal translates.

  • Only administrators see the configuration. The group ID, its display name, the member counts and the read status are on the admin page, behind the Administration permission. The group's display name is shown exactly as it is in Entra ID and is never translated.

  • Storage. Everything is in the solution's own storage account, in the AnalyticsState table, partition LeadershipCohort:

    Row key Holds
    Settings the group's object ID, a revision and when it was saved
    Snapshot the last read: group display name, status, failure kind and HTTP status, times, member and match counts, which page slot is current, and the request ID the worker acknowledges
    RefreshRequest the latest manual request ID and when it was requested; applies to the currently configured group, never restores older settings
    Members:a:000, Members:b:000, … the matched members as internal SQL user IDs (integers), 2,000 per row, in two alternating slots so a read in progress never mixes with the previous one

    No UPNs, names or Entra object IDs of members are stored. There is no database schema change.

  • Turning it off clears Settings, and the report stops using membership immediately on that web instance (other instances within their 60-second cache window). The last Snapshot, RefreshRequest and Members:* rows stay until they are overwritten. To remove them, delete the LeadershipCohort partition's rows with Azure Storage Explorer or the Azure portal's Storage browser.

Troubleshooting

Admin page status Likely cause
Saved — awaiting the activity importer Settings or the manual request were saved, but the worker has not acknowledged them yet. Keep the importer running and use Check status; repeatedly requesting reads can keep a newer request pending.
Permission missing: grant Group.Read.All or GroupMember.Read.All (Graph 401/403) The runtime app has neither permission with admin consent. See Permissions it needs.
Group not found in Entra ID (Graph 404) The object ID is wrong, or the group was deleted. Copy the Object ID, not the group's email or display name.
Group too large to compare The group has more than 10,000 direct members. Use a smaller group.
Failed, Microsoft Graph returned an error Graph returned another error, such as throttling or an outage. The read is retried an hour later. Since Stable 1859 (PR #812): also when Graph returns a page of members blank or without its list of members. Builds before 1859 save the members read up to that page as the whole group, and report Succeeded
Failed, Microsoft Graph couldn't be reached The importer couldn't get a Graph token or connection. Check the runtime credentials and outbound network rules.
Failed, The members couldn't be matched to users in the database The SQL lookup failed. Check the database is reachable.
Succeeded, but matched is well below direct members Those members haven't been imported as users yet, for example because a user-import filter excludes them. See Configuration Reference.
Succeeded, but the card says fewer than 10 hold a licence Only direct members with a Copilot licence count. Members of nested groups are not included.

Importer log lines to search for: Leadership cohort refreshed: on success, and Leadership cohort refresh skipped this cycle for an unexpected error.

Related pages


Previous: Copilot Adoption Settings · Next: Copilot Adoption Troubleshooting

Administrators' path: Setup → Settings → Leadership comparison → Troubleshooting

Start here

Copilot analytics

Using the portal

Install & configure

Beyond Copilot

Operate

Upgrading from an older build

Reference

Project

Clone this wiki locally