Skip to content

Copilot Adoption Specification

Jump to bottom
Sam Betts edited this page Nov 4, 2026 · 1 revision

This page is for contributors and reviewers. It is the specification for Copilot Adoption (Insights → Copilot → Copilot Adoption, #/insights/copilot-adoption): what the tool is for, the design principles every figure has to follow, and the checklist every change is reviewed against. It is not a how-to: programme leads should start with the user guide, and administrators with Setup.

For how the figures relate to Microsoft's research, see Work Trend Index.

Use this page whenever the tool is changed. Every pull request that adds or alters a figure, chart, list or export in this area should be reviewed against the feature review checklist at the bottom, and this page and the reference pages should be updated in the same change. If a proposed feature does not fit the purpose below, that is a finding worth raising rather than a reason to quietly widen the scope.

Contents

1. Purpose

The tool answers two spending questions and nothing else:

  1. Who is paying for a Microsoft 365 Copilot licence they are not using?
  2. Who would benefit from a licence they do not have?

Everything on the page exists to support one of those two questions, or to justify the numbers that answer them.

The reason the tool exists at all is that Microsoft's own Copilot usage reporting cannot answer either question well:

Gap in Microsoft's reporting What this tool does instead
"Active user" is a yes/no flag — one interaction in 28 days counts the same as fifty Grades engagement 0–100 from frequency, depth and breadth, and bands it
Covers licensed users only Also counts unlicensed Copilot Chat use, which is the strongest evidence of unmet demand there is
Unusable when the tenant conceals user information Audit-derived figures are unaffected; the tool says which source each figure came from
No organisational metadata to act on Joins department, job title, country, manager and account status, and exports the lot to CSV
Cannot tell "never started" from "tried it and stopped" Splits those into Never used and Dormant, because they need opposite responses

Audience

Two readers, in this order:

  • The person who owns the Copilot budget — needs the headline figures, the funnel and the department breakdown, and needs to be able to defend every one of them in a renewal conversation.
  • The person who runs enablement — needs the per-user lists, the recommended action for each person, and the CSV to hand to a department lead.

Non-goals

  • Not a general Copilot usage report. Prompt content, sentiment and key phrases for their own sake belong in the Copilot reports (Insights → Copilot → Usage and Insights → Copilot → Prompt insights), not here. If a proposed chart does not change a licence decision or an enablement decision, it does not belong on this page.
  • Not an employee-performance score. Named lists support licence/enablement decisions. The Agents tab's See-PII-gated Heaviest agent users ranking is a targeted licence/advocate view, not a claim about productivity or a general fluency leaderboard.
  • Not a replacement for Microsoft's compliance/audit reporting.

Reporting direction. This portal page replaces the Power BI adoption report requested in #261, which is deprecated. The other Power BI templates remain separate, supported reporting solutions.

2. Design principles

These are the rules that make the tool trustworthy. They are the ones most worth checking a new feature against.

2.1 Every assertion must explain itself, on the spot

Any figure that makes a claim carries an information ("i") button giving:

  • What it claims, in one sentence;
  • How it is worked out, in words;
  • The formula, shown verbatim so it can be checked rather than trusted;
  • The source import, and any caveat that comes with it.

This is not decoration. Every number here ends up in a licence negotiation or a conversation with a department lead, and the first two questions asked about all of them are "what exactly does that mean?" and "how do you know?". A methodology tab alone is not sufficient — by the time the reader has the question they are looking at the number, not at the tab.

Exception: figures that are self-defining need no tip. "Adoption by country" is a count of users by their country attribute; there is no hidden judgement in it. "Habitual users" is the opposite — it is a threshold on a weighted score, and is meaningless without its definition.

Rule of thumb: if the tool applied a threshold, a weight, a normalisation or a classification, it must explain itself.

2.2 Say a thing once

Where an explanation is the same for a whole group of rows, it is stated once for the group, not repeated on every row. A "recommended action" column that prints the identical paragraph for every user in a band is a hundred copies of one sentence dressed up as a hundred findings.

The useful form of that information is the size of each job — "coach 76 people" is a plan. The per-user list carries a short tag; the meaning sits above the table and on the Enablement plan card.

The CSV is the exception: it keeps the full sentence on every row, because a spreadsheet gets sorted, filtered and split up, and cannot rely on a legend being nearby.

2.3 Never show a percentage without its counts

0% across six licences and 0% across six hundred are the same percentage and completely different decisions. Segment tables always show licences, active, habitual and never-used alongside the rate.

2.4 Never claim a metric the data cannot support

  • Cowork adoption is only shown when Cowork activity was actually detected — on a tenant that never enabled it, "0%" reads as a failure rather than "not applicable here".
  • Segments below the minimum licence count are omitted rather than shown as noise.
  • Every missing or degraded import produces an explicit warning on the result, not a silently wrong number.
  • The data sources section names which import supplied each figure and the snapshot date it came from.
  • A past date range never names people, because it can only describe them with today's departments, managers and account status (§3.7).

2.5 The working is always available

Each chart backed by a query exposes its SQL through the </> SQL popover. These numbers get quoted in licence negotiations, so an admin has to be able to verify them independently.

2.6 The export must be exactly what is on screen

Filtering, sorting and paging happen server-side against a single cached analysis, and the CSV export takes the identical parameters. A CSV that quietly disagrees with the chart above it is worse than no CSV.

"Exactly what is on screen" means data fidelity, not layout isomorphism. The Excel workbook is not a screenshot of the page: it re-lays the same numbers into sheets suited to a spreadsheet, and adds native Excel charts so the recipient can pivot and re-chart them. What it must never do is contain a number the page would not produce for the same filters and period — which is why, for example, the workbook's band percentages divide by the same analysed denominator the on-screen donut uses.

2.7 Thresholds are judgement, and must be visible and tunable

Every weight and threshold lives in CopilotAdoptionOptions, is unit-testable, and is returned to the client with the results so a figure can always be traced back to the rule that produced it. The UI must quote the options it was given, never hard-code a duplicate of them.

A regression to watch for: because the options object is serialised, every property on it needs its [JsonProperty] camelCase name. Without it the whole methodology tab renders NaN% and blanks — which is exactly what happened before this was fixed.

2.8 Never encode meaning in colour alone

This page's whole audience is people in meetings: it gets projected, printed, and pasted into decks in greyscale. Around one man in twelve cannot reliably separate the red/green ends of the band palette. So every chart that uses colour to carry meaning must carry it a second way as well — a letter, a number, a label, or a separating hairline.

Applied: the intensity bubble chart draws the band initial (C/E/D/T) inside each bubble and repeats the pairing in a legend; the concentration bar numbers its cohorts and separates adjacent slices with a hairline; the band donut and every table label their slices in text.

2.9 A named number must be reachable

Any aggregate that names a population ("76 people need a first habit") must be clickable through to that exact population, with the same members. A number the reader has to reconstruct by hand from filters on another tab is an invitation to reconstruct it wrongly — and then to quote the wrong one.

The drill-through must filter on the same key the aggregate was grouped by. The enablement plan groups by action code, so its drill-through filters by action code; filtering by band instead would look right and land on a different set of people.

2.10 Design the empty state

Zero data is a normal state on a freshly installed tenant, not an error. A page that answers it with a grid of "no data" cards reads as broken software, tells the reader nothing about which of several quite different causes applies, and gives them nothing to do next. Every screen that can legitimately have nothing to show needs a written answer to "why is this empty, and what do I do about it?"

2.11 Modelled estimates: hours only, never money, never added together

The page carries three modelled estimates, each for a different decision: the licence estimate (the time a licence could give back to the people recommended for one), the Cowork estimate (the time Cowork could add on top of Copilot) and time already saved by seat holders (§3.6). All three follow the same rules:

  • Hours, never money. No currency, hourly rate, price or value appears on the page, in the API, or in the CSV and Excel exports. Pricing a modelled saving would state a figure the product cannot evidence, and there is no defensible fully-loaded hourly rate to price it with.
  • Never added together. Each answers a different question, so no figure sums two of them.
  • A range, labelled as modelled, with its working. Each is a low-to-high range, and the assumptions that produced it are listed on screen and in the workbook. The low end applies a conservative share of every assumption: 50% by default, shared by all three estimates.
  • A reader's own assumptions stay with that reader. Anyone can replace an assumption for their own browser tab. Nothing is written to the database, so one reader's figures cannot become someone else's report. An Excel export carries the changed figures as query parameters, and the estimate's sheet says they were entered in the portal.

6. Feature review checklist

Run through this for every change to the Copilot Adoption area.

Purpose

  • Does this change help answer "who is wasting a licence" or "who needs a licence"? If not, does it belong in the Copilot reports (Insights → Copilot → Usage or Prompt insights) instead?
  • Does it change a decision, or is it interesting-but-inert?

Explainability

  • Does every new figure carry an "i" tip with what, how, the formula, and the source?
  • Does the tip quote the actual options returned by the API rather than hard-coded numbers?
  • Is any new threshold, weight or normalisation added to CopilotAdoptionOptions — with a [JsonProperty] camelCase name — and to the methodology tab and the metrics reference?
  • Is any explanation that is identical across rows stated once rather than per row?

Honesty

  • Are percentages shown with their absolute counts?
  • Does the figure degrade to an explicit warning (not a silent zero) when its import is missing?
  • Is a metric hidden entirely rather than shown as "0%" when the feature is not in use on the tenant?
  • Does a normalised figure keep the same meaning across all period options (7 / 28 / 90 / 180 days, and an explicit date range)?
  • Does a new seat-dependent figure use the period's one seat population (today's holders for a rolling window, licence history for a past range), rather than querying licence assignments itself? (§3.7.)
  • Is a modelled figure in hours only, shown as a range with its assumptions, and never added to another estimate? (Principle 2.11.)

Consistency

  • Is the scoring logic in CopilotAdoptionScoring (one implementation, unit-tested), not duplicated in SQL or TypeScript?
  • Do the CSV export, the on-screen list and the summary all come from the same cached analysis?
  • Do band and action colours match the shared palettes, so two views never appear to disagree?
  • Does an aggregate average the same population its other dimensions describe? (The frequency/intensity bubble colours by the active users' mean, because both its axes are active-only; the department table uses the whole-population mean, because it is reporting on all the licences.)
  • Does a chart's centre total, or a "these counts sum to…" claim, actually equal the sum of what is drawn? Aggregates built from the scored rows are capped by MaxLicensedUsersScored and are not necessarily the licence count.

Charts

  • Does a bubble/area encoding scale by area, with no constant added to the radius? Adding a fixed minimum radius silently inflates the smallest categories.
  • Is there axis headroom so the largest marker and its label are not clipped by the SVG viewBox?
  • Does a treemap lay its rows along the shorter side of the remaining space? Using the longer side still fills the area exactly but produces unreadable slivers.
  • Is the chart type chosen for the question, not for novelty? Doughnuts and stacked areas are for composition; bars and lines are for comparison; a radar is for shape. A radar with many axes, or with axes on different units, looks impressive and says nothing.
  • Does the chart survive greyscale and colour-blindness — is every meaning carried by something other than hue as well? (Principle 2.8.)
  • If a visual is added to the page, is it also added to the Excel export — and vice versa? The two are meant to carry the same report.

Navigation and empty states

  • Does every aggregate that names a population drill through to exactly that population, filtered on the same key it was grouped by? (Principle 2.9.)
  • Is the resulting filter visible and reversible, rather than a hidden state the reader cannot see or clear?
  • Does this screen have a designed empty state that says why it is empty and what to do next? (Principle 2.10.)
  • Are two different measures ever given the same name? "Habit rate" (weighted score) and "how often people open Copilot" (raw days) were one word apart from being indistinguishable.
  • Does each action/label name the step to take rather than the state the reader is in, and is it distinct from every other label? Pinned by EveryActionLabel_IsDistinctAndNamesTheStepToTake.

Excel export

  • Does any new figure appear on the correct sheet, with its explanation?
  • Do the workbook tests still pass? They generate a real file and validate the package, and they are the only thing standing between a malformed part and a customer seeing "Excel found unreadable content".
  • Are new thresholds added to the Report sheet? A snapshot that does not record the rules it was scored by cannot honestly be compared with another one.

Scale — assume a tenant of ~200,000 users

  • Any new query batched rather than per-user, and SARGable (no LOWER() on an indexed column)?
  • Any new aggregation done in the existing in-memory pass rather than as another database round trip?
  • If a new index is proposed, is there a measured before/after benchmark? (See the engineering notes on proving schema changes.)

Documentation

  • Are this page and the reference pages updated in the same pull request?
  • Do new user-visible behaviours get a plain-English release-note line?

Next: Copilot Adoption Metrics Reference: every measure, formula and default.

Reference set: Specification → Metrics reference → Sources and outputs reference → Design notes

Start here

Copilot analytics

Using the portal

Install & configure

Beyond Copilot

Operate

Upgrading from an older build

Reference

Project

Clone this wiki locally