Four measurements of the UCP surface, no shared denominator: a proposal for a common reporting frame

[frmoretto]

Three independent measurements of the live UCP/MCP surface have been published in the last month, and a fourth has been announced:

	date	population	what counts as positive	result
@krisdiallo, #656	28 Aug	70 consumer storefronts	live UCP endpoint, tools/list	49 of 70
@deangoodman, Known Good, #856	17 Sep	50 random tenants on each of 10 platforms (tech-detection dataset, Aug snapshot)	MCP endpoint answering tools/list	Wix 50/50, Shopify 46/46
ours, 10.5281/zenodo.22893539	19 Sep	50 merchants from certified lists in Italy and Denmark, plus 6 US controls	valid /.well-known/ucp document (GET-only, no MCP calls)	12 of 50 in the panel; Shopify 15/15, everyone else 0/41
@unblinkr, Paddock, #779	announced 28 Aug	published profiles, daily	—	—

The results agree on the direction, but the numbers cannot go in one table: 49 of 70 and 12 of 50 are not the same measurement pointing at different values, they are different measurements. They differ on at least three things:

1. Population. Consumer storefronts, platform tenants, and certified merchants are three different frames, and all three are legitimate. The rate you get mostly reflects the platform mix of the frame; in our sample, Denmark doubled Italy entirely because it had twice as many Shopify hosts.
2. What counts as positive. Some count a tools/list that answers, others a UCP profile that parses. A 200 alone is not enough: in our run, 32 responses with status 200 on agentic paths were HTML error pages. Counted as positives, they would have overstated adoption by roughly a third.
3. Denominator. Known Good sets aside tenants that refused by robots.txt or could not be reached. In our run, five panel hosts refused (three 403s, all behind the same zone operator, one 404 on the root, one robots.txt Disallow: /), and three of six controls went unmeasured (one 429, two off-host redirects the instrument does not follow). Unless those hosts are reported, two identical populations can give different rates.

Proposal: a short, shared reporting frame, not a shared tool. Each census keeps its own method and states, in the same fields:

• sampling frame and draw (source, date, seed if any);
• paths checked, and in what order;
• positive criterion (status + content-type + parse, or protocol response);
• exclusions, counted: robots refusals, 429s, unreachable, TLS failures;
• run date and the UCP version each positive declares;
• two fields that are free to record and useful later: whether the profile publishes signing keys (keys vs signing_keys, per @westonale's point in UCP Readiness Checker - an open, API-first validator that grades any store A–F #656), and whether /.well-known/mcp/server-card.json returned a valid document.

With that, the next round is comparable, and a disagreement between two censuses means something instead of reflecting two frames.

I'm happy to write the first draft as a markdown file others can edit or reject.

Before that, it would help to hear from @krisdiallo, @deangoodman and @unblinkr: does this match how you would describe your own run, and is there a field you would drop or add?

[deangoodman]

Yes — that matches our run, and I'd add three fields rather than drop any.

For the record, ours in your frame: sampling frame and draw — 50 tenants per platform from the exact-label rows of a public technology-detection dataset (August snapshot), one seed (20260917), one UTC date, 7,774 requests. Paths, in order — robots.txt, platform fingerprint, the well-known sweep (/.well-known/ucp, /.well-known/mcp/server-card.json, /.well-known/agent-card.json, /.well-known/api-catalog, …), then JSON-RPC handshakes on any endpoint discovered. Positive — an MCP endpoint that answers tools/list; a parseable profile alone is not positive. Exclusions, counted — 4 robots refusals (Disallow: /); 21 "could not ask", of which 18 were our own concurrency colliding with one platform's edge and were re-asked serially (all 18 genuine negatives), the remaining 3 unreachable. Every exclusion is reported beside the count, never folded into "no". UCP version per positive and both platforms' full tool signatures are in our METHOD.md. Server card — recorded: all 96 positives were found through /.well-known/ucp; none through the server card. That last point is the reason for my first added field.

Three fields I'd add:

1. Discovery route per positive — which path actually led to the endpoint (card, UCP profile, documentation link). A census that probes only the card would have recorded every one of our 96 as absent. Route is the difference between "no endpoint" and "no endpoint by the routes we tried", and the second is the only honest form.
2. Attempts per host, and whether a negative was re-asked — one attempt and no retries is a defensible rule, but it has to be stated, and a 429 with Retry-After deserves its own count (it's the server asking, not refusing).
3. Platform per positive, and a hash of the tool signature. Our 96 were two signatures, one per platform, identical names in identical order — 96 tenants, two decisions. A merchant count and a decision count are different numbers, and the frame should let readers compute both.

One small disagreement: I'd keep the positive criterion as "protocol response" for the MCP surface and treat "valid profile document" as a separate, weaker row rather than an alternative — the same reasoning as your HTML-200 point.

Happy to review the draft. We're running a larger pass now that records every field above per host; when it's done I'll publish the frame fields alongside the counts so the two can be compared directly.

[frmoretto]

Thanks, this is the filled-in frame I was hoping someone would post.

All three fields go in.

Discovery route: agreed, and your result (96 found through /.well-known/ucp, none through the server card) is the best argument for it. "Absent by the routes we tried" is the only honest negative.

Attempts and re-asks: agreed, with 429 in its own column. It is neither a refusal nor a "no", and folding it into either changes the story.

Platform and a hash of the tool signature: agreed. Reporting the merchant count and the decision count side by side answers most "is adoption growing" questions before they are asked. 96 tenants and two decisions is a very different finding from 96 adopters.

On the positive criterion I agree with your disagreement: two rows, not two alternatives.
"Protocol response" for the MCP surface, and "valid profile document" as a separate, weaker row.
That also keeps the HTML-200 problem out of both.

I'll write the frame up as a short draft with your fields and the two rows, and post it here for review.
Looking forward to the larger pass.

Francesco Marinoni Moretto

[frmoretto]

As promised, a short draft of the reporting frame, with Dean's three fields and the split rows. Edits welcome, it's meant to be filled per run.

Run fields

field	what goes in it
sample	frame, size, selection rule, seed, UTC date
paths	paths probed, in order
requests	total sent, attempts per host, whether negatives were re-asked, 429s with Retry-After counted separately, User-Agent string
could not ask	robots.txt refusals and unreachable hosts, as separate counts, reported beside the result and never folded into "no"
discovery route	per positive: /.well-known/ucp, server card, documentation link, or feed (dev.ucp.common.feed, #866). Feed hosts are a subset of profile hosts, never added on top
platform	per positive, plus a hash of the tool signature, so readers can compute both a merchant count and a decision count
validator	tool and version, schema commit sha, $def used

Outcome rows (hosts, not requests)

row	counts as
1. HTTP 200	reported, never a positive on its own
2. parses as JSON	body is well-formed JSON
3. valid profile	validates against business_schema in profile.json
4. protocol response	an MCP endpoint that answers tools/list. This is the headline positive

A row is only filled if the run measured it. "Not measured" is a valid entry, a guess is not.

Our run, filled in

	/.well-known/ucp
sample	50 certified merchants + 6 US controls
1. HTTP 200	18
2. parses as JSON	15
3. valid profile	15 (ucp-schema 1.4.1, schemas at d3ccb55, business_schema)
4. protocol response	not measured, we sent no handshakes

One note on our side: at the time, our "valid" row only checked that the body parsed as JSON. We re-ran the 15 through ucp-schema this week and all 15 pass, same result at current main.

Francesco Marinoni Moretto

[deangoodman]

Thanks Francesco — this is the frame, and I'd use it as written.

One addition to "could not ask": split it by whose side the reason is on. robots refusals, dead DNS, TLS failures and 5xx are the site's; 429 with a Retry-After, and timeouts on an origin we were sharing with other hosts in the same run, are plausibly the prober's. We report the second group as its own line because it is the only one the prober can reduce, and folding the two together hides whether a run was polite.

Two small things on the rows. Row 4 should carry the discovery route per positive, as your run-fields table already asks, so that "answered tools/list via the profile's endpoint" and "answered via a server card" stay separable. And for row 3, naming the schema commit is what made your 15/15 re-check possible — keep that mandatory.

Your note that the earlier "valid" row was a JSON-parse check is exactly the kind of thing this frame makes visible, and re-running it is the right response. We'll fill the frame in for our next published pass rather than quote numbers that haven't been through our own checks yet.

Dean

[frmoretto]

Thanks Dean, all three go in.

Splitting "could not ask" by side is the better cut: the site-side line tells readers what the web refused, the prober-side line tells them whether the run was polite, and only the second is something a prober can fix between runs.

Row 4 carries the discovery route, and the schema commit becomes mandatory for row 3. Without it a "valid" count can't be re-checked, and re-checking is what caught our JSON-only row.

I'll post the updated frame as a single top-level comment, so anyone arriving later finds one version rather than a thread of edits.

Looking forward to your next pass in it.

Francesco Marinoni Moretto

[frmoretto]

Updated reporting frame, folding in Dean's review.
This is meant to be the one version to copy; the earlier draft stays in the thread for history.

Run fields

field	what goes in it
sample	frame, size, selection rule, seed, UTC date
paths	paths probed, in order
requests	total sent, attempts per host, whether negatives were re-asked, User-Agent string
could not ask, site side	robots.txt refusals, dead DNS, TLS failures, 5xx
could not ask, prober side	429 with Retry-After, timeouts on an origin shared with other hosts in the same run. Its own line, because it is the only one the prober can reduce
discovery route	per positive: /.well-known/ucp, server card, documentation link, or feed (dev.ucp.common.feed, #866). Feed hosts are a subset of profile hosts, never added on top
platform	per positive, plus a hash of the tool signature, so readers can compute both a merchant count and a decision count
validator	tool and version, schema commit sha, $def used

Both "could not ask" lines are reported beside the result and never folded into "no".

Outcome rows (hosts, not requests)

row	counts as
1. HTTP 200	reported, never a positive on its own
2. parses as JSON	body is well-formed JSON
3. valid profile	validates against business_schema in profile.json at a named schema commit. No commit, no row 3
4. protocol response	an MCP endpoint that answers tools/list, with the discovery route for each positive. This is the headline positive

A row is only filled if the run measured it. "Not measured" is a valid entry, a guess is not.

Our own run, filled in against the first draft, is here; it does not split "could not ask" by side yet.

Francesco Marinoni Moretto