Skip to content

CiteGEO 0.3.0

Choose a tag to compare

@ankit373 ankit373 released this 01 Oct 21:12
· 178 commits to develop since this release
v0.3.0
cf4bea8

First release under the CiteGEO name. v0.2.0 is inherited history and
describes a different repository.

This cycle is mostly about the difference between a number and a number you can
act on. Six findings from a read of the published work on generative engine
measurement are answered here, and every panel that reports a figure now also
reports what to do about it.

Measurement

Searching is measured before being cited is. Pr(cited) decomposes into
whether the surface searched at all, and the share cited among those that did.
Counting an answer that never searched beside one that searched and cited nobody
puts two populations under one denominator, and every rate built on it measures
a mixture.

The provider's own account of whether it searched is kept and preferred over
any inference. Only the provider can tell a search that found nothing from a
search that never ran, and that guess was the wider half of the band.

Browser surfaces are told apart by how they ground. An AI overview is
written from the search result it sits on; a chat surface decides per question.
Calling both optional threw the difference away.

Repetitions. A run can ask each question up to ten times, and the answers are
compared pairwise on the sources they cited. Published work puts the day to day
overlap of cited sources near a third, which makes a single pass close to a coin.

A sampling interval on presence. A Wilson interval beside the rate, and a
flag where the range is too wide for the figure to decide anything.

Mentions are checked against the answer the model wrote. Positions are
recounted from the text, and quoted evidence is confirmed to exist in it.

A cited page is counted once however the assistant spelled the URL. Tracking
parameters are dropped and the rest sorted, so one page is one page.

What to do about it

A task listing on the dashboard and the report, ordered by what is holding the
measurement back:

blocking no number can be produced at all
limiting a number exists and cannot be trusted or read
work the measurement is sound, this is what it says to do

Every task carries the observation behind it, and the button lands on the page
that clears it rather than the page being read. Where the product already knows
the cause it gives it, so "find out why 819 answers failed" reads "clear 561
answers lost to the provider returning an error".

Two messages that named the wrong cause are corrected: blocking an AI crawler is
no longer reported as the reason a brand is not cited, because measured across
millions of citations most sites disallowing those agents were cited anyway, and
the citation notice no longer tells somebody to get a provider with web search
when they have one with it switched off.

Also

The score states what it is built to maximise and what it cannot answer. Page
dates are kept on cited sources, so a stored page keeps the date rather than an
age that silently went stale.

Known issue

The Microsoft Copilot browser surface does not answer. copilot.microsoft.com
now redirects to copilot.com/chat, and the redirect drops the query the engine
navigates with, so it lands on an empty chat. The other three browser surfaces
are unaffected.

Verified

tests 1235  pass 1235  fail 0
9 emitted script blocks parse, 6 stylesheets balanced, 113 doc links resolve

Opened in a browser against a running server before main moved: ten pages render
with content, no console errors.