You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
This commit was created on GitHub.com and signed with GitHub’s verified signature.
The walk and the cap eviction were selecting an author's 100 documents by
different keys. `trimAuthorDocumentsStatement` keeps the newest by
`published_at` — the key every read path orders by — while the walk took
`listed.slice(0, MAX_DOCUMENTS_PER_AUTHOR)`, the first 100 in whatever order the
PDS served. Two rules over one cap, so they fought.
Listing order is not publication order, in two shapes both live in production.
`listRecords` order is per implementation: a Bluesky-hosted PDS serves newest
rkey first, Bridgy Fed serves oldest first and rejects `reverse` with a 400. And
rkey order is *write* order, which for a publication that imported a back
catalogue in one batch spans two decades of `publishedAt` in a single page.
That alone would be cosmetic. What made it reader-visible is that the walk then
pruned every row it hadn't listed, so a reconcile deleted exactly the documents
the firehose had gotten right — one subscribed publication was a reconcile away
from losing its 2026 posts in favour of its February back catalogue, another
from showing 2007 archive imports instead of the last five years.
Three changes. `listAuthorDocuments` pages to `MAX_LIST_PAGES` instead of
stopping at the first full page: the old condition made the page bound and the
row cap the same thing, so the constant's own "up to 500 scanned" was never
true, and selecting a cap needs a set wider than the cap. It returns
`exhaustive` alongside the records, the way `AuthorCollectionListing` already
does. The walk ranks by `publishedAtMs` with a `record_uri` tie-break, matching
the eviction's `ORDER BY published_at DESC, record_uri DESC` exactly. And the
prune runs only against an exhaustive listing — a partial one falls back to cap
eviction, which ranks over everything we hold rather than over what one listing
reached, so firehose rows survive a walk that could not see them. One statement
either way, so `BACKFILL_QUERY_COST` is unchanged.
Still open: a cold-start subscribe to an oldest-first repo larger than
`MAX_LIST_PAGES` pages starts on old documents. It is no longer destructive and
converges as the author publishes, but the first render is wrong; fixing it
needs paging to the end for detected-ascending repos.
RUNBOOK §4e gains the two things this makes non-obvious: shadow-compare shares
the proxy's selection rule, so `clean` is not proof for an author at the cap,
and which drift shapes mean D1 is the better side rather than the broken one.
Co-authored-by: Claude Opus 5 <noreply@anthropic.com>