Public Read Backfill

Public-read backfill and repair

How to fill and repair the four containers behind ADR-011-Public-Read-Surface — commentPeriods, comments, notifications and lists — and the matching eagle-api script that re-sends what the push missed. Design and containers: Sync-Architecture. Query shape: Search-Index-Reference.

Why a backfill exists at all

eagle-api pushes commentPeriod, comment, organization and projectNotification to DEMI on every write, the same way it already does for project and document. The push covers everything written from here on. It does not cover what predates it: the corpus at cutover, and List, which has no eagle-api write controller at all — list items change only by migration, so nothing ever pushes them.

eagle-demi: db:seed-public-reads

src/scripts/seed-public-reads.js, run on the devbox: Operating-DEMI-Data-Plane#reach-the-data-plane or the repo README's "Running anything against the database". Reads eagle-api over HTTPS (EAGLE_API_BASE), writes Cosmos (COSMOS_ENDPOINT, COSMOS_NOSQL_DATABASE) through the same mirrorFromEagle function the push handlers call, so a backfilled row and a pushed row are identical.

npm run db:seed-public-reads                      # dry run: fetches, counts, writes nothing
npm run db:seed-public-reads -- --live
npm run db:seed-public-reads -- --live --only lists
npm run db:seed-public-reads -- --live --since 2026-01-01
npm run db:seed-public-reads -- --live --state ./repair.state.json

Stages run in dependency order and --only selects among them without reordering: lists → organizations → notifications → updates → commentPeriods → comments. Projects are not this script's: run db:seed-nosql first.

Resumable. A finished stage is recorded in the state file (./seed-public-reads.state.json, or --state <path>) and a later run skips it. The comments stage records each comment period whose comments all wrote, so a run killed part way through resumes at the period it stopped on. A stage that logged any error is not marked complete, so a partial stage is retried in full next time. --only overrides the state file: naming a stage runs it whether or not it is marked done, which is how a repair is asked for. --only comments also ignores the per-period checkpoint, so it re-walks every period rather than only the ones the plain run had not yet reached.

Rerunning --only updates rewrites existing rows. A legacy Update with a missing read (absent or null) takes its parent's read on the rerun. read: [] does not, because eagle-api hides it. See Eagle-Mirror-Reference#updates-publish-gate-and-announce.

A stage that skipped rows because their parent was not yet in DEMI is not checkpointed complete either — those rows are still owed, not permanently absent, and a plain rerun retries them once the parent lands. The log line and the state file count this as dropped, apart from skipped: a row outside --since was never wanted, a dropped row still is.

Fetching a comment page treats a missing x-total-count header, or a body shaped as [{total_items, results}] instead of a bare comment array, as an error rather than a page of zero comments. Either shape means eagle-api broke the count=true contract this stage depends on; before this was a silent zero that still checkpointed the period as done.

Parent admission. A comment period's parent is a project or a ProjectNotification — Eagle's project reference holds either id, and a notification wins when both exist for the same id (src/helpers/parent-admit.js, pickParent/admitParent). A period or comment whose parent is not in DEMI is skipped, not written with no ACL.

eagle-demi: reconcile-eagle.js

src/scripts/reconcile-eagle.js diffs the ids Eagle publishes against the rows DEMI holds, for projects, documents, commentPeriods, both lists kinds, notifications and updates on every run. It reports drift and deletes nothing.

node src/scripts/reconcile-eagle.js              # comment periods, lists, notifications, updates
node src/scripts/reconcile-eagle.js --comments   # and comments

comments is swept only behind --comments, because the sweep costs one eagle-api request per comment period plus one Cosmos query per period. The nightly reconcileEagle timer calls it with no flags, so its alert line reports comments: skipped rather than a false zero.

Three findings, not two:

• unresolvedParent — a period or comment Eagle publishes whose own parent project is unpublished or gone. The seed drops these too, so they are reported separately and excluded from drift=.
• misfiledParent — a row that IS mirrored, but sits under a parent the admission rule would not choose today. This is the one check that catches the notification-vs-project collision: a Track project can carry a ProjectNotification id in epic_guid, and before the notification was made to win, a period referencing that id was filed under the Track project's partition and ACL instead. Both sides hold the id, so an id-set diff alone calls it clean; misfiledParent is what finds it, and it counts toward drift=.

The repair, in order:

npm run db:seed-nosql -- --live --only documents
npm run db:seed-public-reads -- --live --only commentPeriods,comments --state ./repair.state.json

Documents first, because a document moves partition on its own re-seed. The period re-mirror then moves the row to the notification partition, deletes the row left under the project, and cascades the corrected access list to its comments; the comments stage follows because a comment stores its period's projectId and nothing else rewrites it. A fresh --state path makes the comments stage revisit periods an earlier run already checkpointed.

Rate limits during a backfill or reconcile sweep

eagle-api allows 200 requests a minute per client (ratelimit-policy: 200;w=60). A comment sweep or the comments backfill stage can spend that inside a minute; the shared fetch helper (src/seed/sources.js) waits a 429 out rather than failing the run. Order it tries, per response:

1. retry-after, if the response names one.
2. ratelimit-reset (seconds left in the current window).
3. The window length off ratelimit-policy (200;w=60).
4. A 5 s, 15 s, 60 s ladder, if none of the above is present.

Every wait is floored at 1 s and capped at 120 s, so a ratelimit-reset: 0 cannot retry instantly and a bad header cannot park the run for an hour. It gives up after 5 waits on the same request. Each wait logs one line — a run gone quiet for a minute at a time is being throttled, not stuck.

eagle-api: scripts/demi-repush.js

Rows seeded into DEMI before eagle-api started pushing are missing the push-only enrichment a controller write adds on the way out — a project's applicableRegulation, resolved pins, featuredDocuments and per-legislation proponent labels, added in api/helpers/demiPush.js, never stored in Mongo. Nothing re-pushes a record that has not been edited since, so this script walks a Mongo collection and pushes every record through the same helper the controllers use.

node scripts/demi-repush.js --kind project                       # dry run, default
node scripts/demi-repush.js --kind project --live
node scripts/demi-repush.js --kind document --live --state ./repush.state.json
node scripts/demi-repush.js --kind commentPeriod --live --concurrency 1

--kind takes project, document, commentPeriod, comment, organization or projectNotification (default project). --since <ISO> filters on the record's own update timestamp where one exists; projectNotification has none and refuses --since. --concurrency sets pushes in flight, default 4. --state <path> checkpoints the last settled _id; a rerun with the same path resumes after it. The write is a PUT on the record id, so a repeated push is harmless.

Exit codes: 0 everything pushed, 1 the run itself failed, 2 bad arguments or DEMI_API_BASE/ DEMI_APIM_KEY not set, 3 the run finished with records DEMI did not accept (see the logged failed ids and retry with --state, which resumes past the last clean batch).

Concurrency 4 hit APIM's rate limit on test. A full --kind project re-push at the default concurrency 4 was throttled partway through; rerunning with --concurrency 1 from the same --state file finished the remaining records (376/376, measured 2026-09-08). Start a fresh environment's first repush at --concurrency 1 rather than assuming the default is safe there.

See also

• Sync-Architecture — the push and the four mirrored kinds
• Search-Index-Reference — the Cosmos-served datasets these containers answer
• ADR-011-Public-Read-Surface — why reads and writes split across two systems
• eagle-demi/docs/public-read-backfill.md — the same detail, kept in the repo for the devbox session that runs it without a wiki checkout