Eagle DEMI Push Sweep

Eagle DEMI Push Sweep

The demi-push-sweep worker (worker/demi-push-sweep.js) retries eagle-api pushes to DEMI that failed. It handles two kinds of leftover: flagged rows and tombstones. For the push itself and the write response, see Eagle DEMI.

Schedule

Helm CronJob eagle-cron-demi-sweep runs the sweep every 5 minutes on test and prod. The API runs it once at startup. The CronJob does not overlap itself (concurrencyPolicy: Forbid).

Flagged rows

The sweep re-pushes rows with demiPushPending: true. Success clears demiPushPending, demiPushFailedAt and demiPushError. Failure updates demiPushFailedAt and demiPushError.

Kind order, parents first: project, projectNotification, organization, group, document, commentPeriod, recentActivity (updates), inspection, inspectionElement, inspectionItem, user, comment.

It pushes one row at a time, 400 ms between starts (about 150 a minute), at most 100 rows per kind at startup and 500 per cron run.

A whole-row push is safe after a DEMI takedown (levelHeldAt). Unflagged rows: Eagle DEMI Re-Push and Backfill.

Tombstones

A hard delete leaves no row to flag. When its push fails, eagle-api writes a row in the demi_push_tombstones collection. The row is unique on kind + targetId.

Field	Meaning
kind	Record kind: document, comment period or project group
targetId	Id of the deleted record
failedAt	Time of the last failed push
error	Reason for the last failure
body	The body the controller pushed
createdAt	Time the tombstone was written

After the flagged-row pass, the sweep drains the tombstones, parents first, oldest first, up to the per-kind limit above. It re-sends the kept body with isDeleted: true through the same push function.

• A landed resend deletes the tombstone.
• A failed resend updates failedAt and error.
• A landed delete push from the controller also removes any tombstone for that id.

A soft delete (project) flags the live row and writes no tombstone.