Retention

Describes robot-council/core v0.8.3.

The change feed is the one table that grows without anybody's help. Every task transition, lock, session change and line of narration is a row, and almost none of it is read twice: an agent pages the feed forward and a developer reads the head of it. So robot-council:prune-events deletes events past robot-council.retention.events_days, which defaults to 30 and is set with ROBOT_COUNCIL_EVENT_RETENTION_DAYS.

ROBOT_COUNCIL_EVENT_RETENTION_DAYS=30   # 0 keeps everything

Zero keeps everything, for a host archiving on its own terms — the command says so and exits rather than reporting that it deleted nothing, because "pruned 0 events" and "pruning is switched off" are different states and only one of them wants looking at.

Run robot-council:doctor after installing, and again after changing anything. It reports what is wrong with this application's configuration without being asked a specific question: whether the sanctum guard names a provider, whether sanctum.expiration is null, whether every migration this version ships has run, whether anything appears to be consuming the queue, whether anybody is on the developer allowlist, whether anything on the fleet can post a directive, whether the Slack mirror would run inside an agent's request, and whether app.timezone can shift under a token's expiry.

Every fault it looks for is invisible until something else goes wrong. It writes nothing and prints no secret, so it is safe to run when worried. A check it cannot reach reports as UNKNOWN with what would make it reachable, which reads differently from one that looked and found nothing.

What a host should watch:

• The row count, not the command's output. A prune that is keeping up reports roughly a day's events each run. A number that climbs run after run means the retention is longer than the disk.
• Agents that are offline longer than the retention. The feed is how an agent without a push connection catches up, and it pages id > cursor. A prune cannot strand one — a cursor is a number, not a row — but an agent that was away for longer than the retention will have missed events rather than read them late. If that matters for a fleet, the retention is the wrong length for it.
• The prune takes no feed lock and deletes in batches, so it does not block writers. It is safe to run by hand at any time, and a host that wants it more often can schedule it itself with the entry turned off in robot-council.schedule.prune_events.

Finished tasks have their own retention, robot-council.retention.tasks_days, defaulting to 90 and set with ROBOT_COUNCIL_TASK_RETENTION_DAYS. It is longer than the feed's because a task is a unit of work somebody may want to look back at, and there are far fewer of them.

A task nobody has finished is never deleted, whatever its age. It is work the fleet still owes somebody, and age is the opposite of a reason to remove it — an old pending task is the one most worth looking at. Only done, failed and cancelled are pruned, and the age is measured from when the task finished rather than when it was filed.

A finished task that still has another task filed under it is also left in place, because parent_task_id is nullOnDelete and deleting the parent would rewrite a row the prune never selected — possibly a task the fleet is still working on. It goes once its children have, which for a finished tree happens within the same run.

The dashboard's queue stops showing a finished task well before the prune removes it. On the unfiltered queue a done or cancelled task is hidden 24 hours after it finished, and a failed one after 7 days, since a failure usually still needs somebody. Choosing a status on the queue shows every task in it, however old, and the queue says how many it is hiding. These are display windows, not retention: nothing is deleted. Each is set in hours under robot-council.dashboard.hide_finished_after_hours, with ROBOT_COUNCIL_DASHBOARD_HIDE_DONE_AFTER_HOURS, ROBOT_COUNCIL_DASHBOARD_HIDE_CANCELLED_AFTER_HOURS and ROBOT_COUNCIL_DASHBOARD_HIDE_FAILED_AFTER_HOURS; 0 never hides.

Free locks have their own retention, robot-council.retention.locks_days, defaulting to 7 and set with ROBOT_COUNCIL_LOCK_RETENTION_DAYS. It is shorter than the other two because a lock row nobody holds carries a name, a previous holder and a number, none of which is read once the lease is over.

A lock somebody is holding is never deleted, whatever the row's age. "Free" here is the same condition an acquisition takes a lock from: no holder, or a lease that has lapsed.

A lock's fence is drawn from one sequence shared by every name, in robot_council_lock_fence, rather than counted per row. That is what makes deleting a lock row safe: every acquisition of any name draws a number above everything the sequence has ever issued, so a name whose row was deleted and then taken again still gets a fence above the one its last holder carried. Upgrading seeds the sequence above the highest fence already issued, so no running installation can reissue a number.

Ended agent sessions have their own retention, robot-council.retention.sessions_days, defaulting to 30 and set with ROBOT_COUNCIL_SESSION_RETENTION_DAYS. Only a session that has gone is ever deleted: an active session is live and a stale one is a single request from active again, so age is the wrong question for both.

A session still holding a task or a live lock is never deleted, whatever its age. robot_council_tasks.claimed_by and robot_council_locks.holder_id are both nullOnDelete, so deleting the row would strip a task of its claimant while its status still said it was held, and free a lock without the event a release writes. The session goes once whatever it held has been released or finished, which is why this prune is scheduled last of the four.

Deleting a session leaves robot_council_events.agent_session_id pointing at nothing, which is deliberate (#50) and harmless: nothing reads it to decide who may see an event. Both the feed and the login lookup read the developer off the event itself.