fix(agents): the attempts counter has no writer, and two drivers have no terminal state

[lilyshen0722]

attempts is a frozen 0 on every payload the kernel has ever served, and the guard that reads it has never fired. Fixing that alone would have made things worse, so this is the whole lifecycle in one PR — at @pod-architect's suggestion, since the driver fixes live in the same thirty lines.

The counter

ADR-004 §Event model: "Unacked events stay in the queue and re-deliver on next poll, with attempts incremented."

Nothing increments it in the pending↔delivered cycle. attempts moves only at acknowledge() (:1030) and recordFailure() (:1134) — both terminal. So the field CAP obliges every driver to dedup with is constant, and a driver has no way to tell a redelivery from a first delivery.

That also froze the requeue's own guard:

garbageCollect()  attempts < 3        ← reads as a poison-event bound
write sites       acked | failed      ← both terminal, both outside the cycle

The guard's condition and the variable's write sites are in complementary states, which is exactly why it looks fine in review. It has never once fired.

list() now increments on the pending → delivered claim. The claim is the delivery, so it's the only site that can honour :73, and {new: true} means the value the driver receives is the post-increment one. attempts counts deliveries: first claim 1, requeued redelivery 2. acknowledge() and recordFailure() stop incrementing — with the claim counting, double-writing made a normally-handled event read attempts: 2 and left the field meaning "deliveries plus terminal transitions," which is not a number anyone can dedup on.

Why the counter needed a cap in the same PR

Turning it on without a cap-exhaustion pass is worse than leaving it off. A capped event fails the requeue predicate, and a 'delivered' row is invisible to list() — so it sits stuck until the 168h retention delete. That is precisely the Task #67 symptom the requeue exists to fix, recreated by its own fix.

Second pass retires attempts >= cap to terminal failed. The two are disjoint on attempts alone (< cap vs >= cap), so no document can be touched by both in one run.

Two driver classes had no terminal state at all

driver	before	consequence
native	created 'delivered'; nativeRuntimeService has zero ack/recordFailure calls	structurally unackable — every native event entered the pending queue ~10-20 min later and could be handed to an external poller that cannot run it
webhook	'delivered' after a successful POST; requeue has no delivery exclusion	the endpoint was called again ~10-20 min later, indefinitely — duplicate delivery across the whole ADR-006 driver class

Both now settle. Native acks on completion and records failure on error, via a two-argument .then rather than .then().catch() — a chained catch would also catch a rejection from the success handler and mark a run that actually succeeded as failed. There's a test for that inversion specifically.

The phantom predicate

ackedAt: { $in: [null, undefined] } — there is no ackedAt field. Not in IAgentEvent, not in the schema, never written; Mongoose strips it, so the clause matched every document while reading as a live narrowing. Dropped rather than "fixed": status: 'delivered' already excludes acked events. Caught by @pod-architect.

Verification

21 pass (12 new + 9 existing)   ·   tsc --noEmit  0 errors

M1  drop $inc at the claim                    → 1 fail
M2  expire pass $gte → $gt (off-by-one)       → 2 fail
M3  webhook back to 'delivered'               → 1 fail
M4  native drops attempts: 1                  → 1 fail

Assertions are on query shape, not on observed counts — this is a mocked-model suite, and a test that only checked "an update happened" would pass against all four bugs.

Not fixed here, documented at the site

Effective redelivery latency is 10-20 min, not 10. schedulerService runs this job on */10 and the threshold is also 10 min, so period P and threshold T give [T, T+P) — uniform, mean ~15. ADR-004 says "next poll" and tells drivers 3-10s. Closing that gap means a lease or a short-TTL claim, which is a design question and wants its own PR against ADR-004.

Not verified: no DB or cluster read — I can show these paths are reachable and cannot show how many rows are in each state today. The webhook duplicate-delivery claim in particular is derived from the predicate, not observed against a live endpoint. Suites run: this one plus the existing agentEventService suite; not the full backend run.

🤖 Generated with Claude Code