fix(consolidation): indefinite retry with backoff + dedup-by-bank guard

[nicoloboschi]

Summary

Two coupled changes to the consolidation task-retry path in execute_task:

1. Indefinite retry with capped exponential backoff (60, 120, 240, 480, 960, then pinned at 1800s) — replaces the generic 60s × 3 cap that other task types still use. Transient outages must eventually recover; capping silently dead-letters a bank's unconsolidated rows.
2. Per-bank dedup guard — if another consolidation op is already pending for the same bank, skip the retry and let the peer cover the work. Without this, indefinite retry would let multiple ops for the same bank retry forever in lockstep during an outage.

Surfaces from the discussion on #1799: the symptom there (event-chain breaks leave memory_units unconsolidated) is largely a retry-budget exhaustion problem during long outages, not a missing-trigger problem. Fixing the retry layer addresses it without polling memory_units.

Why

Before this PR, execute_task retried transient consolidation failures with the generic schedule (60s × 3, then _mark_failed). During a sustained upstream outage:

1. Op A runs → LLM 503 → re-pended via RetryTaskAt.
2. Retain succeeds and enqueues op B for the same bank (submit_async_consolidation dedup only checks pending, not processing).
3. Both A and B burn 3 retry slots each; every additional retain spawns C, D, … each burning three more.
4. After the budget is exhausted, the ops are marked failed and the bank's unconsolidated rows sit untouched until the next retain triggers a fresh op — which then also burns its budget.

This PR breaks the cycle:

• The dedup guard collapses step 3 to a single retrying op per bank.
• The indefinite-with-cap schedule on that single op rides out outages of any length, so step 4 doesn't happen: the next attempt eventually succeeds when the dependency comes back.

Deterministic failures (integrity violations, embedding dimension errors) are still filtered upstream by _is_non_retryable_task_error and marked failed immediately. Other task types keep their existing 60s × 3 generic schedule.

What changed

hindsight_api/engine/memory_engine.py

• _consolidation_retry_backoff_seconds(retry_count) — capped exponential backoff helper, no attempt cap.
• _has_other_pending_consolidation(bank_id, operation_id) — single-row check on async_operations, fails open on DB errors.
• Generic exception branch of execute_task for consolidation tasks: fire failure webhook, run dedup check, then either bare-raise (poller marks failed) or RetryTaskAt with the new backoff.

tests/test_consolidation_retry_dedup_by_bank.py (new) — 7 regression tests:

• Dedup guard: peer pending → no RetryTaskAt; no peer → RetryTaskAt fires; peer in different bank → no suppression.
• Backoff schedule: unit test of the helper (60/120/240/480/960/cap), parameterised execute_task test verifying retry_at matches each step, indefinite-retry test at retry_count=100 confirming the cap holds and we never give up.

Test plan

• pytest tests/test_consolidation_retry_dedup_by_bank.py — 9 passed
• pytest tests/test_consolidation_retry_budget.py tests/test_consolidation_failure_recovery.py tests/test_integrity_violation_not_retried.py — no regressions
• ./scripts/hooks/lint.sh — clean
• CI green

What this PR does not change

• No new env vars, no opt-in flag — these are correctness fixes for the current retry semantics.
• No changes to submit_async_consolidation dedup semantics — that path still only dedups pending.
• No memory_units polling — the discussion on feat(worker): add opt-in periodic scanner for pending consolidations #1799 ruled out the scanner approach.
• Other task types (batch_retain, refresh_mental_model, webhook_delivery) keep their existing 60s × 3 retry schedule.