fix: sweep orphaned attachment rows when their last ref is dropped

[Sinity]

Summary

Fixes an index.db-tier write-path gap where _write_attachments could leave a ref-less attachments row behind forever, unreachable from every session/message read path, and adds a reachability split to the existing attachment-acquisition-debt report so that gap can't hide behind acquisition_status='acquired' again.

Problem

Read-only forensics on the live archive (polylogue-w06b, 2026-08-01) found 1,858 attachments rows in index.db with zero attachment_refs rows:

sqlite3 'file:/realm/db/polylogue/index.db?mode=ro' "
  select acquisition_status, count(*), sum(byte_count) from attachments a
  where not exists (select 1 from attachment_refs r where r.attachment_id = a.attachment_id)
  group by 1;"
-- acquired|1783|439974530
-- unfetched|75|756334471

1,783 of those were acquisition_status='acquired' (440MB of real, successfully-fetched blob content) but permanently unreachable from every session-scoped read path — get_attachments/get_attachments_batch (storage/sqlite/queries/attachment_records.py) both INNER JOIN attachment_refs, so a ref-less row can never be returned to any surface (MCP, CLI read --view, transcript view). Of the archive's 1,933 attachments marked acquired, only 150 (7.8%) were actually queryable.

Root cause classification: (a) — a straightforward missing cleanup step at write time, not a case needing reconstruction/backfill logic. _write_attachments (polylogue/storage/sqlite/archive_tiers/write.py) already recomputes ref_count for attachments whose ref was dropped on a full-replace re-ingest (the owning message disappeared this pass — dropped, renumbered, or excluded as a duplicate native id), via:

UPDATE attachments SET ref_count = (SELECT COUNT(*) FROM attachment_refs WHERE ...) WHERE attachment_id IN (...)

But unlike the identical cleanup already performed by prune_attachments (queries/attachment_mutations.py) and delete_session_sql (queries/sessions_writes.py) — both of which follow their own ref-count refresh with DELETE FROM attachments WHERE ref_count <= 0 — _write_attachments never had that DELETE. The row survives indefinitely, still reporting acquisition_status='acquired'.

Solution

• _write_attachments now runs the same DELETE FROM attachments WHERE ref_count <= 0 AND attachment_id IN (...) sweep the other two writers already perform, scoped to the same affected_attachment_ids set the ref_count UPDATE just touched.
• AttachmentAcquisitionDebtReport (polylogue/storage/blob_integrity.py, backing the attachment-acquisition-debt CLI command) gains acquired_reachable_count / acquired_unreachable_count (+ a sample of unreachable ids), computed via a NOT EXISTS (SELECT 1 FROM attachment_refs ...) check against attachments.acquisition_status='acquired'. This reconciles the "attachment coverage" framing used by polylogue-7r6u/pfdf (AC3 of polylogue-w06b): acquired_count alone overstated real coverage, since it never distinguished "bytes fetched" from "bytes anyone can actually read back".
• CLI plain-text output (ops maintenance attachment-acquisition-debt) now breaks out reachable/unreachable under the Acquired: line and lists a sample of unreachable ids when present.

Non-goals

• Backfilling the 1,783 existing orphaned rows in the live archive. Their owning message was already dropped by the ingest pass that orphaned them; reconstructing the true link (if even possible) requires re-parsing the original source exports, not a mechanical fix from current index.db state alone. This PR does not write to /realm/db/polylogue in any way. A follow-up bead should track the backfill/GC decision for the existing rows (delete them outright vs. attempt re-parse-based relinking) — flagging this for the coordinator to file, since this lane's worktree must not write .beads/.
• Did not change attachment_refs' schema (e.g. adding an ON DELETE CASCADE from attachments back to some caller) — the gap was in application-level write discipline, not the schema.

Verification

• devtools test tests/unit/storage/test_attachment_acquisition.py tests/unit/storage/test_blob_integrity.py tests/unit/storage/test_archive_tiers_write.py tests/unit/storage/test_attachment_first_class_ids.py → 111 passed.
• Anti-vacuity: reverted the new DELETE FROM attachments WHERE ref_count <= 0 ... statement and reran test_orphaned_attachment_ref_is_swept_not_left_unreachable — it fails (assert <sqlite3.Row object at ...> is None), reproducing the exact live-archive bug shape before restoring the fix. Production caller exercised: get_attachments (storage/sqlite/queries/attachment_records.py), the read path every session/message attachment surface goes through.
• devtools test tests/unit/cli/test_archive_maintenance_cli.py -k attachment_acquisition → 2 passed.
• devtools verify --quick → exit 0 (format, lint, mypy --strict, render all --check — regenerated docs stayed in sync, no drift).
• Two unrelated pre-existing failures were observed in tests/unit/cli/test_archive_maintenance_cli.py (test_rebuild_index_full_source_resumes_one_candidate_until_terminal_promotion, test_rebuild_index_byte_budget_defers_then_reaches_terminal_ready_candidate) and in tests/unit/storage/test_repair.py (11 raw-materialization tests) — confirmed to fail identically with this branch's changes reverted, so classified as pre-existing/unrelated, not caused by this PR.

AC matrix (polylogue-w06b)

• Determine why rows exist without a ref — done: full-replace ref-count-refresh-without-delete gap in _write_attachments, evidenced above.
• Decide fix for going-forward writes — done: GC (delete) at write time, matching the two sibling writers' existing pattern. Backfilling the existing 1,783 orphans is explicitly deferred (non-goal above, follow-up bead needed).
• Reconcile the 'attachment coverage' framing (polylogue-7r6u/pfdf) — done via acquired_reachable_count/acquired_unreachable_count on AttachmentAcquisitionDebtReport + CLI output.

Ref polylogue-w06b

[coderabbitai]

Warning

Review limit reached

@Sinity, you've reached your PR review limit, so we couldn't start this review.

Next review available in: 12 minutes

Enable usage-based reviews in Billing to review now. Otherwise, wait until the next included review is available.
You're only billed for reviews past your plan's rate limits ($0.25/file).

How can I continue?

After more reviews become available, a review can be triggered using the @coderabbitai review command as a PR comment. Alternatively, push new commits to this PR.

To avoid repeated limits, reduce automatic review volume by pausing incremental auto-reviews earlier, using label-based review opt-in, excluding WIP or generated PR titles, or requesting reviews manually when the PR is ready. If your team needs uninterrupted high-volume reviews, an organization admin can enable usage-based reviews.

How do review limits work?

CodeRabbit enforces per-developer PR review limits for each organization. Most developers receive the normal plan review availability.

For paid Pro and Pro+ PR reviews, CodeRabbit uses adaptive limits for sustained high-volume activity. When a developer's recent PR review activity reaches the 95th percentile or higher among CodeRabbit users, additional reviews become available more gradually as earlier reviews age out of the rolling window.

Please refer docs for additional details.

Review details

⚙️ Run configuration

Configuration used: Path: .coderabbit.yaml

Review profile: ASSERTIVE

Plan: Pro Plus

Run ID: 18c35e81-a454-4b1a-a4b9-9919174cde97

📥 Commits

Reviewing files that changed from the base of the PR and between 078d3c0 and fa9dfe6.

📒 Files selected for processing (7)

• polylogue/cli/commands/maintenance/_blob_integrity.py
• polylogue/storage/blob_integrity.py
• polylogue/storage/sqlite/archive_tiers/write.py
• tests/unit/cli/test_archive_maintenance_cli.py
• tests/unit/storage/test_archive_tiers_write.py
• tests/unit/storage/test_attachment_acquisition.py
• tests/unit/storage/test_blob_integrity.py

---

Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out.

❤️ Share

• X
• Mastodon
• Reddit
• LinkedIn

_{Comment @coderabbitai help to get the list of available commands.}