Skip to content

fix(capture): recover replaced browser snapshots#2930

Merged
Sinity merged 3 commits into
masterfrom
feature/fix/browser-capture-replacement-reingest
Jul 16, 2026
Merged

fix(capture): recover replaced browser snapshots#2930
Sinity merged 3 commits into
masterfrom
feature/fix/browser-capture-replacement-reingest

Conversation

@Sinity

@Sinity Sinity commented Jul 16, 2026

Copy link
Copy Markdown
Owner

Summary

Recover browser captures after transient parse quarantine and accept later provider-authoritative browser snapshots without weakening divergent-branch protection.

Problem

The 27-session Sol Pro production census found current extension files parsing to 2,551 messages while the index exposed 205. Twenty-two cursor paths were excluded after five transient failures and never reconsidered after atomic replacement. Separately, mutable provider-native snapshots were quarantined because stable context/tool messages can reorder and output attachments can become available without serialized strict-prefix growth.

Solution

  • Bind cursor exclusion to the exact failed stat/inode observation and revive only a changed replacement.
  • Carry browser capture fidelity, stable provider message/attachment identities, provider update time, and durable acquisition order into membership classification.
  • Accept compatible provider progress, DOM-to-native upgrades, and later attachment-only enrichment.
  • Preserve ambiguity for identity loss, same-time transcript divergence, native-to-DOM downgrade, and ordinary non-browser revision branches.
  • Add production-route watcher, membership, reverse/divergence, and browser replacement regressions.

Ref polylogue-3v1.

Verification

  • devtools test tests/unit/archive/test_session_revision_membership.py tests/unit/sources/test_live_batch_support.py::test_browser_capture_replacement_advances_membership_head_and_acquires_attachment tests/unit/sources/test_live_batch_support.py::test_browser_capture_provider_timestamp_advances_reordered_native_snapshot tests/unit/sources/test_live_watcher.py::test_cursor_mark_failed_quarantines_repeated_failures tests/unit/sources/test_live_watcher.py::test_replaced_excluded_file_is_revived_without_retrying_unchanged_poison
    • 12 passed
  • devtools verify --quick
    • exit 0, run 20260716T090428Z-quick-53371-e8ba2806
  • pre-push devtools verify --quick
    • exit 0, run 20260716T090710Z-quick-56296-18a04001
  • Default affected-test verification could not run in the fresh worktree because its testmon database was not seeded; the exact impacted production routes were run explicitly.

Summary by CodeRabbit

  • New Features

    • Improved browser snapshot handling, including native and DOM fidelity, message identity, and attachment preservation.
    • Browser captures are now ordered more reliably when provider timestamps and content changes indicate a newer revision.
    • Replaced files previously excluded after ingestion failures are automatically retried, while unchanged files remain skipped.
    • Added durable acquisition-time lookup for retained revisions.
  • Bug Fixes

    • Prevented valid reordered or attachment-enriched browser snapshots from being incorrectly rejected.
    • Preserved safeguards against revisions that lose message identity.

Problem: browser capture paths remained excluded forever after five transient failures, even when the extension atomically replaced them with complete files. Provider-native snapshot progress was also quarantined when stable messages reordered or attachments were acquired later.\n\nClear exclusion only when the observed file identity changes. Classify mutable browser snapshots with fidelity, provider freshness, stable message and attachment identities, and acquisition order for attachment-only enrichment while retaining ambiguity for lost identities and divergent branches.\n\nRef polylogue-3v1.\n\nCo-Authored-By: Claude <noreply@anthropic.com>
@coderabbitai

coderabbitai Bot commented Jul 16, 2026

Copy link
Copy Markdown

Review Change Stack

Warning

Review limit reached

@Sinity, you've reached your PR review limit, so we couldn't start this review.

Next review available in: 40 minutes

Enable usage-based reviews in Billing to review now. Otherwise, wait until the next included review is available.
You're only billed for reviews past your plan's rate limits ($0.25/file).

How can I continue?

After more reviews become available, a review can be triggered using the @coderabbitai review command as a PR comment. Alternatively, push new commits to this PR.

To avoid repeated limits, reduce automatic review volume by pausing incremental auto-reviews earlier, using label-based review opt-in, excluding WIP or generated PR titles, or requesting reviews manually when the PR is ready. If your team needs uninterrupted high-volume reviews, an organization admin can enable usage-based reviews.

How do review limits work?

CodeRabbit enforces per-developer PR review limits for each organization. Most developers receive the normal plan review availability.

For paid Pro and Pro+ PR reviews, CodeRabbit uses adaptive limits for sustained high-volume activity. When a developer's recent PR review activity reaches the 95th percentile or higher among CodeRabbit users, additional reviews become available more gradually as earlier reviews age out of the rolling window.

Please refer docs for additional details.

Review details
⚙️ Run configuration

Configuration used: Path: .coderabbit.yaml

Review profile: ASSERTIVE

Plan: Pro

Run ID: e0b23274-689c-41a1-b1ca-771ede7bb432

📥 Commits

Reviewing files that changed from the base of the PR and between f5f0b7e and 6899b60.

📒 Files selected for processing (5)
  • polylogue/archive/session_revision_membership.py
  • polylogue/sources/live/batch.py
  • polylogue/sources/live/cursor.py
  • tests/unit/archive/test_session_revision_membership.py
  • tests/unit/sources/test_live_watcher.py
📝 Walkthrough

Walkthrough

The change adds browser snapshot fidelity and provider identity metadata to membership revisions, orders compatible snapshots by provider authority, enriches live ingestion records, exposes raw acquisition timestamps, and revives excluded cursors when files are replaced.

Changes

Browser snapshot membership

Layer / File(s) Summary
Membership classification rules
polylogue/archive/session_revision_membership.py, tests/unit/archive/test_session_revision_membership.py
MembershipRevision stores snapshot fidelity and provider identities; classification orders compatible revisions and tests message reordering, identity loss, and attachment enrichment.
Live ingestion metadata wiring
polylogue/sources/live/batch.py, polylogue/storage/sqlite/archive_tiers/archive.py, tests/unit/sources/test_live_batch_support.py
Live membership application records browser metadata and provider identities, raw acquisition timestamps are retrievable, and reordered native captures are tested end to end.

Watcher exclusion recovery

Layer / File(s) Summary
Replaced excluded file handling
polylogue/sources/live/cursor.py, polylogue/sources/live/watcher.py, tests/unit/sources/test_live_watcher.py
Excluded cursors remain skipped for unchanged observations and are revived with reset retry state when the file observation changes.

Estimated code review effort: 4 (Complex) | ~45 minutes

Sequence Diagram(s)

sequenceDiagram
  participant LiveBatchProcessor
  participant classify_membership_revisions
  participant ArchiveStore
  LiveBatchProcessor->>classify_membership_revisions: submit enriched membership revisions
  classify_membership_revisions->>classify_membership_revisions: order compatible browser snapshots
  classify_membership_revisions-->>LiveBatchProcessor: return strict chain and equivalents
  LiveBatchProcessor->>ArchiveStore: apply raw session membership decisions
  ArchiveStore-->>LiveBatchProcessor: return stored membership state
Loading

Possibly related PRs

  • Sinity/polylogue#2716: Both changes modify membership revision ordering and provider timestamp authority handling.
🚥 Pre-merge checks | ✅ 4 | ❌ 1

❌ Failed checks (1 warning)

Check name Status Explanation Resolution
Docstring Coverage ⚠️ Warning Docstring coverage is 18.18% which is insufficient. The required threshold is 80.00%. Write docstrings for the functions missing them to satisfy the coverage threshold.
✅ Passed checks (4 passed)
Check name Status Explanation
Title check ✅ Passed The title is concise and accurately reflects the main change: recovering replaced browser snapshots.
Description check ✅ Passed The PR description covers Summary, Problem, Solution, and Verification; only optional Changelog/Risks sections are omitted.
Linked Issues check ✅ Passed Check skipped because no linked issues were found for this pull request.
Out of Scope Changes check ✅ Passed Check skipped because no linked issues were found for this pull request.
✨ Finishing Touches
🧪 Generate unit tests (beta)
  • Create PR with unit tests
  • Commit unit tests in branch feature/fix/browser-capture-replacement-reingest

Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out.

❤️ Share

Comment @coderabbitai help to get the list of available commands.

@chatgpt-codex-connector chatgpt-codex-connector Bot left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

💡 Codex Review

Here are some automated review suggestions for this pull request.

Reviewed commit: f5f0b7e11c

ℹ️ About Codex in GitHub

Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you

  • Open a pull request for review
  • Mark a draft as ready
  • Comment "@codex review".

If Codex has suggestions, it will comment; otherwise it will react with 👍.

Codex can also answer questions or update the PR. Try commenting "@codex address that feedback".

Comment on lines +636 to +641
if (
cursor.byte_size,
cursor.st_dev,
cursor.st_ino,
cursor.mtime_ns,
) == (size, stat.st_dev, stat.st_ino, stat.st_mtime_ns):

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

P2 Badge Bind quarantine to the failed file observation

When a previously successful source is replaced by malformed content, _record_failed_cursor deliberately preserves the last accepted cursor observation while incrementing failures. After the fifth failure sets excluded, this comparison therefore sees the still-malformed file as a replacement on the next watcher scan, clears the exclusion and resets failure_count to zero. The same poison file then enters another retry cycle instead of remaining quarantined; store the failed stat observation separately or update it when exclusion is reached.

Useful? React with 👍 / 👎.

Copy link
Copy Markdown
Owner Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Fixed in 3340459. Quarantine now records the exact failed stat observation only when the retry budget is exhausted; ordinary backoff still preserves the last accepted cursor. Added a regression covering an accepted file replaced by unchanged malformed content.

@coderabbitai coderabbitai Bot left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Actionable comments posted: 1

🤖 Prompt for all review comments with AI agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.

Inline comments:
In `@polylogue/archive/session_revision_membership.py`:
- Around line 117-144: Update the DOM-to-native branch in
_browser_snapshot_dominates to compare the newer and older frontiers
elementwise, requiring every frontier dimension to be greater than or equal. Do
not use tuple lexicographic comparison, so event or attachment regressions
cannot pass.
🪄 Autofix (Beta)

Fix all unresolved CodeRabbit comments on this PR:

  • Push a commit to this branch (recommended)
  • Create a new PR with the fixes

ℹ️ Review info
⚙️ Run configuration

Configuration used: Path: .coderabbit.yaml

Review profile: ASSERTIVE

Plan: Pro

Run ID: c1742cb6-cbc0-4e9a-b3b1-d58a5b305b1c

📥 Commits

Reviewing files that changed from the base of the PR and between dde52a8 and f5f0b7e.

📒 Files selected for processing (8)
  • polylogue/archive/session_revision_membership.py
  • polylogue/sources/live/batch.py
  • polylogue/sources/live/cursor.py
  • polylogue/sources/live/watcher.py
  • polylogue/storage/sqlite/archive_tiers/archive.py
  • tests/unit/archive/test_session_revision_membership.py
  • tests/unit/sources/test_live_batch_support.py
  • tests/unit/sources/test_live_watcher.py

Comment thread polylogue/archive/session_revision_membership.py
Sinity and others added 2 commits July 16, 2026 11:23
A replaced malformed source inherited the prior successful cursor stat, so
exclusion appeared stale immediately and the watcher revived unchanged poison.
Record the failed file observation when the retry budget is exhausted while
preserving accepted cursor authority during ordinary backoff.

Co-Authored-By: Claude <noreply@anthropic.com>
A lexicographic DOM-to-native frontier comparison could accept a snapshot that
gained messages while losing events or attachments. Require every frontier
dimension to remain monotonic and cover the shrinking-attachment case.

Co-Authored-By: Claude <noreply@anthropic.com>
@Sinity
Sinity merged commit d2573d4 into master Jul 16, 2026
1 check passed
@Sinity
Sinity deleted the feature/fix/browser-capture-replacement-reingest branch July 16, 2026 09:30
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant