feat: native Twitter Bookmark sync via connector framework#39
Merged
doodlewind merged 4 commits intomainfrom Apr 7, 2026
Merged
feat: native Twitter Bookmark sync via connector framework#39doodlewind merged 4 commits intomainfrom
doodlewind merged 4 commits intomainfrom
Conversation
Replace OpenCLI-based Twitter bookmark fetching with a native connector that uses X's internal GraphQL API authenticated via Chrome cookie extraction (macOS). This is the first connector built on the new universal sync architecture. Connector framework: - Connector interface with fetchPage() contract - SyncEngine with dual-frontier model (forward + backfill) - SyncScheduler with priority queue, lifecycle events, exponential backoff - 16 enumerated SyncErrorCode values with user-facing hints - connector_sync_state DB table for crash-safe sync state Twitter Bookmarks connector: - Chrome cookie extraction (AES-128-CBC decryption via macOS Keychain) - GraphQL API pagination with retry/rate-limit handling - Maps tweets to CapturedItem for unified search UI integration: - Connectors section in SourcesPanel with live sync progress - Progress bar + "page N · M new" real-time status - Error display with actionable hints (needs login, etc.) - HomeView chips show connector capture counts from DB - IPC: connector:list, connector:sync-now, connector:get-capture-count Removes Twitter strategies from OpenCLI (replaced by native connector). Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
- backfillMaxPages: 10 → 50 (1000 items per cycle instead of 200) - maxMinutesPerRun: 5 → 10 (allow longer sync runs) - Show "syncing history" indicator when tail is not complete - Users can see that historical data is still being fetched Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
Sync engine changes: - Remove hardcoded page limits (forwardMaxPages, backfillMaxPages) - Loops run until completion, timeout, or cancellation - maxMinutes=0 means unlimited (used by CLI) - Scheduler applies maxMinutesPerRun=10 as the only time bound - Default page delay 600ms → 1200ms to avoid rate limits - Extract common fetchLoop() to deduplicate forward/backfill logic - Errors save progress before returning (no data loss on API failure) - Stale page detection applies to BOTH forward and backfill phases - Checkpoint state every 25 pages for crash safety CLI: - New `spool connector-sync [id]` command for full sync - --reset flag: delete all data and sync from scratch - --delay flag: configurable page delay - Ctrl+C graceful shutdown (saves progress) - Reports real DB count on completion UI: - Replace fake progress bar with indeterminate pulse indicator - Show "syncing history" when tail is incomplete Verified: 1751 bookmarks fully synced in 171 seconds (94 pages). Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
a93f32e to
18c3e1b
Compare
- Merge theme/settings/sources into unified settings panel with sidebar nav (General/Connectors/Agent) - Connectors default to disabled, require user opt-in via "Connect" click - Drill-down detail view for each connector with enable toggle, status, sync - Remove OpenCLI onboarding flow from UI (code preserved for future refactor) - StatusBar simplified to single gear icon entry point - Normalize font sizes across all tabs to match design system (12px/11px) - Add connector:set-enabled IPC handler and preload API - Pulsing colored dot replaces orange progress bar during sync Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
Connectorinterface →SyncEngine(dual-frontier) →SyncScheduler(priority queue + lifecycle events + backoff)SyncErrorCodevalues with user-facing hints for actionable error displayNew files
Test plan
Known limitations
totalSyncedin sync state may lag behind actual DB count after crashes (UI reads from DB directly as workaround)🤖 Generated with Claude Code