feat(sync): sync_queue primitives (day 1 of #194)#195
Draft
Gradata wants to merge 1 commit into
Draft
Conversation
|
Important Review skippedDraft detected. Please check the settings in the CodeRabbit UI or the ⚙️ Run configurationConfiguration used: Organization UI Review profile: ASSERTIVE Plan: Pro Run ID: You can disable this status message by setting the Use the checkbox below for a quick retry:
✨ Finishing Touches🧪 Generate unit tests (beta)
Comment |
Gradata
added a commit
that referenced
this pull request
May 17, 2026
…194) (#200) * feat(sync): write-through Brain.correct() -> /api/v1/ingest (day 3 of #194) Every Brain.correct(draft, final) now enqueues the correction into a local sync_queue table and a background daemon thread drains the queue by POSTing each row to api.gradata.ai/api/v1/ingest. Removes the dependency on session-end hooks and the cron sync band-aid. See /home/olive/gradata-office-hours-memo.md for the architectural rationale. ## Components - src/gradata/_sync_queue.py: enqueue/peek/mark_synced/mark_failed CRUD + enqueue_correction helper - src/gradata/_sync_worker.py: daemon-thread SyncWorker with start/stop(drain)/_tick, handles 2xx + dedup + 422 poison + 429 backoff + 5xx + network - src/gradata/brain.py: Brain.__init__ starts worker when API key resolvable AND GRADATA_DISABLE_WRITE_THROUGH != 1; Brain.correct enqueues post-local-write; Brain.close drains and stops - src/gradata/_migrations/__init__.py + onboard.py: sync_queue table + idx_sync_queue_pending (idempotent, no-op if #195 lands first) ## Env vars - GRADATA_API_KEY (existing): write-through requires it - GRADATA_DISABLE_WRITE_THROUGH=1: opt-out, fall back to hook+cron path - GRADATA_CLOUD_INGEST_URL (default https://api.gradata.ai/api/v1/ingest) - GRADATA_SYNC_TICK_SEC (default 30) ## Failure modes - Cloud unreachable: row stays pending, retried next tick - 422 permanent: poison row marked synced + failed so it doesn't block batch - 429: bail batch, retry next tick - No API key: worker doesn't start, no enqueue, local correct() unaffected - _write_through_enqueue exception: caught, local correct() always succeeds ## Tests - tests/test_sync_queue.py (9): existing CRUD primitives + enqueue_correction - tests/test_sync_worker.py (9): HTTPServer stub /ingest with all status branches + at-least-once + stop-drain - tests/test_brain_write_through.py (5): enqueue happy path, disabled, no-key, cloud-unreachable, sabotaged enqueue 38 passed locally. Series: #195 (day 1 SDK queue), Gradata/gradata-cloud#58 merged (day 2 cloud receiver), this PR (day 3 wire-up). * fix(tests): update test_sync_with_no_events_returns_zero for PR #198 heartbeat PR #198 changed empty /sync behavior to POST an empty heartbeat to cloud so brains.last_sync_at advances. The pre-#198 test still asserted mock_post.assert_not_called() — now expects exactly one call. --------- Co-authored-by: data-engineer <data-engineer@gradata.ai>
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Day 1 of #194. Adds local SQLite queue table and CRUD primitives. No wiring into Brain.correct() yet — that's day 3.
Deliverables
Verify
uv run pytest tests/ -k sync_queue -x
Next
Day 2: cloud /api/v1/ingest endpoint.