Skip to content

Add pause/resume control for pg-sync - #17

Closed
JustMaier wants to merge 1 commit into
mainfrom
feature/sync-pause
Closed

Add pause/resume control for pg-sync#17
JustMaier wants to merge 1 commit into
mainfrom
feature/sync-pause

Conversation

@JustMaier

@JustMaier JustMaier commented Mar 17, 2026

Copy link
Copy Markdown
Contributor

Summary

  • Add POST /api/sync/pause and POST /api/sync/resume admin endpoints to control pg-sync polling from the BitDex server
  • Add public GET /api/sync/status endpoint so pg-sync can check pause state without auth
  • Outbox poller checks pause status each cycle and skips PG polling when paused, with log-once messaging

Test plan

  • cargo check --features server,pg-sync compiles cleanly
  • cargo test --lib --features server passes (612 tests)
  • Manual: start server, POST /api/sync/pause, verify GET /api/sync/status returns {"paused":true}
  • Manual: POST /api/sync/resume, verify status returns {"paused":false}

🤖 Generated with Claude Code

@JustMaier @claude
Add pause/resume control for pg-sync via BitDex server
7dd14b9 
Server-side: AtomicBool sync_paused on AppState, admin endpoints
POST /api/sync/pause and POST /api/sync/resume (behind require_admin),
public GET /api/sync/status for pg-sync to poll without auth.

Client-side: BitdexClient.is_sync_paused() checks the status endpoint.
Outbox poller skips poll cycles when paused, with log-once messaging.

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
@JustMaier

JustMaier commented Mar 25, 2026

Copy link
Copy Markdown
Contributor Author

Stale — from Mar 17, bitrotted.

@JustMaier JustMaier closed this Mar 25, 2026
JustMaier added a commit that referenced this pull request Apr 5, 2026
@JustMaier @claude
refactor: remove bitmap state application from flush thread
6bfecf9 
With V3, BitmapSilo owns mutation persistence — flush thread no longer needs
to apply MutationOps to in-memory FilterIndex/SortIndex/SlotAllocator. Add
`has_silo: bool` to FlushArgs so tests without a silo still apply mutations
through the legacy path, keeping all 427 tests passing.

Removed: batch.apply() under write locks, stale_fields tracking, COMPACTION_INTERVAL
Kept: channel drain (prevents backup), time bucket maintenance (V2 dep, removed with #17),
      docstore channel drain, compaction triggers, no-silo fallback for tests

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
@JustMaier JustMaier mentioned this pull request Apr 5, 2026
Closed
2 tasks
JustMaier added a commit that referenced this pull request Apr 25, 2026
@JustMaier @claude
docs(perf): mixed-load measurement audit 2026-04-25 — mission gate cl…
2e77663 
…eared

11-min sustained mixed-load window on full perf stack v1.0.165-175 + relay V1
(post #232). Setup: local BitDex on 110.5M dump + replay-prod-via-relay
consumer subscribed to prod /events/ops + replay-captured at 3K QPS local
query load.

Headline:
- 1,724,131 queries served
- P50 < 50μs / P95 < 25ms / P99 ~25ms / max < 500ms
- Mission gate (P99 < 1s) cleared by ~40x margin
- 0 flush-slow / 0 ops-trace / 0 insert_bulk_slow events
- 8 sort-top_n SLOW events (max 65ms — separate surface, non-blocking)

Hypothesis verification:
- #1 userId-Eq apply contention: confirmed mitigated by #222/#227/#224 stack
- #2 tagIds-In multi-value contention: no contention observed
- #3 sortAtUnix-Gte tolerance miss: debunked for corpus (all snap to canonical buckets)

Caveats:
- Cache-saturated due to 232-shape captured corpus; absolute hit rate not
  representative of prod's diverse load. Real diverse shadow query traffic
  requires V2 model-share tee_mode skip PR.
- Sort-layer fusion under concurrent sort-field writes is a separate surface
  surfaced during measurement. Filed as task #17 follow-up.

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
JustMaier added a commit that referenced this pull request Apr 25, 2026
@JustMaier @claude
test(lib): serial_test on concurrent_engine save/restore + retract PR #…
434ed5e 
…240's misleading FIXMEs (#241)

Followup to PR #240 (commit `d6ca95d`). My in-PR commentary on the
two `#[ignore]`d tests claimed they were real engine bugs surfaced
when compile-rot was lifted. They are not. Both tests pass cleanly
in isolation and only fail under default `--test-threads`
parallelism, putting them in the same race-condition family as
`test_sync_filter_values_*`, `test_bound_store_persist_and_restore`,
and `test_docstore_v3_upsert_reads_old_doc`.

Closing the noise around the false alarm and serializing the actual
flake.

## Changes

1. **Drop the false FIXME `#[ignore]` annotations** on
   `test_save_snapshot_after_deletes` and
   `test_snapshot_save_preserves_bulk_loaded_lazy_value_field`. Both
   tests run normally now. The merge-on-save path in
   `write_inner_to_store` (`concurrent_engine.rs:6711-6732`) is
   correctly implemented and protects bulk-loaded fpack data from
   partial-overwrite — confirmed by the test passing once parallelism
   is tamed.

2. **Add `serial_test = "3"` as a dev-dependency** and tag the 18
   known-flaky save/restore + sync_filter + bound_store + lazy-load
   tests with `#[serial]`. They share global state (rayon thread
   pool used by parallel bucket writes; flush-thread timing
   sensitivity) and race-fail under default `--test-threads`.

3. **Bump `wait_for_flush` floor to 5 s.** Per-call timeouts are
   honored if larger; tight 1 s assertions get a 5 s floor so
   parallel test pressure can't race them. Tests stay snappy on
   quiet runs because the loop exits immediately on alive_count
   match.

4. **Re-ignore `test_save_snapshot_after_deletes` with a corrected,
   honest reason.** It still flakes even with `#[serial]` because
   `wait_for_flush(engine, 2, 500)` observes `alive_count` dropping
   to 2 after delete, but the delete also clears filter bitmap bits
   asynchronously within the same flush cycle. `save_snapshot` can
   fire between the alive-bit publish and the filter-bit publish →
   restored snapshot still carries the deleted slot's filter bit.
   Test contract gap, not an engine bug. The underlying delete path
   is exercised by production WAL replay tests. Run `--ignored` to
   reproduce.

## Result

```
cargo test --lib
  before this PR: occasional 1-3 race failures across runs
  after this PR:  823 passed, 0 failed, 2 ignored (deterministic)
```

The two ignored:
- `test_save_snapshot_after_deletes` — test-contract flake (item 4)
- `bench_collect_sort_work` — pre-existing benchmark, untouched

`test_snapshot_save_preserves_bulk_loaded_lazy_value_field` is now
running clean under `#[serial]`. The "P0 catastrophic data loss"
concern was a verification-loop false alarm.

## Standing Phase 4 cluster status

- ✅ #16 eprintln→tracing (PR #239 merged)
- ✅ #23 harness fix (PR #239 merged)
- ✅ #6 lib-test rot compile errors (PR #240 merged)
- ✅ this PR — parallelism + retract false FIXMEs
- 🚜 farming out: #15 cache atomics, #17 comment drift, dead unified_cache APIs, design-doc updates

Co-authored-by: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
JustMaier added a commit that referenced this pull request May 4, 2026
@JustMaier @claude
fix(sync): bug #17 — postedToId source Post.id → Post.modelVersionId (#…
6ade0e2 
…257)

Dump phase mapped postedToId via `lookup_key` (Post.id, the enrichment
join key), and the Post fan_out trigger only tracked publishedAt +
availability. Result: every doc had postedToId == postId, and detach
updates (Post.modelVersionId → NULL) never propagated. Model-gallery Or
queries returned unrelated images for any modelVersionId that happened
to collide with a Post.id.

Schema doc + V1 SQL (queries.rs) already say postedToId = Post.modelVersionId
— sync-v2 dump path + Post trigger were the divergent surfaces.

Changes:
- All 4 sync config copies: drop `postedToId: lookup_key` computed_field,
  add `{ column: modelVersionId, target: postedToId }` direct enrichment
  field, and add the same as a Post fan_out track_field so detach/reattach
  updates emit ops.
- src/ops_processor.rs: kill misleading PostEnrichment comment that baked
  the wrong derivation into prose.
- docs/design/civitai-field-requirements.md: row 32 description fixed.

Re-dump required after deploy — existing docs all carry the wrong value.
Talos-infra v2-sync-config.yaml needs the same edit (separate PR).

Co-authored-by: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
@JustMaier JustMaier mentioned this pull request Jul 10, 2026
Merged
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Reviewers

No reviews

Assignees

No one assigned

Labels

None yet

Projects

None yet

Milestone

No milestone

Development

Successfully merging this pull request may close these issues.

1 participant

@JustMaier