feat(cli): server-executed backfill client (#30, #26, PR-2b)#41
Merged
Conversation
…tatus/count/wait/cancel (#30, #26) Make the CLI a thin one-shot client over the always-on control API that PR-2a added. Backfill work is executed by the long-lived server, not the CLI process, so a `send` (which needs the store write lock) is never blocked by a CLI that is busy backfilling. core/control-client.js (new): loopback control client built on Node's global `fetch` (no hand-rolled http.request). readControlFile/pingServer/enqueueBackfill/ cancelBackfill hit the existing PR-2a endpoints (so enqueue/cancel logic is never reimplemented), all authed with the token from control.json. ensureServer reuses a reachable server (covering the race where another process started one) or spawns `tgcli server --idle-exit 60s` detached + unref'd and polls ping for ~10s; the idle-exit lets the auto-started server shut itself down once idle. cli.js: branch the `backfill` action on `--chat`. With `--chat` it ensureServer → enqueueBackfill; `--background` prints { jobId, channelId, status } and exits, otherwise it follows progress by polling listJobs under a BRIEF acquire→read→ release read lock each ~1s tick (never held across the wait), rendering progress to stderr (respecting --quiet) and a final summary to stdout; exits non-zero on job error. Ctrl-C installs a SIGINT handler that detaches (prints a note, exits 0) without cancelling. Long command, so no send default timeout is applied (a user --timeout is still honored). New `count`/`wait`/`cancel` subcommands and an enhanced `status` (queue + per-chat in_progress/pending progress + server up/down via pingServer) all use the same brief read-snapshot helper, which closes the DB handle directly rather than via shutdown() to avoid mutating server-owned jobs. `cancel` routes through the control client when a server is up and falls back to a direct cancelJobs when not. The bare `backfill` (no --chat) keeps the unchanged legacy in-process sync, and `sync` stays a silent alias. Because the parent `backfill` now declares --chat/--depth/--min-date for `backfill --chat`, commander binds those duplicated flags to the parent scope; `backfill jobs add` and `backfill cancel` read them via optsWithGlobals so both the bare command and the subcommands keep working. Tests: control-client (fetch + spawn mocked) covers ensureServer reuse vs spawn-and-wait, the race, ping/enqueue/cancel; backfill-client drives the real commander surface (services/locks/store/control-client stubbed) for foreground poll-to-completion, --background, error exit, --quiet, SIGINT-detach-without- cancel, count, status, and cancel routing up/down. 279 tests pass. Docs: docs/cli.md and SKILL.md document the server-executed/auto-start model and the new foreground/--background/status/count/wait/cancel surface.
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
PR-2b of #30 — the CLI thin client over the PR-2a control API. Backfill is now executed by the long-lived server, not the CLI process, so a
send(which needs the store write lock) is never blocked by a CLI that's busy backfilling.Closes #30. Closes #26.
New surface
backfill --chat <id> [--depth N] [--min-date ISO] [--background]—ensureServer(auto-startstgcli server --idle-exit 60sif none is up) → enqueue via the control API.listJobsunder a brief acquire→read→release read lock each ~1s (never held across the wait, so it can't block other CLI writers), renders progress to stderr (respects--quiet), final summary to stdout. Long command → no default timeout (a user--timeoutis still honored). Ctrl-C = detach (prints a note, exits 0) — it does NOT cancel; the server keeps draining.--background: prints{ jobId, channelId, status }and exits.backfill status [--json]— enhanced: queue overview + per-chat in_progress/pending progress (done/target, %, cursor) + whether the server is up.backfill count [--json]— number of in_progress backfills (cheap poll).backfill wait [--json]—ensureServerthen drain the queue.backfill cancel --chat <id>— routes through the control API when a server is up; falls back to a directcancelJobswhen not.backfill(no--chat) keeps the unchanged legacy in-process sync;syncstays a silent alias.Principles honored (from our review)
core/control-client.jsuses Node's built-in globalfetch(+AbortSignal.timeout) — no hand-rolledhttp.request. ReusesCONTROL_FILE/CONTROL_TOKEN_HEADERfromcontrol-server.jsandcore/duration.js.withReadSnapshothelper for the read paths; nothing parameterized for the sake of it.Notable find (flagged separately, not fixed here)
Pre-existing bug:
MessageSyncService.shutdown()resets allin_progressjobs topending(message-sync-service.js:2100-2106). Any read command that doescreateServices()+shutdown()would, if run while the server is mid-backfill, reset the server's running job. PR-2b's new read paths avoid this (they open the DB read-only and close the handle directly, never callingshutdown()); the existing read commands are flagged as a separate cleanup task.Tests
npm test→ 279 passing (21 files). New:tests/control-client.test.js(fetch + spawn mocked: ensureServer reuse vs spawn-and-wait, the start race, ping/enqueue/cancel) andtests/backfill-client.test.js(real commander surface, services/locks/store/client stubbed: foreground poll-to-completion,--background, error exit,--quiet, SIGINT-detach-without-cancel, count/status, cancel routing up/down). No real network/Telegram.Size note
~450 lines production (
control-client.js166 +cli.jsclient surface) / ~394 lines new tests / ~50 docs — net-new CLI functionality, not churn.