feat(agent,daemon): wire async-promote queue + 5 HTTP routes (PR 2/4)

[branarakic]

Stacked on #657. Review #657 first; this PR's base will be re-targeted to main after #657 merges. The diff against main is +1,633/-2 across 4 files; the diff against #657's branch is +633/-2 across 4 files.

What

PR 2 of the async-promote-queue series. Builds on PR #657 (the TripleStoreAsyncPromoteQueue library) by exposing the queue to user code via two layers:

1. Agent surface (packages/agent/src/dkg-agent.ts, +85)

• Lazy agent.promoteQueue accessor — single shared TripleStoreAsyncPromoteQueue instance, constructed on first use against agent.store. The control graph (urn:dkg:promote-queue:control-plane) lives in the same triple store as everything else, so the queue survives daemon restarts.
• 5 new methods on agent.assertion:
  • promoteAsync(cgId, name, opts) → { jobId }
  • getPromoteAsyncStatus(jobId) → PromoteJob | null
  • listPromoteAsyncJobs(filter?) → PromoteJob[]
  • cancelPromoteAsync(jobId)
  • recoverPromoteAsync(jobId)
• agent.configurePromoteQueue(config) for tests / future daemon config plumbing — must be called before first access.

The worker-side surface (claimNext / heartbeat / succeed / fail / recordCommitMarker / recoverOnStartup) is deliberately NOT exposed on agent.assertion. PR #3's worker reaches in via agent.promoteQueue directly so user-facing callers can't accidentally drive the lifecycle.

2. HTTP routes (packages/cli/src/daemon/routes/assertion.ts, +173)

Method	Path	Behaviour	Errors
POST	/api/assertion/:name/promote-async	Enqueue → 202 { jobId, state, enqueuedAt }	400 invalid args, 409 + existingJobId on duplicate (uniqueness conflict)
GET	/api/assertion/promote-async	List, scoped by ?contextGraphId, ?state=queued,running,..., ?limit=	400 garbage filter
GET	/api/assertion/promote-async/:jobId	Single job	404 if missing
DELETE	/api/assertion/promote-async/:jobId	Cancel queued/failed_retrying	409 if running, 404 if missing
POST	/api/assertion/promote-async/:jobId/recover	Requeue failed	409 if not failed, 404 if missing

All share the SMALL_BODY_BYTES (256 KB) cap, per RFC §3.1.

Routing precedence in the linear startsWith/endsWith dispatch:

• The collection GET /promote-async is checked BEFORE the per-job GET /promote-async/:jobId so the list route isn't claimed by the per-job handler.
• The per-job GET filters out /recover-suffixed paths so the recover POST keeps its place.
• All /promote-async routes are checked AFTER the existing sync /promote route to keep that contract untouched.

3. Tests (packages/cli/test/promote-async-routes.test.ts, +375)

Mirrors import-artifact-routes.test.ts pattern: real HTTP server, real TripleStoreAsyncPromoteQueue backed by OxigraphStore, mock agent whose assertion subsurface delegates to the queue. Tests exercise the wire contract AND the queue invariants end-to-end:

• ✅ happy path enqueue + 202 response
• ✅ duplicate enqueue returns 409 with existingJobId
• ✅ missing/invalid params return 400
• ✅ explicit entities array round-trips
• ✅ GET status, list (incl. ?contextGraphId / ?state / ?limit)
• ✅ 400 on garbage filter values
• ✅ DELETE happy path + 409 on running job + 404 on missing
• ✅ POST .../recover happy path + 409 + 404
• ✅ routing precedence assertion (list vs per-job)

All 19 pass:

✓ test/promote-async-routes.test.ts (19 tests) 1547ms

Verification

# PR #1 queue tests (still green — no regression from new wiring)
cd packages/publisher && pnpm exec vitest --config vitest.unit.config.ts run test/async-promote-queue.test.ts
# → 32 passed

# PR #2 route tests
cd packages/cli && pnpm exec vitest --config vitest.unit.config.ts run test/promote-async-routes.test.ts
# → 19 passed

# tsc clean across the dependency chain (chain → core → node-ui → publisher → agent → cli)
pnpm -r build  # all green

What's still NOT in this PR (and why)

• No worker loop. Until PR the context graph explodes even after 4-5 rounds in the game #3 lands, jobs sit in queued and never transition to running. The HTTP routes still work for inspecting/cancelling/recovering — this lets us validate the agent surface against real test clients before adding the live worker.
• No memoryGraphChanged events. Enqueueing isn't a memory change; that event will be emitted from the worker (PR the context graph explodes even after 4-5 rounds in the game #3) when a job hits succeeded.
• No importLimits.promoteWorkerConcurrency on /api/status. Comes with PR the consensus is hardcoded #4.

Stacked review order

PR #657 → PR #658 (this) → PR #3 → PR #4. Each builds on the previous; each is reviewable as-is.

Made with Cursor

[github-actions]

🔴 Bug: This endpoint is now publicly available, but this PR only enqueues jobs. The diff explicitly says the worker that drains the queue ships later, so every successful POST /promote-async will sit in queued forever and never perform the WM→SWM promote. Please either wire the worker in the same PR or return 501/feature-gate this route until the queue is actually drained.

[branarakic]

Addressed in 6fe24c7.

Routes now gate on daemonState.promoteWorkerAvailable (added to daemon/state.ts, defaults to false). POST /api/assertion/:name/promote-async, the list endpoint, and GET /api/assertion/promote-async/:jobId all return 503 Service Unavailable with the body { "error": "async-promote worker is not available[: <reason>]" } until something explicitly flips the flag.

This forward-merges cleanly with the worker PR — feat/async-promote-queue-worker sets promoteWorkerAvailable = true on supervisor start, and promoteWorkerUnavailableReason = <err> if startup fails — so we never silently accept jobs into a dead queue.

Added tests in packages/cli/test/promote-async-routes.test.ts:

• POST /:name/promote-async returns 503 when the worker is unavailable
• POST /:name/promote-async surfaces the unavailability reason when set
• GET /promote-async returns 503 when the worker is unavailable
• GET /promote-async/:jobId returns 503 when the worker is unavailable

[github-actions]

🔴 Bug: Returning the raw PromoteJob here bakes the internal queue shape into the HTTP API (request, attempt, lease, numeric timestamps, and even lease.claimToken once a job is running). That does not match the RFC contract this route references, and it leaks worker-internal state that clients should not depend on. Serialize to the documented wire schema instead, and use the same mapper for the list endpoint.

[branarakic]

Addressed in 6fe24c7.

Introduced PromoteJobView (RFC §3.2) + promoteJobToView() mapper in packages/cli/src/daemon/routes/assertion.ts. The wire schema is now:

interface PromoteJobView {
  jobId: string;
  state: PromoteJobState;
  contextGraphId: string;
  assertionName: string;
  subGraphName?: string;
  entities: readonly string[] | 'all';
  enqueuedAt: string;     // ISO-8601
  updatedAt: string;      // ISO-8601
  startedAt?: string;     // ISO-8601, from lease.acquiredAt
  finishedAt?: string;    // ISO-8601, from result.succeededAt | lastError.recordedAt | updatedAt
  entitiesPromoted?: number;
  attempts: number;
  maxAttempts: number;
  nextRetryAt?: string;
  lastError?: { code: string; message: string; retryable: boolean };
  reason?: string;
}

Both GET /api/assertion/promote-async (list) and GET /api/assertion/promote-async/:jobId (status) now route through the same mapper, so the surfaces stay in lockstep. Stripped fields: request, attempt, lease (including lease.claimToken and lease.workerId), commitMarker, and the raw numeric epoch timestamps. lastError.code is mapped from the internal PromoteFailureClassification (transient | cap_exceeded | fatal).

Test additions in packages/cli/test/promote-async-routes.test.ts:

• GET /promote-async/:jobId returns the documented wire schema (RFC §3.2) — pins the flat top-level identity, entities, attempts, and ISO-8601 timestamps.
• GET /promote-async/:jobId never leaks the internal queue shape (claimToken, request, attempt, commitMarker) — forces the job into running so it has a real lease + claim token, then asserts the response carries neither.

[github-actions]

🔴 Bug: Applying the worker-availability gate to the read-only queue endpoints makes existing jobs impossible to inspect exactly when the supervisor is down. A client polling an existing jobId will get 503 instead of the last durable state (queued/failed/succeeded), and operators lose the ability to see which jobs need recovery. Keep the guard on mutating routes only; the GET routes should still read persisted queue state from storage.

[github-actions]

🔴 Bug: recover currently requeues a failed job even when promoteWorkerAvailable is false. In that state the API returns queued, but no worker can ever drain it, which recreates the same silent black-hole behavior the enqueue route is trying to prevent. Add the same availability check here (or otherwise reject recoveries while the worker is unavailable).

[github-actions]

🟡 Issue: This comment says DELETE can cancel failed_retrying jobs, but AsyncPromoteQueue.cancel() only allows the queued state. That mismatch makes the public contract harder to trust for callers and tests; either update the comment or extend the implementation if failed_retrying cancellation is intended.