Skip to content

feat(phase7): dispatch pipeline, file writing, requeue, PR#28 review fixes#29

Merged
screenleon merged 9 commits intomainfrom
feat/phase7-dispatch-fixes
Apr 27, 2026
Merged

feat(phase7): dispatch pipeline, file writing, requeue, PR#28 review fixes#29
screenleon merged 9 commits intomainfrom
feat/phase7-dispatch-fixes

Conversation

@screenleon
Copy link
Copy Markdown
Owner

Summary

  • Dispatch pipeline (connector/service.go): full task-claim → execute → submit loop with progress ticker and file-writing support
  • Migration 035: backfill project_members for existing projects so FK constraints are satisfied post-migration
  • Requeue (RequeueDispatch): task store method + handler wired to POST /api/tasks/:id/requeue-dispatch
  • File writing: connector submit result now handles files[] as path-objects or strings; files_applied count displayed in dispatch badge
  • PR feat(phase6c+phase3b): LLM router, activity SSE, context pack v2, evidence panel #28 review fixes (all blocking/critical + all 5 Copilot comments):
    • B1: Fixed send-on-closed-channel panic in activity/Hub.Update() — removed close(c) from unsub()
    • B2: SSE per-user cap (max 3) via SubscribeWithCap(connectorID, userID)
    • B3: Idle activity purge after 5 min via Hub.StartPurge(ctx)
    • B4: Reasoning sanitization in connector/suggest.go (≤1024 chars, strip control chars)
    • B5: suggest_role_test.go — 4 new handler tests covering 503/404/200+error/200+success cases
    • C1: SuggestRole always returns HTTP 200 (advisory endpoint per API-008)
    • C2: fetched.current = true set only after successful context snapshot fetch
    • C3: computeQualitySummary errors now logged instead of silently discarded
    • C4: ListActive validates project exists before listing connectors
    • C5: useConnectorActivity only starts polling in SSE onerror fallback

Test plan

  • go test ./internal/activity/... ./internal/handlers/... — all pass
  • go build ./... — no compile errors
  • Connector pairs, heartbeats, and dispatch cycle works end-to-end in local mode
  • SSE stream for connector activity connects and updates; 4th concurrent connection returns 503
  • Idle connector activity entries purge after 5 min
  • POST /api/backlog-candidates/:id/suggest-role returns HTTP 200 always (error in body on failure)
  • Dispatch badge shows "N files applied" when files_applied is in execution result
  • Context drawer retries on transient fetch error (fetched.current not set on failure)

🤖 Generated with Claude Code

screenleon and others added 6 commits April 27, 2026 12:04
…dpoint

Add suggest-only role recommendation flow: SuggestRoleFromContext uses
prompt templates in prompts/meta/ to call the configured LLM and returns
a suggested role + rationale without auto-applying. Operator must confirm
before any catalog write. CandidateRoleEditor gains a "Suggest role" button
that surfaces the recommendation inline. DECISIONS.md updated; entries
before 2026-04-22 archived to DECISIONS_ARCHIVE.md.

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
Add ActivityHub (SSE fan-out) + ActivityReporter so local connectors can
stream real-time phase/step updates. GET /api/me/local-connectors/:id/activity-stream
serves the SSE channel; GET /activity returns the latest snapshot.
Frontend: useConnectorActivity hook (SSE-primary, 15 s poll fallback, 90 s
stale detection) + ConnectorActivityBadge (compact/standard/full variants).
Badge shown inline on active planning runs in PlanningRunList. Dogfood
checklist added in docs/phase6c-dogfood-notes.md; operating-rules and
rules-quickstart updated with activity-SSE and advisory-router constraints.
CI workflow and Makefile gain affected-test helpers.

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
…+ quality view

PR-1 — PlanningContextV2 wire struct (schema_version, pack_id, role,
intent_mode, task_scale, source_of_truth[]) + scale.EstimateTaskScale
heuristic (word-count + keyword overrides). Migrations 032/033 add
planning_context_snapshots table and context_pack_id column on planning_runs.
ContextSnapshotStore Save/GetByRunID.

PR-2 — GET /api/planning-runs/:id/context-snapshot returns structured
evidence (sources, counts, role, intent_mode, task_scale). Snapshot saved
fire-and-forget in ClaimNextRun where V1 context is built; Orchestrator
gets SnapshotSaver interface as hook for future server-provider path.

PR-3 — Migration 034 adds feedback_kind/feedback_note columns on
backlog_candidates. Optional feedback popover in CandidateReviewPanel
(skippable, never blocks approve/reject). QualitySummary computed in
PlanningRunStore.GetByID; quality row shown in PlanningRunList once all
candidates are reviewed.

PR-4 — DashboardSummary gains avg_planning_acceptance_rate and
planning_runs_reviewed_count (7-day window, CASE WHEN for
SQLite+Postgres compat). ProjectOverviewTab shows acceptance rate inline.

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
Phase 3A spike findings (docs/phase3a-spike-findings.md):
- Connector route viable for CLI tools (Claude, Codex, OpenCode); not viable
  for Copilot/ChatGPT (no CLI automation surface — use server_provider).
- Critical gap identified: Phase 3B backend had GET /context-snapshot but zero
  frontend; Phase 6d not ready yet (needs dogfood data).

Evidence Panel frontend (Phase 3B PR-2 completion):
- ContextSnapshot + ContextSnapshotSourceRef types added to types/index.ts.
- getContextSnapshot(runId) added to api/client.ts.
- PlanningRunContextDrawer: lazy-loaded collapsible drawer per run showing
  source counts, V2 envelope (role/intent/scale), byte budget, pack_id,
  source_of_truth files, and truncation warnings when dropped_counts > 0.
- PlanningRunList wires the drawer on all completed/failed runs.

Connector v2 dispatch upgrade (Gap 2):
- LocalConnectorClaimNextRunResponse gains optional planning_context_v2 field.
- saveContextSnapshot now returns *wire.PlanningContextV2 so ClaimNextRun can
  include v2 envelope in the response; connectors can read role/intent_mode/
  task_scale without a second round-trip.
- Backward-compatible: planning_context (v1) still present for older connectors.

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
…ew fixes

Phase 7 dispatch fixes:
- Migration 035: backfill project_members from planning_runs + local-admin so
  ClaimNextDispatchTask finds tasks (was always empty due to missing ownership rows)
- ProjectStore.CreateWithOwner: transactionally creates project + owner member row;
  ensureLocalProject and Create handler both use it going forward
- connector/service.go: applyExecutionResultFiles writes files[] from execution
  result JSON to disk under repo_path; RepoPath added to ClaimNextTaskResponse
- TaskStore.RequeueDispatchTask + RequeueDispatch handler: lets operator retry
  stuck failed role_dispatch tasks via POST /api/tasks/:id/requeue-dispatch
- planning_runs.go: PromoteToPlannedIfDraft called at run-creation time (fixes
  dual-badge: requirement showing both "awaiting planning" and "to review")
- 30-second progress ticker in connector service logs elapsed time during CLI
  execution so operators can see whether the agent is still running

PR #28 review fixes (critic, risk-reviewer, Copilot):
- activity/hub.go: remove close(c) from unsub() — fixes send-on-closed-channel
  panic when Update() and unsub() race (B1); add SubscribeWithCap enforcing
  max 3 SSE connections per user (B2/DECISIONS §(g)); add StartPurge goroutine
  that evicts idle entries older than 5 min (B3/DECISIONS §(g))
- connector/suggest.go: sanitizeReasoning strips control chars and truncates to
  1024 chars (B4/DECISIONS §(f)); applies to Reasoning and alternatives[].Reason
- handlers/planning_runs.go: SuggestRole always returns HTTP 200; LLM errors are
  expressed in error_kind/error_message body fields per API-008 (C1/Copilot)
- handlers/connector_activity.go: Stream uses SubscribeWithCap; ListActive
  validates project exists before listing connectors (C4/Copilot)
- store/planning_run_store.go: log computeQualitySummary errors instead of
  silently discarding (C3/Copilot)
- frontend/hooks/useConnectorActivity.ts: polling starts only after SSE onerror
  fires — no longer running both SSE and polling simultaneously (C5/Copilot)
- frontend/PlanningRunContextDrawer.tsx: fetched.current set after success so
  transient errors can be retried on next open (C2/Copilot)
- models/local_connector.go: PlanningContextV2 gets TODO comment citing Phase 3A
  spike Gap 2 so connector implementors know the field is currently inert
- handlers/suggest_role_test.go: 4 new tests for suggest-role endpoint (503,
  404, LLM-error→200, success→200) — fills the coverage gap flagged by MT3
- activity/hub_test.go: 3 new tests (concurrent unsub race, per-user cap,
  StartPurge smoke) covering B1/B2/B3
- DECISIONS.md: add Phase 3B PR-3 entry for candidate feedback / quality summary
- docs/api-surface.md: document suggest-role endpoint contract
- docs/data-model.md: update migration watermark to 034

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
…dispatch badge

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
Copilot AI review requested due to automatic review settings April 27, 2026 07:03
@screenleon screenleon marked this pull request as draft April 27, 2026 07:04
Copy link
Copy Markdown

Copilot AI left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Pull request overview

Implements Phase 6c/7 end-to-end improvements across backend + connector + frontend, including connector activity visibility (SSE + polling fallback), advisory role suggestion, planning context snapshot retrieval UI, and dispatch retry tooling; also updates CI/pre-PR scripts and governance rules to support faster validation.

Changes:

  • Adds connector activity reporting + persistence + SSE endpoints, plus frontend badge/hook to display real-time connector phase.
  • Adds planning context snapshot storage/retrieval (context.v2) and a frontend drawer to inspect snapshot metadata on completed runs.
  • Adds task dispatch requeue endpoint + UI retry affordance, plus a “test affected” workflow and CI path-based job skipping.

Reviewed changes

Copilot reviewed 88 out of 88 changed files in this pull request and generated 5 comments.

Show a summary per file
File Description
scripts/test-affected.sh New script to run backend Go tests for changed pkgs + reverse deps.
scripts/pre-pr-check.sh Adds “--fast” mode to run affected-only frontend/backend tests.
rules/global/core.md Strengthens global agent rules (ambiguity handling, scope control, test-first bugfix).
rules/domain/frontend-components.md Adds UI-007 (SSE fan-out bus) + UI-008 (advisory UI must require confirmation).
rules/domain/backend-api.md Adds API-007 (SSE payload routing fields) + API-008 (advisory endpoints always 200).
go.work.sum Adds Go workspace sum entries.
frontend/src/types/index.ts Adds connector activity types, context snapshot types, quality summary, feedback fields, dashboard summary fields.
frontend/src/pages/ProjectDetail/planning/hooks/usePlanningWorkspaceData.ts Wires suggest-role + feedback submit handlers; reacts to planning-run-changed window events.
frontend/src/pages/ProjectDetail/planning/PlanningRunList.tsx Shows activity badge for active runs; adds quality summary row; adds context snapshot drawer.
frontend/src/pages/ProjectDetail/planning/PlanningRunContextDrawer.tsx New lazy-loaded context snapshot drawer component.
frontend/src/pages/ProjectDetail/TasksTab.tsx Parses execution_result files/files_applied; adds requeue “Retry” button for failed role_dispatch tasks.
frontend/src/pages/ProjectDetail/ProjectOverviewTab.tsx Displays 7-day planning acceptance rate summary when available.
frontend/src/pages/ProjectDetail/PlanningTab.tsx Plumbs role suggestion + feedback submit callbacks into candidate review panel(s).
frontend/src/pages/ProjectDetail.tsx Passes dashboard summary acceptance-rate fields into overview tab.
frontend/src/hooks/useConnectorActivity.ts New hook for connector activity via SSE + polling fallback.
frontend/src/components/ConnectorActivityBadge.tsx New badge component for connector activity display.
frontend/src/api/client.ts Adds client methods for requeue dispatch, suggest-role, context snapshot, connector activity endpoints.
frontend/src/App.tsx Fans out planning-run-changed SSE events via window CustomEvent bus.
frontend/package.json Adds test:affected script (vitest run --changed).
docs/rules-quickstart.md Updates quickstart checklist to reflect new GLOBAL-* rules.
docs/phase6c-dogfood-notes.md Adds dogfood run template/checklist for Phase 6c.
docs/phase3a-spike-findings.md Adds Phase 3A feasibility findings + recommended roadmap.
docs/operating-rules.md Adds role-dispatch safety/visibility model; updates scope control + bugfix test-first guidance.
docs/data-model.md Updates migration “current applied” marker; documents connector activity + context snapshot tables/fields.
docs/api-surface.md Documents new endpoints (context snapshot, suggest-role, activity endpoints, requeue dispatch).
backend/internal/store/task_store.go Adds RequeueDispatchTask to reset failed role_dispatch tasks to queued.
backend/internal/store/summary_store.go Computes 7-day average planning acceptance rate + reviewed run count.
backend/internal/store/project_store.go Adds CreateWithOwner to insert project_members row atomically on create.
backend/internal/store/planning_run_store.go Persists context_pack_id; attaches QualitySummary on GetByID; updates scan/selects.
backend/internal/store/local_connector_store.go Persists/restores connector activity snapshots (current_activity_json/current_activity_at).
backend/internal/store/context_snapshot_store_test.go Tests context snapshot save/get + pack_id set on run create.
backend/internal/store/context_snapshot_store.go New store for planning_context_snapshots table.
backend/internal/store/backlog_candidate_store.go Adds feedback_kind/note persistence + validation; expands select/scan/update.
backend/internal/store/backlog_candidate_feedback_test.go Store tests for candidate feedback + quality summary aggregation.
backend/internal/router/router.go Wires new routes: context snapshot, suggest-role, requeue dispatch, activity endpoints.
backend/internal/roles/catalog_test.go Updates prompt dir comparison; adds meta prompt file validation test.
backend/internal/roles/catalog.go Adds meta role dispatcher (CategoryMeta).
backend/internal/prompts/render.go Embeds meta/*.md prompts in addition to roles.
backend/internal/prompts/meta/dispatcher.md New dispatcher meta-prompt definition.
backend/internal/planning/wire/context_v2_test.go Tests context.v2 constants + v1→v2 upgrade behavior.
backend/internal/planning/wire/context_v2.go Adds PlanningContextV2 envelope + UpgradeV1ToV2 helper.
backend/internal/planning/scale/scale_test.go Tests task-scale heuristic behavior.
backend/internal/planning/scale/scale.go Implements task-scale estimation heuristic.
backend/internal/planning/orchestrator.go Adds SnapshotSaver interface + orchestrator wiring hook.
backend/internal/models/summary.go Adds dashboard acceptance-rate fields to API model.
backend/internal/models/requirement.go Adds feedback kinds, quality summary model, context_pack_id + quality_summary on PlanningRun, router_no_match error kind.
backend/internal/models/local_connector.go Adds connector activity models + ClaimNextRun response includes PlanningContextV2.
backend/internal/handlers/tasks.go Adds POST requeue-dispatch handler.
backend/internal/handlers/suggest_role_test.go Handler-level tests for suggest-role advisory contract and error cases.
backend/internal/handlers/projects.go Ensures project create inserts owner membership via CreateWithOwner.
backend/internal/handlers/planning_runs_context.go Adds GET context snapshot handler with structured/raw modes.
backend/internal/handlers/planning_runs.go Adds suggest-role handler + feedback_kind validation + context snapshot store wiring.
backend/internal/handlers/local_connectors.go Saves context snapshots on ClaimNextRun; publishes planning-run-changed SSE; wires broker + snapshot saver.
backend/internal/handlers/connector_dispatch.go Adds repo_path to claim-next-task response so connector can apply files.
backend/internal/handlers/connector_activity_test.go Handler/store tests for activity report + polling/list endpoints.
backend/internal/handlers/connector_activity.go Adds connector activity ingest + polling + SSE stream + active-connectors list.
backend/internal/handlers/candidate_feedback_test.go Handler tests for feedback_kind validation on PATCH candidate.
backend/internal/database/database.go Adds SQLite rewrite for gen_random_uuid() in migrations.
backend/internal/connector/suggest.go Implements server-side dispatcher meta-prompt execution + output validation/sanitization.
backend/internal/connector/dispatch.go Adds repo_path to connector-side claim-next-task response DTO.
backend/internal/connector/client.go Adds client.ReportActivity() for connector.
backend/internal/connector/app.go Starts ActivityReporter and wires into connector service startup.
backend/internal/connector/activity_reporter.go Adds async/coalescing activity reporter for connector → server updates.
backend/internal/activity/hub_test.go Adds tests for hub broadcast, cap, persistence, and concurrency behaviors.
backend/internal/activity/hub.go Implements in-process activity hub with persistence, SSE cap, idle purge.
backend/db/migrations/035_backfill_project_members.sql Backfills project_members for existing projects/users (incl local-admin).
backend/db/migrations/035_backfill_project_members.down.sql No-op down migration.
backend/db/migrations/034_candidate_feedback.sql Adds feedback_kind + feedback_note columns.
backend/db/migrations/034_candidate_feedback.down.sql No-op down migration for SQLite compatibility.
backend/db/migrations/033_planning_runs_pack_id.sql Adds context_pack_id to planning_runs.
backend/db/migrations/033_planning_runs_pack_id.down.sql No-op down migration.
backend/db/migrations/032_planning_context_snapshots.sql Adds planning_context_snapshots table + index.
backend/db/migrations/032_planning_context_snapshots.down.sql Drops planning_context_snapshots table + index.
backend/db/migrations/031_connector_activity.sql Adds current_activity_json/current_activity_at columns.
backend/db/migrations/031_connector_activity.down.sql Notes/no-op down guidance.
backend/cmd/server/main.go Wires context snapshot store, role suggester, activity hub + purge, and SSE broker; ensures local project has owner membership.
Makefile Adds test-affected targets for backend/frontend.
AGENTS.md Expands pre-PR gate to include additional reviewer roles.
.github/workflows/ci.yml Adds path-change detection job and conditional job execution.
.claude/agents/testing-reviewer.md Adds testing-reviewer agent doc.
.claude/agents/architecture-reviewer.md Adds architecture-reviewer agent doc.

Comment thread backend/internal/handlers/tasks.go
Comment thread frontend/src/hooks/useConnectorActivity.ts Outdated
Comment thread frontend/src/components/ConnectorActivityBadge.tsx
Comment thread backend/internal/activity/hub_test.go Outdated
Comment thread backend/internal/planning/scale/scale.go Outdated
…c/risk review

Blocking:
- B1: defer close(done/runDone) in RunOnceTask/RunOnce so ticker goroutines exit
  on all paths including panic (previously only closed inline, not deferred)
- B2: reject non-absolute repoPath in applyExecutionResultFiles to prevent
  accidental writes outside the repo root on misconfigured projects
- B3: export PurgeIdle() so tests can invoke it directly; rewrite
  TestStartPurge_EvictsIdleEntries to TestPurgeIdle_EvictsOldIdleOnly which
  actually exercises the eviction path (previous test was a false green)

Should-fix:
- Add explicit nil-user guard in RequeueDispatch (returns 401, not confusing 409)
- Update data-model.md migration watermark to 035
- Add requeue-dispatch endpoint to api-surface.md
- Add Phase 7 DECISIONS.md entry covering defer-close, best-effort file writes,
  stuck-running gap (Phase 8 TODO), PurgeIdle export, migration 035 strategy

Minor:
- UTF-8 safe truncation in sanitizeReasoning (trim to valid rune boundary)

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
@screenleon screenleon marked this pull request as ready for review April 27, 2026 07:46
screenleon and others added 2 commits April 27, 2026 17:06
C1 (tasks.go:235): RequeueDispatch now checks task existence with GetByID first
  — returns 404 for unknown IDs instead of the misleading 409 that
  ErrDispatchOwnership produces for any zero-row UPDATE

C2 (useConnectorActivity.ts): remove direct EventSource creation (UI-007).
  Per-connector activity stream cannot be centralized in App.tsx without a
  global stream endpoint (Phase 8 TODO). Converted to polling-only; 15s
  interval is acceptable for dogfood use. SSE type removed from ActivitySource.

C3 (ConnectorActivityBadge.tsx): move from frontend/src/components/ to
  frontend/src/pages/ProjectDetail/planning/ per DECISIONS.md (ProjectDetail
  tab/panel components must not live in src/components/)

C4 (hub_test.go): already fixed in prior commit — TestStartPurge_EvictsIdleEntries
  replaced with TestPurgeIdle_EvictsOldIdleOnly which calls PurgeIdle() directly

C5 (scale.go:35): remove unused smallKeywords variable (compile error risk)
  and simplify classification rule comments (rules 3+4 were both "small",
  now collapsed to rule 3)

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
Keep HEAD versions for all conflicts:
- DECISIONS.md: Phase 7 entry retained
- hub.go/hub_test.go: PurgeIdle export + TestPurgeIdle test retained
- suggest.go: unicode/utf8 safe truncation retained
- scale.go: smallKeywords removal retained
- data-model.md: migration 035 watermark retained
- useConnectorActivity.ts: polling-only (UI-007) retained
- PlanningRunList.tsx: local ConnectorActivityBadge import retained

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
@screenleon screenleon merged commit 2535fb5 into main Apr 27, 2026
@screenleon screenleon deleted the feat/phase7-dispatch-fixes branch April 27, 2026 08:10
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants