-
Notifications
You must be signed in to change notification settings - Fork 3
plat 088
PLAT-088 — every scheduled workflow and Pulse turn was billed to chat, so Pulse cost could not be measured
| Coordination | Value |
|---|---|
| Assigned agent | unassigned |
| Ticket state |
implemented — attribution fixed and tested; runtime reverify pending |
| Last synchronized | 2026-08-11 |
-
Priority: P1 — not a crash, but it makes the "what does Pulse cost us
versus the goal work?" question unanswerable, and silently inflates the
chatbucket with automation spend nobody typed. -
Owner: cost scope attribution (
handleQueryobserver construction,pkg/costobserver) - Found on: 2026-08-11, while answering "how do we measure cost we spend on Pulse vs the goal process".
Every cost row for the instagram scheduled run
(schedule-manual--bae435e5_1786432476119732000, 2026-08-11) landed in
scope='chat' — including the Pulse Gate, Review+Fix parent, and Finalize
turns, which the logs show running at 08:56, 08:58, 09:48 and 09:53 UTC:
08:56:19 chat schedule-manual--bae435e5_… $1.56 ← Pulse Gate turn
08:58:36 chat schedule-manual--bae435e5_… $0.82 ← Pulse Review+Fix turn
09:48:05 chat schedule-manual--bae435e5_… $1.06 ← Pulse Finalize turn
09:53:59 chat schedule-manual--bae435e5_… $3.18 ← Pulse post-finalize turn
Only Pulse's delegated background children were tagged correctly, because
their execution ids literally contain the string "pulse"
(bg-pulse-review+fix:-engineering…), which matchPhaseScope matches:
09:19:38 pulse bg-pulse-review+fix:-engineeri $11.14
09:34:58 pulse bg-pulse-review+fix:-engineeri $12.87
09:40:28 pulse bg-pulse-review+fix:-engineeri $4.90
09:47:20 pulse bg-pulse-review+fix:-engineeri $4.56
So Pulse's measured cost for that run was $33.47 when the real figure was
roughly $40 — and the missing ~$6.62 was charged to chat. Corroborating the
scale of the problem: the ledger contains zero rows with scope='builder'
in its entire history, because the code path that should produce them cannot.
handleQuery captures isWorkflowPhase := req.AgentMode == "workflow_phase"
(server.go:3215), then destructively rewrites the field a hundred lines
later, purely to route the request down the standard agent path:
// Convert to multi-agent mode so it falls through to the standard agent path
req.AgentMode = "multi-agent" // server.go:3321The cost observer is constructed ~2200 lines further down (server.go:5561)
and inferred its scope from that rewritten value:
inferCostScope(req.AgentMode, workflowPhaseID) // ("multi-agent", "workflow-builder")InferScope matches "multi-agent" against neither chief nor workflow,
so it fell through to its default — ScopeChat. The routing rewrite and the
cost attribution are ~2200 lines apart with no stated relationship, so nothing
flagged that one silently destroyed the other's only input.
Separately, no signal on that path could have identified a Pulse turn even
without the rewrite: a scheduled Pulse turn runs in the same session, with
the same agent mode and the same workflow-builder phase id, as the workflow
orchestration turns before it. Mode and phase genuinely cannot tell them
apart.
Two parts, matching the two causes:
-
Honor the scheduler's explicit marker. New
costobserver.ScopeForScheduledLLMRole(llmConfigSource)maps the stamp the scheduler already sets when it swaps in the Pulse/maintenance LLM (applyPulseLLMToReqMap→llm_config_source = "scheduled_pulse") onto a cost scope.scheduled_auto_improvemaps to Pulse deliberately: Goal Advisor and Strategy Auditor are Pulse modules that merely run on the maintenance LLM. Returns""for anything else so callers keep their own default rather than inheriting a wrong one. -
Stop reading the rewritten mode. Scope resolution now uses
isWorkflowPhase— captured before the rewrite — to pass the realworkflow_phasemode, so scheduled orchestration turns resolve tobuilderrather thanchat.
Priority order is marker first, then mode+phase, because only the marker can separate Pulse from workflow inside one scheduled session.
Historical rows are not back-filled and must not be read as if they were.
Every scheduled run before this fix has its Pulse-stage and
workflow-orchestration spend sitting in chat. Any "Pulse vs goal" comparison
over pre-2026-08-11 data understates Pulse and overstates chat. (Distinct
from, and additional to, the 15,235 legacy unknown-scope rows, which stopped
being produced on 2026-07-12 and are a separate pre-attribution artifact.)
-
go build ./pkg/costobserver/clean;go vetclean;gofmtclean. - New tests (
plat088_scheduled_role_scope_test.go): every scheduled-role marker maps to the right scope and unknown markers return"";InferScope("workflow_phase", …)yieldsbuilderwhile the rewritten"multi-agent"yieldschat— pinning the exact regression rather than implying it. - The full
./cmd/server/...suite could not be run at commit time: another session's in-flightCreateSchedule/UpdateSchedulesignature refactor leavesinteractive_workshop_manager.gouncompilable. Confirmed by stashing that work that HEAD builds clean and that no build error is attributable to this change. Re-run the suite once that refactor lands. -
Not yet reverified live — needs a restart, then one scheduled run;
expect its Pulse turns to appear under
scope='pulse'and its orchestration turns underscope='builder', with neither inchat.
- A scheduled run's Pulse Gate / Review+Fix / Finalize turns are recorded as
scope='pulse', notchat. - A scheduled run's workflow-orchestration turns are recorded as
scope='builder', notchat. -
chatcontains only genuine interactive chat.
Auto-synced from docs/ on main. Edit there, not here.