Skip to content

Repository navigation

Token Spend

Jump to bottom
Autumn edited this page Oct 17, 2026 · 5 revisions

Every model session the issue pipeline runs spends tokens. This page lists each place in development work that spends them, what each place records about its spend, and how to get from a commit to the sessions that produced it and what they cost. Paths are relative to $ISSUE_PIPELINE_ROOT, the installed pipeline's root, as on Running the issue pipeline. <slug> is the repo as <owner>__<repo>, and N is the issue's number. Every time in these records is in UTC.

The pipeline records only its own sessions. Spend outside it, such as an operator's own agent sessions, isn't recorded by the pipeline.

Places that spend tokens

Under the orchestrator, each stage job runs its sessions as pi --mode rpc (runLaunch in src/job/launch.ts). The stage's code chooses the agent and the model, and the job names each session by a launch key, iN-<name>-<seq>, where <seq> counts the job's launches. The model comes from MODELS in src/workflow/sweep-config.ts, and a repo config's models key overrides any of them. The model a session actually ran with is in its context log line, since a config can change between sessions.

Place Stage Launch name Agent Model key (default) Commits it makes
Triage triage triage issue-pipeline-reader triage (Opus) None
Spec writer spec spec issue-pipeline-specifier spec (Opus) Tests, stubs, the spec draft and wiki pages, ending Pipeline-Agent: spec writer
Implementer implement implement issue-pipeline-writer implementer (Opus) Code, ending Pipeline-Agent: implementer
Review lenses review review-<lens> issue-pipeline-reader correctness and tests (Opus), scope (Sonnet) None
Aggregator, only when a lens found something blocking review aggregate issue-pipeline-reader aggregator (Sonnet) None
Fixer fix fix-r<round> issue-pipeline-writer fixer (Opus) Code, ending Pipeline-Agent: fixer round <round>
Re-reviewers fix rereview-<lens>-r<round> issue-pipeline-reader the lens's key None
CI fixer (check fixer) checks cifix-<attempt> issue-pipeline-writer fixer (Opus) Code, ending Pipeline-Agent: check fixer
Check-fix reviewers checks cifix-review-<lens>-<attempt> issue-pipeline-reader the lens's key None
Acceptance verifier accept accept issue-pipeline-reader verifier (Sonnet) None
Security reviewer security security issue-pipeline-reader security (Opus) None
Conflict resolver sync for the code; merge and publish for the wiki conflicts-<attempt> issue-pipeline-writer fixer (Opus) The rebased commits, which keep their own messages, and any repair after the rebase, ending Pipeline-Agent: conflict resolver

A continuation of a writer or spec writer session adds -c<k> to its name. Triage and the spec writer run on the role runner (src/roles/registry.ts); every other place runs on the sweep's stage code in src/workflow/stages/. MODELS also has an acceptance key, but no launch reads it: the acceptance verifier uses verifier.

Two more places spend tokens outside stage jobs:

  • Escalation agent. The orchestrator runs it once in a tick that wrote a new escalation, as pi --mode json with escalationAgentModel (default Sonnet). It is off unless escalationAgent is set. Its session is runs/<slug>/escalation-agent/session-<time>.jsonl. It makes no commits.
  • Lanes. A lane's sweep (workflow.js, superseded by the orchestrator) runs the same roles as pi-subagents children. pi-subagents keeps those sessions; nothing on this page records them.

These steps use no model: the ops (ops.mjs; the issue-pipeline-ops agent is an external command), wiki-check (a link check), verify and CI, merge, publish (except its conflict resolver) and cleanup. Two commits come from code, not a session: start-pr's first commit, "Start work on #N" with Pipeline-Agent: coordinator, and the WIP commit the job makes when it stops a writer at its context limit, with Pipeline-Agent: stage job.

The repo's other development tooling spends no tokens. The CI workflows (.github/workflows/integration.yml and release.yml) smoke-test with signed-out requests only, and no script calls a model. The web app's tests refuse to run unless TUTOR_MODEL is mock (apps/web/scripts/test-db/test-database.ts, apps/web/playwright.config.ts).

At runtime, the web app spends tokens on users' buddy chats, metered per user under claude-spend-quota. That spend isn't commit-driven, so this page doesn't cover it.

Codemode rule

Every session's task, the escalation agent's included, carries one rule on pi's codemode tool (CODEMODE_RULE in src/workflow/prompts.ts, #427). The rule tells a session to batch independent reads, greps, file searches and shell commands in one codemode script with Promise.all, and to filter or summarize large output, such as test logs, diffs and search results, in the script before it reaches the context; a single small read or an edit stays a plain tool call. Every role's agent has the tool, and a script calls only the session's own tools, so the rule grants no new access. One exception: pi keeps MCP server tools callable through codemode when --tools names no mcp__ tool, so an MCP server configured for pipeline sessions would widen every role. In the 24 hours before the rule (to 2026-10-17 21:45 UTC), 5.7% of the pipeline's top-level tool calls went through codemode.

Spend records

A stage job writes most of its records into the issue's run folder, runs/<slug>/issue-N/. That folder exists only on the machine that ran the job, and runs/ is disposable. The records on GitHub survive anywhere.

Each stage job has a job id, a UUID the orchestrator makes and passes as --job-id; a job started by hand makes its own. The job id and the job's start time join the records below. A session is named by its launch key, which is unique within one job.

Record Where Kept Tokens Cost Session key
Session transcript runs/<slug>/issue-N/sessions/<key>.jsonl Local. A later session with the same key moves the earlier file aside to <key>.<stamp>.jsonl. Per call: input, output, cacheRead, cacheWrite, cacheWrite1h, totalTokens Per call: usage.cost in US dollars, as pi prices the model The file name
Context log runs/<slug>/issue-N/context.jsonl Local, appended, never overwritten Per session: usage (the counts summed, with cacheWrite1h), peakContext, firstContext, contextTokens Per session: usage.cost key, session (the transcript's file name) and job, with role, stage, model, at, headSha and reviewedSha
Job status runs/<slug>/issue-N/job.json Local. The next job overwrites it. Per launch, as in the context log Per launch key, the job's id and startedAt, and the SHAs each writer session pushed (pushes)
Pushes log runs/<slug>/issue-N/pushes.jsonl Local, appended None None One line per push: { key, job, repo, sha, at }
Jobs log runs/<slug>/issue-N/jobs.jsonl Local, appended Per job: its sessions' counts summed, and issueTokens Per job job and jobStartedAt
Last push runs/<slug>/issue-N/pushed.json Local, last push only None None None: { sha, op, at }
Job log runs/<slug>/issue-N/job.log Local, appended None None None: each job's summary line, and any usage record the job couldn't write
Orchestrator log runs/<slug>/orchestrator.log Local, appended issueTokens on each escalated line None job and pid on each started and resumed line
Rate ledger runs/<slug>/orchestrator.json Local, one hour Per launch: input, output and cache writes None The launch key
Usage comment A hidden comment on the issue, marker <!-- issue-pipeline:N:usage --> GitHub Per session and per job, for every stage job Per session and per job key, job and jobStartedAt
State comment The issue's machine state, reportedTokens GitHub One running total of input, output and cache writes, for every session None None
Signed comments Each pipeline comment on the issue and PR GitHub None None run <key> in the signature; triage and the spec writer sign with the job id
Commit trailers The trailer lines of each pipeline commit message GitHub, and in the squash commit's body None None Pipeline-Session: <key>@<job start>, after Pipeline-Agent: <role>

The fields of job.json and of the context log are listed under Stage jobs. The usage records' fields are in Pipeline roles: shared.

Token counts mean different things in different records. A transcript's totalTokens and the context log's usage include cache reads, which every call re-reads, so they are far larger than what reportedTokens and the rate ledger count. cacheWrite1h is the part of cacheWrite written at the one-hour rate, so the five-minute writes are cacheWrite - cacheWrite1h. Cost is pi's dollar price, summed from each call's cost.total. The pipeline records no credits.

Context log fields for tracing

These context log fields join a session to its job and its commits:

  • session: the transcript's file name under sessions/, or null when the session has no launch record.
  • resumed: whether the session resumed an earlier transcript.
  • job: the job id.
  • steers: how many steers the session got, as { context, socket }. context counts the job's own steer at the soft context limit, and socket counts steers sent with --steer, such as the watchdog's.
  • headSha: the code checkout's HEAD when the session ended.
  • reviewedSha: for a session that can't commit, the HEAD it read; null for a writer.

A session with no launch record logs ended: "unrecorded", with zero measures, instead of no line.

Orchestrator log fields

Each started and resumed line carries pid, the job's process id, and job, the job id it was started with. Before a start that lifts a stopped issue, the orchestrator logs a lifted line, whose reason is the stop's code, or stopped when the state saved none.

Each escalated line carries two more fields:

  • attempt: the watchdog's restarts of the issue's stage, plus 1. For an issue saved at stopped, the stage is the one it stopped in (stoppedStage).
  • issueTokens: the issue's reportedTokens so far.

A split parent's escalated line, from a failed close, carries neither.

Trace a commit to its spend

To find the sessions behind a commit on integration and what they spent, using only GitHub, follow these steps. OWNER/REPO is the repo, such as axolotl-logic/project-buddy, and SHA is the commit.

  1. Find the PR and its branch. Each squash commit's title ends with (#<issue>) (#<pr>), and GitHub lists the PR a commit came from:

    gh api "repos/OWNER/REPO/commits/SHA/pulls" --jq '.[] | [.number, .head.ref] | @tsv'

    The branch is agent/issue-ISSUE_NUMBER, where ISSUE_NUMBER is the issue.

  2. Read the sessions that committed. The squash commit's body keeps every branch commit's message, trailers included:

    gh api "repos/OWNER/REPO/commits/SHA" --jq .commit.message \
  | grep -E '^Pipeline-(Agent|Session):'

    Each Pipeline-Session value is KEY@JOB_START: the session's launch key and its job's start time.

  3. Save the issue's usage records. ACCOUNT is the pipeline's GitHub account; the filter skips usage comments from anyone else:

    gh api --paginate "repos/OWNER/REPO/issues/ISSUE_NUMBER/comments" \
  --jq '.[] | select(.user.login == "ACCOUNT")
        | select(.body | test("<!-- issue-pipeline:[0-9]+:usage")) | .body' \
  | grep '^{' > usage.jsonl

  4. Find the record of each session from step 2. KEY and JOB_START are the two parts of its trailer:

    jq -c --arg k "KEY" --arg t "JOB_START" '
  select(.kind == "session" and .key == $k and .jobStartedAt == $t)
  | {key, role, model, job, tokens}' usage.jsonl

    tokens.cost is the session's cost in dollars, and tokens.cacheWrite1h its one-hour cache writes.

  5. List every session and job of the issue, with its cost:

    jq -r 'select(.kind == "session")
       | [.at, .job, .key, .role, .model, .tokens.cost,
          .tokens.cacheWrite1h, .tokens.cacheWrite - .tokens.cacheWrite1h]
       | @tsv' usage.jsonl
jq -r 'select(.kind == "job")
       | [.jobStartedAt, .job, .role, (.stages | join(",")), .outcome,
          .tokens.cost, .issueTokens] | @tsv' usage.jsonl

    Sessions that commit nothing (triage, the reviewers, the aggregator, the acceptance verifier and the security reviewer) belong to the PR as a whole, so their lines are the rest of the issue's spend.

  6. Optional: To check the totals on the machine that ran the issue, read the run folder. SLUG is the repo as <owner>__<repo>:

    RUN="$ISSUE_PIPELINE_ROOT/runs/SLUG/issue-ISSUE_NUMBER"
jq -r '[.at, .key, .session, .job, .model, .usage.cost] | @tsv' "$RUN/context.jsonl"
jq -r '[.at, .key, .job, .repo, .sha] | @tsv' "$RUN/pushes.jsonl"

    pushes.jsonl gives each SHA as its session pushed it. A rebase in sync or merge gives the branch's commit a new SHA, but the commit keeps its message and trailers.

To start from a commit on an agent branch instead, skip step 1: the branch agent/issue-N names the issue, and the commit's own trailers name its session. Wiki commits keep their messages and trailers when publish lands them on the wiki's master, so steps 2 to 5 work for them too.

Two commits come from code, not from a session's pipeline_commit, so they carry no Pipeline-Session trailer. "Start work on #N" spends no tokens. The WIP commit a job makes when it stops a writer at its hard context limit names the writer's launch key in its subject, "WIP: work KEY left uncommitted…".

Commits made by a job started before the install that added the trailer also carry none. Such a job gives its writers a commit scope without the job's start, and pipeline_commit then commits without the trailer rather than refusing (PR #407). On project-buddy, that install was on 2026-10-17 at 17:55 UTC. It writes no pushes.jsonl either, so trace its commits with the transcripts, as in Example: issue #381.

Example: issue #381

Commit ad7243c on integration is "Orchestrator: schedule lifted issues at their stopped stage (#381) (#384)", the squash merge of PR #384 from agent/issue-381.

  1. GitHub lists PR #384 for the commit, with head agent/issue-381.

  2. The body holds two branch commits: "Start work on #381" with Pipeline-Agent: coordinator, and the change with Pipeline-Agent: implementer.

  3. runs/<slug>/issue-381/context.jsonl holds eight sessions. The issue had no fix round, CI fix or conflict.

  4. The transcripts price them at $7.13 in all:

    Session Model Peak context Tokens Cost
    i381-triage-2 Opus 40,710 256,242 $0.47
    i381-spec-4 Opus 132,591 3,112,224 $2.25
    i381-implement-4 Opus 118,124 3,474,448 $2.15
    i381-review-correctness-5 Opus 75,542 1,156,969 $0.98
    i381-review-tests-6 Opus 61,553 340,374 $0.66
    i381-review-scope-7 Sonnet 37,568 84,492 $0.16
    i381-accept-4 Sonnet 34,713 105,001 $0.16
    i381-security-4 Opus 35,652 105,482 $0.30

  5. i381-implement-4's pipeline_commit results show code commit ce51dd8 and wiki commit 006d5e6. The rebase before merge made the code commit f27df36, the PR's head, which pushed.json records with op sync-base. publish landed the wiki commit as 08a0e55, still ending Pipeline-Agent: implementer.

#381 ran before sessions had usage records, cost or commit trailers, so on GitHub alone its chain is shorter. The PR's comments name six session keys, from i381-implement-4 to i381-security-4. #381's spec writer still ran on the sweep's stage code, so its comment on the issue names i381-spec-4; triage's names none. The issue's usage comment holds one record, the triage job's tokens. The state comment's reportedTokens is 559,398: the input, output and cache writes of every session but triage's.

Gaps closed and remaining

The records above close the gaps this page used to list:

  • Session usage records. The usage record's model accepts a provider prefix, such as anthropic/claude-opus-5-5. A record the serializer refuses is reported in job.log instead of being dropped silently.
  • Usage comments for every stage. Every stage job writes a session record per session and a job record at its end, from triage to publish.
  • Cost outside transcripts. The context log, job.json, jobs.jsonl and the usage records carry cost and cacheWrite1h.
  • A session key on commits. Each pipeline_commit commit carries a Pipeline-Session trailer, which matches one session record's key and jobStartedAt.
  • Commit SHAs per session. pushes.jsonl keeps every push with its session and job, and no job overwrites it.
  • Triage and spec in reportedTokens. Sessions on the role runner add to the issue's reportedTokens, and each job record's issueTokens is the running total.
  • Session keys in role-runner comments. Triage and spec writer comments are signed with the job id, since they post before any session key exists. The job id matches the usage records' job.

These gaps remain:

  • The escalation agent. Its spend is only in its transcript, runs/<slug>/escalation-agent/session-<time>.jsonl; it writes no usage record.
  • Lanes. Lane sweeps' sessions are kept only by pi-subagents.
  • Older records. Usage records written before these fields existed have no key, jobStartedAt, cacheWrite1h or cost, and commits by jobs started before the trailer was installed carry no Pipeline-Session trailer. Trace them with the transcripts, as in Example: issue #381.

Usage report

Each stage job leaves its records in its issue's hidden usage comment (see "Usage log" in Pipeline roles: shared). To see where context bloats, run:

node "$ISSUE_PIPELINE_ROOT/report.mjs" usage --config "$ISSUE_PIPELINE_ROOT/repos/<owner>__<repo>.json" [--since <date>] [--role <role>] [--issue <n>]

It reads the repo's issue comments in one paged listing (changed at or after --since, when given), keeps the config account's usage comments, and leaves out records from before --since. It prints:

  • per role, state and model, the session count and the median, 90th percentile and maximum of the first turn's context, the peak context and the total tokens, by nearest rank;
  • the flagged sessions, each with its reasons: first-context (the first turn's context passed usageFlags.firstContext), past-soft (the peak passed soft), tool-output (the largest tool output passed usageFlags.toolOutputShare of the peak) and first-grew (the first turn's context grew from the previous attempt's at the same issue and state);
  • each role's five tools with the most output.

The report reads only session records, so it finds no sessions for issues that ran before those reached the usage comment. For those issues, read the context log as shown under Stage jobs.

usageFlags in the repo config sets the thresholds (defaults: firstContext 40000 tokens, toolOutputShare 0.25, which must be above 0 and at most 1). The flags only label the report: they stop nothing. A missing --config, an unknown flag or a bad date, role or issue exits 1 with the reason.

Home

Clone this wiki locally