Skip to content

fix(kiro): estimate input tokens from the full prompt, not a 500-char slice - #909

Merged
iamtoruk merged 1 commit into
getagentseal:mainfrom
ozymandiashh:fix/swarm-provider-accuracy
Aug 4, 2026
Merged

fix(kiro): estimate input tokens from the full prompt, not a 500-char slice#909
iamtoruk merged 1 commit into
getagentseal:mainfrom
ozymandiashh:fix/swarm-provider-accuracy

Conversation

@ozymandiashh

Copy link
Copy Markdown
Collaborator

Money-path accuracy fix from the systematic audit (find -> verify -> adversarial -> manual confirmation + mutation check).

kiro.ts parseChatFile estimated input tokens from pendingUserMessage - the last human turn sliced to 500 chars - while output correctly summed every bot char. So a multi-turn Kiro session, or any final prompt over 500 chars, undercounted input tokens and therefore costUSD severalfold. The asymmetry was the tell: output used a full accumulator, input used one truncated turn. The sibling modern-execution path in the same file already does the right thing with an inputChars accumulator; this makes parseChatFile match it, keeping the 500-char slice for the display userMessage only.

Mutation-checked: a 2400-char prompt reports 125 input tokens before the fix, 600 after (the real full-length estimate). tsc --noEmit clean, kiro suite green.

… slice

parseChatFile estimated input tokens from pendingUserMessage - the last
human turn sliced to 500 chars - while output summed every bot char, so a
multi-turn session or any prompt over 500 chars undercounted input tokens
and therefore costUSD severalfold. Accumulate every human turn's full
length (inputChars), matching the modern-execution path; keep the 500
slice for the display userMessage only. Mutation-checked: a 2400-char
prompt reports 125 tokens before, 600 after.

@iamtoruk iamtoruk left a comment

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Reviewed on the merged tree. Correct money-path accuracy fix: parseChatFile estimated input tokens from pendingUserMessage (the last human turn truncated to 500 chars) while output summed every bot char, so a multi-turn session or any prompt over 500 chars undercounted input tokens and costUSD severalfold. The new inputChars accumulator sums every non-identity human turn's full length (the identity skip stays ahead of it), matching the modern-execution path, and the 500-char slice is kept for the display userMessage only. Mutation-checked: reverting to the slice makes a 2400-char prompt report the truncated estimate instead of 600 input tokens, and the test also pins userMessage.length at 500 so display is unchanged. tsc clean, kiro suite green. Good to merge.

@iamtoruk
iamtoruk merged commit 0ca1d1c into getagentseal:main Aug 4, 2026
4 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants