Skip to content

strip harmony-style control tokens from local-engine render#26

Merged
cjus merged 1 commit into
mainfrom
carlos/solrac-scrub-local-control-tokens
May 16, 2026
Merged

strip harmony-style control tokens from local-engine render#26
cjus merged 1 commit into
mainfrom
carlos/solrac-scrub-local-control-tokens

Conversation

@cjus
Copy link
Copy Markdown
Owner

@cjus cjus commented May 16, 2026

Summary

  • Local models (gpt-oss variants surfaced via Ollama/LMStudio) leak harmony channel markers like <|channel>thought<channel|> and stray <|start|> / <|end|> / <|message|> / <|return|> tokens into the user-facing Telegram/web reply.
  • Added scrubLocalControlTokens(text) in src/local.ts and applied it at all four render touch-points: renderStreamingStub, renderFinal, renderToolLoopStub, renderToolLoopFinal.
  • Display-onlyaudit.response and recentChatTurns history still see the raw model output for forensics. Local engine only — Claude tiers are untouched.

Why

Before: <|channel>thought<channel|>It is 8:44 PM in London. rendered to the user.
After: It is 8:44 PM in London.

Three-pass scrubber:

  1. Collapse paired blocks <|name>...<name|> (drops the channel-header content too).
  2. Suppress unclosed opener at end of buffer (prevents mid-stream flicker until the closing marker arrives in a later delta).
  3. Strip stray symmetric harmony tokens (<|start|>, <|end|>, etc.) and orphan closers.

Test plan

  • npm run typecheck clean
  • bun test — 766 pass, 0 fail (added 7 new tests under describe("scrubLocalControlTokens", ...) in src/local.test.ts)
  • Manual: send a message via Telegram / web UI with LMStudio backend serving a gpt-oss / similar model; confirm channel markers are gone from the rendered reply but raw output is preserved in audit.response

local models (gpt-oss variants surfaced via Ollama/LMStudio) leak channel
markers like `<|channel>thought<channel|>` and stray `<|start|>` / `<|end|>`
tokens into the user-facing message. Add a scrubber applied at the four
render touch-points in local.ts (single-shot + tools-on, stub + final).

display-only: audit.response and recentChatTurns history still see the
raw model output for forensics. local engine only.
@cjus cjus merged commit 8b24003 into main May 16, 2026
1 check passed
@cjus cjus mentioned this pull request May 16, 2026
2 tasks
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant