plat 201

← Pulse platform issue index

PLAT-201 — diff_patch_workspace_file doesn't mangle Unicode punctuation; the finding's own evidence matches an agent silently ASCII-folding text while hand-copying it

Coordination	Value
Assigned agent	unassigned
Ticket state	resolved — verified the tool is not the bug; shipped guidance to prevent the real cause
Last synchronized	2026-08-28

• Priority: P3 — no platform defect confirmed. A genuine agent-facing failure mode with a real (if unconfirmed) root cause, closed with preventive guidance rather than a code fix aimed at an unconfirmed target.
• Owner: agent_go/pkg/workspace/advanced_tools.go (tool description only — no behavior change).
• Related: harness:tool:diff_patch_workspace_file and harness:tool:diff_patch_workspace_file:unicode-context-mismatch (confida-login, both medium) — same incident, filed twice under two target keys with near-identical evidence; treated as one investigation. Distinct from PLAT-192 (harness:diff_patch_workspace_file:silent-partial-apply, a different failure shape: applied:true on a partially-applied patch) and from harness:diff_patch_workspace_file (2026-08-23, a JSON file, not investigated by this ticket).

The finding

A ~124-line replacement hunk against knowledgebase/notes/app-structure.md (prose containing em dashes, right-arrows, curly quotes), built by hand-copying context/removal lines from a preceding read of the same file, was rejected: "could not find matching context lines in the file. Closest match: 13 of 124 expected lines differ." A byte-count/content spot check showed the file hadn't changed in between. The identical logical edit applied immediately when the diff was instead generated by python3 difflib.unified_diff() run directly against the file. The finding's own hypothesis: "silent normalization or transport-level mangling of non-ASCII characters... between the read path and the diff-apply path."

Verified: the apply path does not mangle anything

Read applyDiffPatchFlexible/applyAgentGeneratedDiffFallback (workspace/handlers/diff_patch.go) end to end. Context-line comparison is strings.TrimSpace(resultLines[i+j]) != strings.TrimSpace(el) — a plain byte-for-byte string comparison after whitespace trimming. No Unicode normalization, no ASCII-folding, no encoding transcoding anywhere in the match or apply path; only normalizeLineEndings (CRLF→LF) touches content at all.

Proved this live with a new permanent test, TestApplyDiffPatchDoesNotMangleUnicodePunctuation (workspace/handlers/diff_patch_test.go):

• A hunk whose context is a byte-exact copy of file content containing em dashes (—), arrows (→), and curly quotes (" ") applies cleanly.
• The same hunk with every one of those characters ASCII-folded to its plain look-alike (-, ->, ") — exactly what the finding's own hand-copied hunk would look like if the agent silently substituted those characters while retyping — is correctly rejected as a context mismatch, with the real file content (including the genuine Unicode characters) shown back to the caller. Not silently corrupted, not misattributed to encoding: an accurate, evidence-bearing rejection.

This reproduces the finding's exact observed behavior (hand-copied hunk fails, difflib-generated hunk from the same file succeeds) without any platform-side mangling — consistent with the far more mundane explanation: LLMs are well known to silently substitute ASCII look-alikes for em dashes, curly quotes, and arrows when regenerating text token-by-token, which is indistinguishable to a human glance but not byte-identical to the source.

Fix: guidance, not a code change

No code behavior changed — there was nothing to fix in the apply path. Added this to diff_patch_workspace_file's diff parameter description (agent_go/pkg/workspace/advanced_tools.go): context/removal lines must be byte-exact copies of current file content, not retyped from memory, with an explicit call-out that em dashes/curly quotes/arrows are exactly the characters retyping silently corrupts — matching PLAT-190's precedent of embedding load-bearing rules directly in the tool's own description rather than a separate guidance doc an agent might not read.

Explicitly not done

• Did not add Unicode-aware/fuzzy context matching (e.g. NFKC-normalizing both sides before comparing) as a "fix" — that would make the tool more willing to apply a hunk whose actual character content silently differs from the file, which is a real corruption risk PLAT-023/192 both already guard against. The tool's current fail-closed behavior here is correct, not the bug.
• Did not confirm, and cannot confirm from this vantage point, that agent hand-copy drift is the cause of the original incident — only that it is fully consistent with the evidence and that the apply path itself is provably not the cause. If a future recurrence includes the agent's exact hand-typed hunk text next to the true file bytes, that would either confirm this diagnosis or reopen the platform-mangling hypothesis with real evidence instead of inference.

Verification

• go build ./... clean (both workspace and agent_go modules).
• go test ./... (workspace module) and go test ./pkg/workspace/... ./cmd/server/... (agent_go module): all pass; TestAllSchemaFunctionsReturnValidJSON (validates every tool description, including this edit) passes.
• New TestApplyDiffPatchDoesNotMangleUnicodePunctuation (2 subtests) passes, proving both the no-mangling claim and the fail-closed-on-real-mismatch behavior directly.