github-bound write path mangles non-ASCII characters in issue/comment bodies

[polaris-is-online]

The github-bound issue_write and add_issue_comment paths mangle non-ASCII characters in the body before they reach GitHub. Every UTF-8 multi-byte sequence starting with byte 0xE2 is truncated to a literal a-with-circumflex (a^, U+00E2) in the rendered issue body, with the trailing bytes silently dropped.

Repro

Body sent (UTF-8): str: ok-checkmark | str: cross | em-dash separator

• ok-checkmark was U+2713 (E2 9C 93)
• cross was U+2717 (E2 9C 97)
• em-dash was U+2014 (E2 80 94)

Body rendered on GitHub:

• str: a^ | str: a^ | a^ separator

The 9C 93, 9C 97, 80 94 trailing bytes are all stripped; only the first byte (E2) survives, and gets re-encoded as UTF-8 from Latin-1 (E2 -> C3 A2, which renders as a^).

Affected paths (observed)

• github-bound issue_write --body ... (used to file MCP CLI shim: list-typed parameters not serialized as arrays (breaks tavily_extract) #31; em-dashes in the body all became a^)
• github-bound add_issue_comment --body ... (the follow-up comment on MCP CLI shim: list-typed parameters not serialized as arrays (breaks tavily_extract) #31 same session, same corruption)

I have not tested whether read paths (get_issue, list_issues body output) exhibit the same corruption -- the response JSON shows the corrupted bytes have already been stored on GitHub's side, so the corruption is happening on the write path, not the read path.

Pattern

The corruption pattern (E2 9C 93 -> C3 A2; loss of trailing bytes; first byte then re-encoded UTF-8) is consistent with a pipeline that:

1. Receives the body as a UTF-8 byte string;
2. Decodes it as Latin-1 / ISO-8859-1 somewhere in the middle (where 0x9C, 0x93, 0x94, etc. are unprintable C1 control characters);
3. Strips the control characters;
4. Re-encodes the surviving bytes as UTF-8 before forwarding.

This is a classic Node.js Buffer-to-string mis-encoding (buf.toString('binary') or 'latin1' instead of 'utf8'), or an explicit text/plain; charset=ISO-8859-1 somewhere in the HTTP layer talking to GitHub.

Scope

Likely also affects any non-ASCII character whose UTF-8 leading byte falls in C1 control territory after decoding as Latin-1 -- including curly quotes, arrows, en-dashes, emoji, CJK, accented Latin characters, etc. Issues filed via github-bound will silently lose typographic punctuation.

Proposed fix

Audit the github-bound issue/comment write path for the body parameter and ensure UTF-8 is preserved end-to-end. A targeted check: log the byte length of the body at each transport hop and verify it does not change between bash CLI -> shim -> MCP transport -> GitHub API. The mismatch will be obvious for any non-ASCII body.

Related

• MCP CLI shim: list-typed parameters not serialized as arrays (breaks tavily_extract) #31 (the issue that exposed this -- file via this same pipeline, body has the corruption visible in the rendered output).

Filed via github-bound issue_write with strictly-ASCII body to avoid the bug while reporting it.

[polaris-is-online]

Update -- bug is intermittent across sessions. Issue #19 (also filed by me, same day, same github-bound issue_write path) has an em-dash in the body that rendered correctly as U+2014. The corrupting session was #31 / this one. Possible variables:

• Different host claimed the relay between the two sessions;
• Different bash subprocess locale / LC_ALL / LANG (worth checking what env the shim runs the body through);
• Different argv vs. stdin pathway depending on body length or content.

I have not isolated which. Adding it as an observation in case it narrows the search.

[karashiiro]

Reopening for further investigation as something stranger seems to be happening - while there's another instance of a corrupted emdash in #148, a PR review comment in #147 (comment) has an emdash that went through just fine. We should consider reverting 9935744 and doing a deeper investigation.

[karashiiro]

Should be fixed with cdc792a