fix: write binary skill other-files without UTF-8 corruption - #2568
Merged
dyoshikawa merged 1 commit intoAug 4, 2026
Merged
Conversation
Skill other files (e.g. images inside a skill directory) were read and
written through a UTF-8 text path: the buffer was decoded with
toString('utf-8') and written back with writeFileContent, so invalid byte
sequences were replaced with U+FFFD and binary files were silently
corrupted on both import and generate.
Route binary other files through a buffer path instead:
- isBinaryBuffer() detects content that a UTF-8 round-trip would corrupt
- change detection compares bytes via Buffer.equals (readFileBufferOrNull)
- writes go through writeFileBuffer, keeping text files on the existing
text path (trailing-newline normalization + structured equivalence)
Owner
|
@greenhands-always Thank you! |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
2 participants
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Problem
Skill other files (anything besides
SKILL.mdinside a skill directory — e.g.wechat-qrcode.jpg,assets/hero.gif) are corrupted on bothimportandgenerate.DirFeatureProcessor.writeAiDirsreads and writes those files through a UTF-8 text path:readFileContentOrNull(UTF-8) +fileContentsEquivalentfile.fileBuffer.toString("utf-8")thenwriteFileContent(UTF-8)Invalid byte sequences in binary content are replaced with U+FFFD during the round-trip, so a 50 KB JPEG silently becomes a ~87 KB file with a broken header:
Fix
Route binary other files through a buffer path while keeping text files on the existing text path:
isBinaryBuffer()(new, inutils/file.ts) — detects whether a UTF-8toString+Buffer.fromround-trip would alter the bytes (valid UTF-8 text incl. CJK round-trips unchanged; JPEG/GIF etc. do not)Buffer.equalsusing a newreadFileBufferOrNull()writeFileBuffer()Verification
spawn bun ENOENT— bun not installed in this environment, unrelated)oxfmt,oxlint,tsgo(typecheck),tsdown(build) all cleanwechat-qrcode.jpgandassets/hero.gif— files remain valid (filereports JPEG/GIF) and md5 matches the source