Skip to content

docs(samples): keep the manual test fixtures where the samples are - #550

Merged
PathGao merged 1 commit into
masterfrom
docs/encoding-and-link-samples
Aug 8, 2026
Merged

docs(samples): keep the manual test fixtures where the samples are#550
PathGao merged 1 commit into
masterfrom
docs/encoding-and-link-samples

Conversation

@PathGao

@PathGao PathGao commented Aug 8, 2026

Copy link
Copy Markdown
Collaborator

Five documents that existed only in scratch directories while #207, #372 and
#393 were being worked on, and got regenerated by hand every time somebody
needed to check something. They belong beside stress-test.md, which is the
same kind of thing — and two open issues now point at them as reproductions.

file for
link-test.md 805 B the three link cases the editor and preview treat differently (#393 item 5, and #549's reproduction)
encoding-gbk.md 7 KB GBK below the 50 KB preview threshold — the full-read path
encoding-gbk-large.md 101 KB the same content at twice the threshold — the truncated-preview path, and the reproduction for #547
encoding-shift-jis.md 600 B Japanese, so a detector cannot pass by guessing a Chinese encoding
encoding-cp1252.md 359 B French, no CJK at all

Each carries a header saying what it is and what should happen, written in
its own encoding
, so opening one in Markpad is itself the instructions.

Why the CP-1252 one is worth 359 bytes

#372 reads as a Chinese-and-Japanese problem and is not. A single é is enough
to make a file invalid UTF-8, and every "my document opened as ���" report from
a European user has the same cause. This fixture is a French note with no CJK
character in it, and it reproduces the bug in full.

Why the large GBK one is twice the threshold

The preview renders the first 50,000 bytes and a background read fetches the
rest. At 101 KB the missing half is most of the document, so a full load that
fails to land is obvious rather than subtle — which is what #547 needs, and it
matches the 121 KB document the symptom was first seen on.

What writing them demonstrated

Shift_JIS refused the simplified 样 and an em dash; CP-1252 refused Chinese
outright. Each had to be written in the language its encoding was built for.
That is the same refusal encode_text raises when someone pastes an emoji into
a GBK document (#372) — pleasant to have run into it from the other side while
producing the fixtures for it.

.gitattributes

samples/encoding-*.md get -text -diff.

-text opts them out of * text=auto eol=lf. Normalisation would not actually
corrupt them — no GBK or Shift_JIS trail byte can be 0x0A or 0x0D — but it is
not worth resting on.

-diff is the separate half and the one that shows: without it git diffs them
as text and prints their bytes through a UTF-8 lens, so every review that
touches one reads a screen of mojibake. With it, "Binary files differ".

Every fixture is verified to decode as the encoding it claims and to be invalid
UTF-8; 926 tests and npm run check are unaffected.

Five documents that existed only in scratch directories while #207, #372 and
#393 were being worked on, and got regenerated by hand every time somebody
needed to check something. They belong beside `stress-test.md`, which is the
same kind of thing.

- **`link-test.md`** — the three link cases the editor and the preview treat
  differently: an `https://` URL, a `file://` URL, and both again as Markdown
  links. Monaco's link detector recognises exactly `http://`, `https://` and
  `file://`, and the `file:` branch of its opener is
  `mainWindow.location.href = href`, which in Tauri means the app's own window.
  This is the document that showed the app's CSP refuses that navigation
  (#393).

- **`encoding-gbk.md`** and **`encoding-gbk-large.md`** — the same GBK content
  either side of the 50 KB preview threshold, so the full-read path and the
  truncated-preview path can both be exercised. The large one is also a
  reproduction for #547.

- **`encoding-shift-jis.md`** — Japanese, so a detector cannot pass by
  guessing a Chinese encoding.

- **`encoding-cp1252.md`** — 359 bytes of French with no CJK in it at all. The
  point of this one is that #372 is not a Chinese problem: a single `é` is
  enough to make a file invalid UTF-8, and every "my document opened as ���"
  report from a European user has the same cause.

Each carries a header saying what it is and what should happen, in its own
encoding, so opening one in Markpad is itself the instructions.

Writing them was its own demonstration: Shift_JIS refused the simplified 样 and
an em dash, and CP-1252 refused Chinese outright — which is exactly the
`unmappable` refusal `encode_text` raises when someone pastes an emoji into a
GBK document.

`.gitattributes` gives them `-text -diff`. `-text` opts them out of
`* text=auto eol=lf`; line-ending normalisation would not in fact corrupt them
— no GBK or Shift_JIS trail byte can be 0x0A or 0x0D — but it is not worth
resting on. `-diff` is the separate half and the one that shows: without it git
diffs them as text and prints their bytes through a UTF-8 lens, so every review
that touches one reads a screen of mojibake.
@PathGao
PathGao merged commit 7e59ca3 into master Aug 8, 2026
4 checks passed
@PathGao
PathGao deleted the docs/encoding-and-link-samples branch August 8, 2026 10:33
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant