Script: read and write text files as UTF-8, not the platform default

[rbeezer]

Resolves a Windows-only failure reported on pretext-support: Fatal (but not really?) charmap error.

PreTeXt's Python opened text files without naming an encoding, so Python fell back on the platform default: UTF-8 on Linux and macOS, but cp1252 on Windows. Since every file PreTeXt writes is UTF-8, reading one of its own files back on Windows fails on the first byte outside the cp1252 repertoire, with 'charmap' codec can't decode byte 0x9d.

The reporter's build looked "fatal but not really" because the PDF is genuinely complete when this happens: the failure is in reading the LaTeX log afterward to decide whether another LaTeX pass is needed, which marks the run failed and leaves the finished PDF in the temporary directory. The same bug in a second place stops -V validation, which reads back the assembled source and so meets the author's own prose.

Thirty-eight sites were affected — every text-mode open() plus the subprocess.run that captures jing's output — so this would have kept resurfacing. Two were worse than a crash: the Asymptote path decoded a diagram as cp1252 and re-encoded it as UTF-8, silently mangling non-ASCII content instead of reporting anything. All now name encoding="utf-8". Two reads additionally take errors="replace", because they consume output from programs whose encoding we do not control: the LaTeX log, which is whatever the TeX engine and its packages emitted (strict UTF-8 also fails on the reported byte), and jing's stdout, a JVM program whose encoding varies by platform and Java version.

Verified by forcing Python's default encoding to ASCII (PYTHONUTF8=0 PYTHONCOERCECLOCALE=0 LC_ALL=C), a stricter form of the Windows condition. Before the change, validation of the sample article fails with UnicodeDecodeError at position 7290, matching the reporter's position 7315 at the same site in their own document. After it, HTML, LaTeX, EPUB, PDF, braille, and validation all complete under that setting and produce byte-identical output to a normal run. On Linux the change is a no-op across every format.

Claude Opus 5 (1M context), acting as a coding assistant for Rob Beezer