v1.7.9
markitdown-skill v1.7.9
Fixed — silent mojibake on compressed responses
url_fetch.fetch_html() never looked at Content-Encoding, and urllib does not
decompress transparently (unlike requests). A server answering with gzip therefore had
its compressed bytes decoded as UTF-8 with errors="ignore" — the conversion produced
garbage while still exiting 0.
Reproduced live on both a Windows host and a Linux deployment:
| target | before | after |
|---|---|---|
fetch_html(https://www.python.org/) |
garbage, 6,259 chars | 52,649 chars of clean HTML |
url_to_markdown(...) output |
4,724 B of junk | 14,554 B of readable Markdown |
Content-Encoding is now undone first (gzip / deflate, including raw deflate / brotli /
zstd where the optional module is present), then the bytes are decoded using the charset
declared in Content-Type. The response is closed properly via with. Sites that do not
compress are unaffected.
Tests
scripts/tests/test_url_fetch.py grows to 5 cases: gzip, deflate (incl. raw), plain
pass-through, declared charset, and the PinnedHandler regression added in 1.7.8.
The new cases fail against the pre-fix code.