New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

tests/ Illegal byte sequence #2669

zsau opened this Issue Dec 28, 2017 · 4 comments


None yet
2 participants

zsau commented Dec 28, 2017

Mac OS X 10.13.2, QL master branch. I'm running via the run launcher from the QL application bundle— if that's wrong, let me know.

self = <tests.test_util_cover.TCoverManager testMethod=test_file_encoding>
fn = '\udcff\udcff\udcff\udcff - cover.jpg'

    def add_file(self, fn):
        f = self.full_path(fn)
>       open(f, "wb").close()
E       OSError: [Errno 92] Illegal byte sequence: '/var/folders/j2/1zv9_6pn5656sn85r0yfxy0r0000gn/T/QL-TEST-97517g3x/tmpyqlcgujg/\udcff\udcff\udcff\udcff - cover.jpg'

tests/ OSError

This comment has been minimized.


lazka commented Dec 29, 2017

Thanks, most likely due to the filesystem change from HFS+ to APFS in 10.13.


Another difference between macOS/Linux... 😞

@lazka lazka added bug macOS labels Dec 29, 2017


This comment has been minimized.


zsau commented Dec 29, 2017

Maybe this should be separated into two tests: one with valid non-ASCII UTF-8, and one with invalid UTF-8 that only runs on Linux?


This comment has been minimized.


lazka commented Dec 29, 2017

Having one with non-ascii unicode should be enough imo (text2fsn())

Now that we are Python 3 only we no longer have to deal with bytes->unicode auto decoding issues anyway.

@lazka lazka closed this in 39aaeb3 Jan 4, 2018


This comment has been minimized.


lazka commented Jan 4, 2018

There might be more problems like this - I haven't tested on macOS.

@lazka lazka added this to the 4.0.1 milestone Jan 4, 2018

lazka added a commit that referenced this issue Jan 10, 2018

tests: Fix under macOS 10.13+. Fixes #2669
HFS+ accepted invalid utf-8 and escaped the invalid bytes.
while the new APFS errors out instead.

Instead of using invalid utf-8, use non-ascii, the next best thing
for catching potential encoding errors. As a bonus this makes it work
on Windows.
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment