New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

tests/test_util_cover.py: Illegal byte sequence #2669

Closed
zsau opened this Issue Dec 28, 2017 · 4 comments

Comments

Projects
None yet
2 participants
@zsau
Contributor

zsau commented Dec 28, 2017

Mac OS X 10.13.2, QL master branch. I'm running setup.py via the run launcher from the QL application bundle— if that's wrong, let me know.

self = <tests.test_util_cover.TCoverManager testMethod=test_file_encoding>
fn = '\udcff\udcff\udcff\udcff - cover.jpg'

    def add_file(self, fn):
        f = self.full_path(fn)
>       open(f, "wb").close()
E       OSError: [Errno 92] Illegal byte sequence: '/var/folders/j2/1zv9_6pn5656sn85r0yfxy0r0000gn/T/QL-TEST-97517g3x/tmpyqlcgujg/\udcff\udcff\udcff\udcff - cover.jpg'

tests/test_util_cover.py:139: OSError
@lazka

This comment has been minimized.

Member

lazka commented Dec 29, 2017

Thanks, most likely due to the filesystem change from HFS+ to APFS in 10.13.

See: https://bugs.python.org/issue31380

Another difference between macOS/Linux... 😞

@lazka lazka added bug macOS labels Dec 29, 2017

@zsau

This comment has been minimized.

Contributor

zsau commented Dec 29, 2017

Maybe this should be separated into two tests: one with valid non-ASCII UTF-8, and one with invalid UTF-8 that only runs on Linux?

@lazka

This comment has been minimized.

Member

lazka commented Dec 29, 2017

Having one with non-ascii unicode should be enough imo (text2fsn())

Now that we are Python 3 only we no longer have to deal with bytes->unicode auto decoding issues anyway.

@lazka lazka closed this in 39aaeb3 Jan 4, 2018

@lazka

This comment has been minimized.

Member

lazka commented Jan 4, 2018

There might be more problems like this - I haven't tested on macOS.

@lazka lazka added this to the 4.0.1 milestone Jan 4, 2018

lazka added a commit that referenced this issue Jan 10, 2018

tests: Fix test_util_cover.py under macOS 10.13+. Fixes #2669
HFS+ accepted invalid utf-8 and escaped the invalid bytes.
while the new APFS errors out instead.

Instead of using invalid utf-8, use non-ascii, the next best thing
for catching potential encoding errors. As a bonus this makes it work
on Windows.
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment