Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

tests/test_util_cover.py: Illegal byte sequence #2669

Closed
zsau opened this issue Dec 28, 2017 · 4 comments
Closed

tests/test_util_cover.py: Illegal byte sequence #2669

zsau opened this issue Dec 28, 2017 · 4 comments
Milestone

Comments

@zsau
Copy link
Contributor

@zsau zsau commented Dec 28, 2017

Mac OS X 10.13.2, QL master branch. I'm running setup.py via the run launcher from the QL application bundle— if that's wrong, let me know.

self = <tests.test_util_cover.TCoverManager testMethod=test_file_encoding>
fn = '\udcff\udcff\udcff\udcff - cover.jpg'

    def add_file(self, fn):
        f = self.full_path(fn)
>       open(f, "wb").close()
E       OSError: [Errno 92] Illegal byte sequence: '/var/folders/j2/1zv9_6pn5656sn85r0yfxy0r0000gn/T/QL-TEST-97517g3x/tmpyqlcgujg/\udcff\udcff\udcff\udcff - cover.jpg'

tests/test_util_cover.py:139: OSError
@lazka
Copy link
Member

@lazka lazka commented Dec 29, 2017

Thanks, most likely due to the filesystem change from HFS+ to APFS in 10.13.

See: https://bugs.python.org/issue31380

Another difference between macOS/Linux... 😞

@zsau
Copy link
Contributor Author

@zsau zsau commented Dec 29, 2017

Maybe this should be separated into two tests: one with valid non-ASCII UTF-8, and one with invalid UTF-8 that only runs on Linux?

@lazka
Copy link
Member

@lazka lazka commented Dec 29, 2017

Having one with non-ascii unicode should be enough imo (text2fsn())

Now that we are Python 3 only we no longer have to deal with bytes->unicode auto decoding issues anyway.

@lazka lazka closed this in 39aaeb3 Jan 4, 2018
@lazka
Copy link
Member

@lazka lazka commented Jan 4, 2018

There might be more problems like this - I haven't tested on macOS.

@lazka lazka added this to the 4.0.1 milestone Jan 4, 2018
lazka added a commit that referenced this issue Jan 10, 2018
HFS+ accepted invalid utf-8 and escaped the invalid bytes.
while the new APFS errors out instead.

Instead of using invalid utf-8, use non-ascii, the next best thing
for catching potential encoding errors. As a bonus this makes it work
on Windows.
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
Linked pull requests

Successfully merging a pull request may close this issue.

None yet
2 participants