Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Test Unicode and bytes handling (Python 2 and 3) in all string arguments #122

Open
mgeier opened this issue Mar 29, 2015 · 1 comment
Open
Labels
Milestone

Comments

@mgeier
Copy link
Contributor

mgeier commented Mar 29, 2015

After merging #119, the file argument should support str and unicode in Python 2 and str and bytes in Python 3. The arguments mode/format/subtype/endian should support str and unicode in Python 2 and only str in Python 3 (bytes should be disallowed there).

There are some facts that are especially annoying when testing this:

  • in Python2, unicode can be implicitly converted/compared to str (as long as the string consists of only ASCII characters), this is not possible for Python3's str and bytes. That means that test cases that pass in Python2 may fail in Python3.
  • file names should be tested with both Unicode and byte strings. A bytes object may also contain non-ASCII characters. All combinations of Unicode/bytes and ASCII/non-ASCII should be tested.
  • not only the success cases but also the expected failures should be tested.
  • an (invalid) file extension may contain non-ASCII characters (but should still lead to a reasonable error message
  • If local files, the actual file system encoding is unknown, it may be hard to test sys.getfilesystemencoding().
  • as always, 'RAW' files are special, so separate test cases have to be constructed for them.
@mgeier
Copy link
Contributor Author

mgeier commented Mar 31, 2015

I repeat my recommendation here: Anyone who wants to know about the pitfalls of handling Unicode should watch this: http://nedbatchelder.com/text/unipain.html

@mgeier mgeier modified the milestone: 0.7.x May 16, 2015
@bastibe bastibe modified the milestones: 0.8.x, future Oct 4, 2015
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
Projects
None yet
Development

No branches or pull requests

2 participants