-
-
Notifications
You must be signed in to change notification settings - Fork 30.1k
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Improve encoding alias handling in locale coercion tests #81516
Comments
Locale coercion tests on Solaris are failing because 646 ASCII alias is not recognized. Its addition into the _handle_output_variations function fixes this problem. This was changed/fixed in Python 3.8 and later, where aliases are correctly translated to their canonical Python codec name so no patch is needed there. |
test_c_locale_coerce should use "codecs.lookup(encoding).name" to get the normalized name of an encoding, rather than fragile: data = data.replace(b"ANSI_X3.4-1968", b"ascii")
data = data.replace(b"US-ASCII", b"ascii") the proposed pattern is even more dangerous: data = data.replace(b"646", b"ascii") I'm not sure where encodings should be normalized. Maybe around _check_child_encoding_details(). For the PR, please write it for the master branch. |
I just added that in the way it was already there but I see why the current solution is not the best. Also I wanted to push this into 3.7 only as this problem is not present in 3.8 (as discussed in the PR 11195 opened incorrectly against the master). Just to be sure: what you propose is to rewrite current replaces to use "codecs.lookup(encoding).name" instead and then push it into the master? |
I suggest to remove the code which does the .replace(), but instead normalize the encoding when checking for the expected encoding (near .assertEqual()). I still see the .replace() code in master, so yeah, the code should first be changed in master: @staticmethod
def _handle_output_variations(data):
"""Adjust the output to handle platform specific idiosyncrasies
|
Python 3.8+ encodings are always normalized and thus no output variations handling is necessary (the code is no longer necessary). Python 3.7 (and possibly lower) can have variations in encodings - that should be fixed with codecs.lookup functions. |
Thanks Jakub Kulik. test_c_locale_coercion should pass again on 3.7, 3.8 and master branches on Solaris. |
Note: these values reflect the state of the issue at the time it was migrated and might not reflect the current state.
Show more details
GitHub fields:
bugs.python.org fields:
The text was updated successfully, but these errors were encountered: