-
Notifications
You must be signed in to change notification settings - Fork 106
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
ISO-8859-2 should be detected #7
Comments
For info, I can see in src/nsSBCSGroupProber.cpp that it has been disabled lines 83-84:
I'm unsure why we'd need latin1 (ISO 8859-1) support first. Anyway files in latin1 will be wrongly detected, now they would still be wrongly detected, but in another encoding. |
I have added some ISO-8859-1 and ISO-8859-2 test files and now I understand the issue. |
Fixed with commit 6832552. |
In your README, ISO-8859-2 is not supported. Yet I can find a model for it in src/LangHungarianModel.cpp. I tried it with a ISO-8859-2 file I built myself:
https://cloud.libreart.info/public.php?service=files&t=40140bd3fd105b2c03d7716dfe4b498a
And it fails detecting it as "windows-1252".
On the other hand python-chardet was able to properly detect the ISO-8859-2 encoding:
Considering they are both supposed to be based on the same algorithm from Mozilla and that you have mention of this encoding in your code, I'm thinking it would be cool if it were supported.
The text was updated successfully, but these errors were encountered: