-
Notifications
You must be signed in to change notification settings - Fork 7.7k
Commit
This commit does not belong to any branch on this repository, and may belong to a fork outside of the repository.
Revert/fix substitution character fallback
The introduced checks were not correct in two respects: * It was checked whether the source encoding of the string matches the internal encoding, while the actually relevant encoding is the *target* encoding. * Even if the correct encoding is used, the checks are still too conservative. Just because something is not a "Unicode-encoding" does not mean that it does not map any non-ASCII characters. I've reverted the added checks and instead adjusted mbfl_convert to first try to use the provided substitution character and if that fails, perform the fallback to '?' at that point. This means that any codepoint mapped in the target encoding should now be correctly supported and anything else should fall back to '?'.
- Loading branch information
Showing
5 changed files
with
30 additions
and
39 deletions.
There are no files selected for viewing
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters