This repository has been archived by the owner on Jun 1, 2023. It is now read-only.
Unicode mixed script confusables #229
Assignees
Labels
Comments
|
The remaining question if certain languages need alias for sets of Scripts, because they use multiple scripts by default. Such as Japanese for Hiragana and Katakana (what about Kanji? = Han?), |
rurban
pushed a commit
that referenced
this issue
Dec 3, 2016
Document the new unicode mixed script confusable security restriction. Declare valid unicode scripts via use utf8 arguments. This bug was introduced with 5.16. See #229.
rurban
pushed a commit
that referenced
this issue
Dec 3, 2016
add isLATIN_or_COMMON_uni with data from from lib/unicore/To/Sc.pl add dummy utf8_check_script(uv) to check if the script was declared. See #229.
rurban
pushed a commit
that referenced
this issue
Dec 4, 2016
let them be generated by regen/regcharclass.pl change API from UV cp to U8*s. Spares a costly utf8n_to_uvchr conversion in hot code. Do this only when throwing the error. Slowdown <1% See GH #229.
rurban
pushed a commit
that referenced
this issue
Dec 4, 2016
and uvuni_get_script, and a utf8_error_script helper. done via the simple Unicode::UCD::charscript method. See #229
ghost
removed
the
Sign up for free
to subscribe to this conversation on GitHub.
Already have an account?
Sign in.
In order to avoid TR39 confusable security hacks, we add the following unicode rules for identifiers and literals:
See http://www.unicode.org/reports/tr39/#Mixed_Script_Detection
This holds for all identifiers (all names: package, gv, sub, variables) and literal numbers.
The scriptname is returned by
Unicode::UCD::charscript($codepoint_as_uv)Currently there exist 131 scripts:
perl -alne'/; (\w+) #/ && print $1' lib/unicore/Scripts.txt | sort -u > scripts.lst
The text was updated successfully, but these errors were encountered: