Additional confusables characters + deduping existing

[diox]

More confusable characters for our custom normalization, but also deduping the existing ones:

• Remove characters already handled by unicodedata normalization
• Remove characters already known to be confusables by unicode
• Remove characters already considered confusable with another letter: we only use this table to normalize into a single character, so having confusable characters in more than one list is not useful. This was mostly a problem for i and l.

Testing

With the patch applied, you can check what we think a given character is confusable with by doing:

>>> s =  'ⵊ'
>>> list(generate_lowercase_homoglyphs_variants_for_string(normalize_string_for_name_checks(s)))
['i', 'l']

That allows spot-checking any characters you are having doubts for.

[eviljeff]

Do you think it's feasible have a test that checks for duplication programmatically? Manually de-duping seems like an inefficient process.

[diox]

I automated the deduping, so I'll look into add a test so that we don't accidentally add more dupes in the future yeah. I fear it might be a bit too slow though...

[diox]

I've added a script to check for dupes in a0fd570 but left it separate in scripts/, with the expectation that we run it next time we upgrade homoglyphs_fork or add new characters. It's too slow to run as part of the test suite IMHO.