Tamil locale is using outdated encoding

[MuthuA]

BPO	40543
Nosy	@malemburg, @vstinner, @benjaminp, @ezio-melotti, @ammaraskar
Files	Screen Shot 2020-05-06 at 11.04.18 PM.png

^{Note: these values reflect the state of the issue at the time it was migrated and might not reflect the current state.}

Show more details

GitHub fields:

assignee = None
closed_at = None
created_at = <Date 2020-05-07.07:41:48.784>
labels = ['type-feature', '3.9']
title = 'Tamil locale is using outdated encoding'
updated_at = <Date 2020-05-13.08:33:22.679>
user = 'https://bugs.python.org/MuthuA'

bugs.python.org fields:

activity = <Date 2020-05-13.08:33:22.679>
actor = 'ammar2'
assignee = 'none'
closed = False
closed_date = None
closer = None
components = []
creation = <Date 2020-05-07.07:41:48.784>
creator = 'Muthu A'
dependencies = []
files = ['49137']
hgrepos = []
issue_num = 40543
keywords = []
message_count = 2.0
messages = ['368325', '368767']
nosy_count = 6.0
nosy_names = ['lemburg', 'vstinner', 'benjamin.peterson', 'ezio.melotti', 'ammar2', 'Muthu A']
pr_nums = []
priority = 'normal'
resolution = None
stage = 'needs patch'
status = 'open'
superseder = None
type = 'enhancement'
url = 'https://bugs.python.org/issue40543'
versions = ['Python 3.9']

[MuthuA]

Tamil locale (TA_IN, TA_SL, TA_SG, TA_MY) is using outdated encoding of TSCII. Tamil community is widely using UTF-8 encoding.
Further, the 'locale' standard library package in Python3 should be updated with these strings.

Should the maintainers desire assistance on this task, as a native speaker of the language, I can propose a patch or recommed other native speakers for this task.

[ammaraskar]

Hi Muthu, thanks for reporting this! Looks like this is related to bpo-20087 whereby the X11 locale data for TA_IN is using TSCII but the glibc supported file has the UTF-8 alias.

Pending a resolution to that bug, I think your best course of action would be to try to get this corrected upstream in x11, the code is here:

https://gitlab.freedesktop.org/xorg/lib/libx11/-/blob/master/nls/locale.alias.pre#L1078

and their issue tracker is here:

https://gitlab.freedesktop.org/xorg/lib/libx11/-/issues

Assuming that it gets accepted upstream, it should be simple enough to regenerate the locale module to use it.