Languages Names(2)

[glasseyes]

Please fill in the following items if you don't know the root cause.

Which distribution and version?:
Ubuntu bionic, Debian buster

Which desktop environment and version?:
GNOME 3.28, MATE 1.20.1

Which session type?:
X11

Which application and version?:
ibus-setup 1.5.17
using ibus-keyman 10.99.45 (https://downloads.keyman.com/linux/alpha/10.99.45/ibus-keyman-10.99.45.tar.gz which requires https://downloads.keyman.com/linux/alpha/10.99.45/keyman-keyboardprocessor-0.0.0~201812051519.tar.gz as a build dependency and also https://downloads.keyman.com/linux/alpha/10.99.45/keyman_config-10.99.45.tar.gz for installing Keyman keyboards)
But it can be demonstrated with other ibus-engines

IBus version?:
1.5.17

Issue description:
If you create an IBusEngineDesc with a language that isn't in iso 639-2 then it is given language "Other" e.g. Northern Thai (iso 639-3 code "nod") instead of the name of the language

Steps to reproduce:

1. Create an engine in your favourite engine that has a language "nod"
2. open ibus-setup tab "Input Method" and click Add. Search for Thai. "Thai, Northern" does not appear. You have to go to language "Other" to add the input method.
3. There are many (potentially over 6000) other languages that this happens for.

Can you reproduce your problem when you restart ibus-daemon? (yes / no):
yes

Do you see any errors when you run ibus-daemon with the verbose option?:
no

Can you reproduce your problem with a new user account instead of the current your account? (yes / no):
yes

This is #2062 in the template

[fujiwarat]

Thank you for compiling the report again.

Can you attach the output of /usr/lib*/ibus-engine-keyman --xml in this issue?
I don't know the content of kmp_language.languages in ibus_keyman_add_engines().

[glasseyes]

This is the xml for just the Northern Thai keyboard. I'll attach the full file with my currently installed keyboards which has other examples separately.

        <engine>
            <name>/usr/local/share/keyman/sil_boonkit/sil_boonkit.kmx</name>
            <longname>Boonkit Unicode</longname>
            <description>This keyboard layout is designed for Northern Thai with Lana Script.
© SIL International 2018</description>
            <language>nod</language>
            <license>mit</license>
            <author></author>
            <icon>/usr/share/keyman/icons/default.png</icon>
            <layout>en</layout>
            <layout_variant></layout_variant>
            <layout_option></layout_option>
            <hotkeys></hotkeys>
            <symbol></symbol>
            <setup></setup>
            <version>0.6.3</version>
            <textdomain></textdomain>
            <icon_prop_key></icon_prop_key>
            <rank>0</rank>
        </engine>
</engines>

[glasseyes]

example output of ibus-keyman-engine --xml with a few Keyman keyboards installed
ibus-keyman.xml.gz

[fujiwarat]

OK, why do you wish to use "nod" instead of "th" in the engine compose file?
The "nod" expresses "Thai, Northern" but I don't want to increase the number of the languages on ibus-setup.
Actually your problem is ibus_get_language_name() in ibus/src/ibusutil.c which supports iso_639.xml only but I think it's good not to separate Thai between north and south in the input method categories.
I think you can use <longname> tag to explain the engine is for north.

[glasseyes]

Northern Thai probably wasn't a helpful example. The language is a separate language from Thai so I can't get "th" when generating the IBusEngineDesc. Also, this keyboard is for input in the Lana script not the Thai script so it definitely doesn't support the "th" language.

Another example is Tai Nüa "tdd" which doesn't have something else you could use.

Sorry, yes, ibus_get_language_name() and other places that parse the iso_639.xml is what I changed in the PR #2061

The languages only appear in ibus-setup if you have installed keyboards to support them so I don't think they should be a noticable increase unless you choose to install a lot of keyboards.
It is confusing to users to try to find the keyboard in language "Other" instead of searching for their language among other languages.

[fujiwarat]

OK. Since there are multiple categories for Greek and Indic, probably I'm fine to use multiple categories for Thai.

Sorry, yes, ibus_get_language_name() and other places that parse the iso_639.xml is what I changed in the PR #2061

You don't have to mind the referred script but ibus_get_language_name() only

Probably it's better to use /usr/share/iso-codes/json/iso_639-3.json with glib-json since the xml files have a comment that files are deprecated. But I have no time at the moment.

[glasseyes]

I agree that iso_639-3.json is better to use because the xml files are deprecated and may disappear at some point. I'll work on that soon.
I did the xml first because the patch is smaller. It would be great if you would be happy to accept that PR first but it will be superceded by the later one when it is done.

[fujiwarat]

Thank you for the patch and integrated the most part.
As I noted, you don't have to update iso639converter.py which is used for simple.xml and I will keep the language names with two characters for the engine.

[glasseyes]

Thanks :) I've got the change from using xml files to json basically done but I still want to test it before I submit the PR.