settings.yml: remove locales section.

[dalf]

What does this PR do?

In settings.yml, remove the locales section.
The available locales are detected from the searx/translations directory.
Babel provides the locale names and write direction (rtl, ltr).

There are some exceptions:

    "ar": "العَرَبِيَّة (Arabic)",  # value from settings.yml, Babel & wikipedia uses العربية , I don't know which value to use.
    "fil": "Wikang Filipino (Filipino)",  # value from settings.yml. search engines, wikipedia use "Filipino", but see https://en.wikipedia.org/wiki/Filipino_language
    "oc": "Lenga D'òc (Occitan)",  # not in Babel
    "nl_BE": "Vlaams (Dutch, Belgium)",  # value from settings.yml (Babel doesn't get the name from language + territory)

The locale names: the names are changed for the rows that start with ! :

	code	searx.locales	settings.yml	rtl
	ar	العَرَبِيَّة (Arabic)	العَرَبِيَّة (arabic)
	fil	Wikang Filipino (Filipino)	Wikang filipino (filipino)
	oc	Lenga D'òc (Occitan)	Lenga d'òc (occitan)
!	nl_BE	Vlaams (Dutch, Belgium)	Vlaams (dutch_belgium)
	bg	Български (Bulgarian)	Български (bulgarian)
	bo	བོད་སྐད་ (Tibetan)	བོད་སྐད་ (tibetan)
	ca	Català (Catalan)	Català (catalan)
	cs	Čeština (Czech)	Čeština (czech)
	cy	Cymraeg (Welsh)	Cymraeg (welsh)
	da	Dansk (Danish)	Dansk (danish)
	de	Deutsch (German)	Deutsch (german)
!	el_GR	Ελληνικά, Ελλάδα (Greek, Greece)	Ελληνικά (greek_greece)
	en	English	English
!	eo	Esperanto	Esperanto (esperanto)
	es	Español (Spanish)	Español (spanish)
	et	Eesti (Estonian)	Eesti (estonian)
	eu	Euskara (Basque)	Euskara (basque)
!	fa_IR	فارسی, ایران (Persian, Iran)	(fārsī) فارسى (persian)	rtl
	fi	Suomi (Finnish)	Suomi (finnish)
	fr	Français (French)	Français (french)
	gl	Galego (Galician)	Galego (galician)
	he	עברית (Hebrew)	עברית (hebrew)	rtl
	hr	Hrvatski (Croatian)	Hrvatski (croatian)
	hu	Magyar (Hungarian)	Magyar (hungarian)
!	ia	Interlingua	Interlingua (interlingua)
	it	Italiano (Italian)	Italiano (italian)
	ja	日本語 (Japanese)	日本語 (japanese)
	lt	Lietuvių (Lithuanian)	Lietuvių (lithuanian)
	nl	Nederlands (Dutch)	Nederlands (dutch)
	pl	Polski (Polish)	Polski (polish)
	pt	Português (Portuguese)	Português (portuguese)
!	pt_BR	Português, Brasil (Portuguese, Brazil)	Português (portuguese_brazil)
	ro	Română (Romanian)	Română (romanian)
	ru	Русский (Russian)	Русский (russian)
	sk	Slovenčina (Slovak)	Slovenčina (slovak)
!	sl	Slovenščina (Slovenian)	Slovenski (slovene)
!	sr	Српски (Serbian)	Српски (serbian)
	sv	Svenska (Swedish)	Svenska (swedish)
	ta	தமிழ் (Tamil)	தமிழ் (tamil)
	te	తెలుగు (Telugu)	తెలుగు (telugu)
	tr	Türkçe (Turkish)	Türkçe (turkish)
!	uk	Українська (Ukrainian)	Українська мова (ukrainian)
	vi	Tiếng việt (Vietnamese)	Tiếng việt (vietnamese)
!	zh_CN	中文, 中国 (Chinese, China)	中文 (chinese)
!	zh_TW	中文, 台灣 (Chinese, Taiwan)	國語 (taiwanese mandarin)

code to generate this table

if __name__ == "__main__":
    print(f"|   | code     | {'searx.locales':30} | {'settings.yml':30} | rtl  |")
    print('|---|----------|--------------------------------|--------------------------------|------|')
    from searx import settings
    for code, name in LOCALE_NAMES.items():
        settings_code = code if code != 'zh_CN' else 'zh'
        rtl = 'rtl' if code in RTL_LOCALES else ''
        status = ' '
        if name != settings['locales'][settings_code]:
            status = '!'
        print(f"| {status} | {code:8} | {name:30} | {settings['locales'][settings_code].capitalize():30} | {rtl:4} |")

Why is this change important?

Related to #195 and #228 :

• Depending of the setup settings.yml might not be synchronized with the git repository.
• With this PR is possible to add / remove new translation language easily without any change of settings.yml (from weblate).

How to test this PR locally?

• make test
• check searx_extra/update/update_currencies.py
• check searx_extra/update/update_engine_descriptions.py (optional the result is not currently used)

Author's checklist

Related issues

• Move from transifex to Weblate #195
• settings.yml: remove locales setting #228

[dalf]

ping @MarcAbonce

[return42]

@dalf I placed commit 4bee131 on top / see commit message about details.

[dalf]

delimiter is *minus* '-' --> delimiter is *minus* '_'

[return42]

Ooops .. fixed.

[MarcAbonce]

For most of the exceptions I would advocate for sticking with Babel's (actually Unicode CLDR's) names:

• Both the Occitan Wikipedia and the Unicode CLDR draft for Occitan use the term Occitan to describe the language's name.
• I'm pretty sure that wikang means language, so Wikang Kastila is the Spanish language, Wikang Ingles is English and so on.
• I think the only difference between اَلْعَرَبِيَّةُ and العربية are the diacritics, which seem to be optional and not used by default.

On the other hand, I think that using Vlaams over Nederlands for nl_BE is pretty reasonable, but maybe @unixfox knows better?

Edit: But it's worth mentioning that I don't know anything about these languages other than by skimming their Wikipedia articles. But precisely because of that I would prefer to trust the Unicode CLDR over our own personal intuitions, at least in most cases.

[dalf]

Updated. Thank you !

[return42]

LGTM .. lets merge.