Flask-Babel does not translate zh_TW and zh_CN

[u10313335]

CKAN Version if known (or site URL)

≧ 2.8.0

Please describe the expected behaviour

When switching to zh_TW or zh_CN languages on the pages written in Flask (ex. / and /user), the pages should be shown in corresponding languages.

Please describe the actual behaviour

It shows English instead of zh_TW or zh_CN.

What steps can be taken to reproduce the issue?

Switch language to Chinese on the above-mentioned pages.

[u10313335]

The problem is caused by the Flask-Babel module which is used to translate pages written in Flask.

Pylons uses Python's gettext module which is directory-based. However, Flask-Babel tries to find the directory containing translated strings through Babel's Locale.parse() function (ref1, ref2), which "normalizes" the zh_TW and zh_CN to zh_Hant_TW and zh_Hans_CN respectively. Since there is no dirs named zh_Hant_TW and zh_Hans_CN, those pages will not shown in desired languages.

A similar situation happened on flask-admin and they decided to rename the dirs. But I think renaming dirs will break backwards compatibility in CKAN if people bookmark urls with old /zh_TW or /zh_CN path. Any ideas or suggestions are welcomed.

[tino097]

@u10313335 Please check #4402. I think its fixing this issue

[u10313335]

@tino097 thanks. But #4402 is about the "label" shown on the language switcher. The problem I have described in this issue is the language of a CKAN site.

[tino097]

Sorry, i didnt read your comment, just went through the issue description :/

[TkTech]

flask-babel needs to allow passing through args to babel.parse, specifically resolve_likely_subtags=False would fix this problem.

[u10313335]

@TkTech thanks.

But I got the babel.core.UnknownLocaleError: unknown locale 'zh_TW' exception after setting resolve_likely_subtags=False.

[amercader]

@TkTech any pointers on how to progress with this?

[ltf9651]

I didn't change resolve_likely_subtags=False
I just cp zh_CN(or zh_TW) zh_Hans_CN(zh_Hant_TW)
And it works!

[u10313335]

@ltf9651 You should set the resolve_likely_subtags argument to babel.parse in flask-babel just like @TkTech said. However, I have tried that but no luck.

And, renaming dirs resolve this issue but it will break backwards compatibility in CKAN if people bookmark urls with old /zh_TW or /zh_CN path.

Please refer to my previous comment here: #4413 (comment)

[ltf9651]

@u10313335
cd /ckan/i18n
copy zh_CN zh_Hans_CN
My problem fixed, you should have a try

[u10313335]

@ltf9651 I know that renaming dirs will resolve this issue. But it will break the links with /zh_TW or /zh_CN in the older versions of CKAN.

[0wenwu]

@u10313335
cd /ckan/i18n
copy zh_CN zh_Hans_CN
My problem fixed, you should have a try

It does work properly, thanks

[linforestzhang]

@u10313335
cd /ckan/i18n
copy zh_CN zh_Hans_CN
My problem fixed, you should have a try

Thanks. It is working.

[linforestzhang]

@ltf9651 I know that renaming dirs will resolve this issue. But it will break the links with /zh_TW or /zh_CN in the older versions of CKAN.

The links remain zh_CN or zh_TW when using the workaround above provided by @u10313335.
For example, the URL is http://127.0.0.1:5000/zh_CN/dataset/example_xxx_dataset
rather than http://127.0.0.1:5000/zh_Hans_CN/dataset/example_xxx_dataset

[linforestzhang]

When switch the language to zh_CN, an error occurs with the details page of an organization:
...
File "/usr/local/lib64/python3.6/site-packages/markupsafe/init.py", line 249, in getitem
return self._kwargs[key]
KeyError: 'organization'

[Note:

This is the only problem After renaming the the locale folder 'zh_CN' to 'zh_Hans_CN' and Update the locale name from 'zh_CN' to 'zh_Hans_CN' in the config file ckan.ini but not changing any other things. That is to say, everything is okay but displaying the details page of an organization in this locale and the page can be normally displayed in the other locales.

The page shows "Error [500] - CKAN" on the page tile bar and "Internal server error " in the section of the oganization details.
]

The details page of an organization is okay when switch the language to other language such as ja or zh_TW.
And ckan/i18n uses the folder named 'zh_Hans_CN' and 'zh_Hans_TW'.
But the po and mo files remain intact as the original files released by the CKAN GitHub project.

The error is still there even after the database has been cleared and initialized.
It's okay with other languages such as en, ja or zh_TW.

[linforestzhang]

I didn't change resolve_likely_subtags=False
I just cp zh_CN(or zh_TW) zh_Hans_CN(zh_Hant_TW)
And it works!

Did you encounter the internal server error about showing the details page of an organization?

[linforestzhang]

The problem is caused by the Flask-Babel module which is used to translate pages written in Flask.
... Any ideas or suggestions are welcomed.

Is there any solution or workaround for this issue? Thanks.

[linforestzhang]

flask-babel needs to allow passing through args to babel.parse, specifically resolve_likely_subtags=False would fix this problem.
@u10313335
How to set resolve_likely_subtags=False? The python file 'core.py' in the folder '/usr/local/lib/python3.6/site-packages/Babel'? Thanks

[linforestzhang]

Workaround to be fully evaluated:
Temporarily COMMENT the line for calling the feeds snippet in the read-base of "organization". That's all. Any idea or suggestion? THX.

[linforestzhang]

The formal solution seems within the snippet feeds.html

[amercader]

@u10313335 @linforestzhang @TkTech could you please check #6008? I think that is a good compromise between doing things as Babel expects them and supporting old URLs.

[u10313335]

LGTM, thanks @amercader!