Translation fallbacks are not loaded in the front-end

[clarkwinkelmann]

I investigated the issues a few people had with translations and found that the fallback translations simply aren't loaded in the front-end. Most recent discussion about this on Discuss https://discuss.flarum.org/d/19684-missing-translations-flarum-shows-translation-id

Situation is as follow: an extension isn't translated in a given language. The visitor switches the language of the forum to that language.

Expected: get texts in english.

Actual: no translations available, we end up with the keys being displayed.

We can point the lack of implementation to https://github.com/flarum/core/blob/9115b9e28f9c5d1b53708090f183d71e095678ce/src/Frontend/AddTranslations.php#L57 where we essentially only give the front-end strings in the active language. Meaning the fallback strings are never retrieved.

I did some tests to see what is needed to implement it. It seems as "simple" (see issue below) as taking the example from the Symfony documentation https://symfony.com/doc/current/components/translation/usage.html#retrieving-the-message-catalogue

$catalogue = $this->locales->getTranslator()->getCatalogue($locale);
$translations = $catalogue->all('messages');
while ($fallbackCatalogue = $catalogue->getFallbackCatalogue()) {
    $translations = array_replace($fallbackCatalogue->all('messages'), $translations);
}

Trying to run this code, I kept getting "Maximum execution time exceeded" PHP errors. So I suppose maybe the current implementation was made that way because of performance concerns.

Still, it's very problematic. We need those fallback translations in the front-end.

If I find an efficient way of merging those arrays I'll make a PR but in the meantime I wanted to officially report it so we can refer to this issue.

[luceos]

Marking this as bug, but I guess we need to identify a solution to this. Also I notice that the Language manager is setting the fallback locale to "en", but that should obviously be the configured default language.

[clarkwinkelmann]

is setting the fallback locale to "en", but that should obviously be the configured default language

I don't think this is necessary the best solution. What if the current language is the default language that is not English ? Then we'd probably like to have English as fallback. And if the current language is different from the default language and both aren't English, we almost need two fallback languages, first the default, then English. Because Flarum core is only available with English, it's the only language we can guarantee to have a fallback for every core translation. Setting another third-party language as fallback might result in translations completely missing.

It gets even more complex if some extensions are not written in English fist (right now I think all of them are). Those extensions might benefit from having a custom language as their own fallback 😅

Also if an extension is only available as English, but we set the fallback to another user language and fix the issue reported here, we will end up with no translations again, because there are none in the current nor default language that are both not English.

[luceos]

You are assuming the English language pack is installed. This might not be the case at all. But yeah the issue is slightly more complex than visible at first sight.

[jordanjay29]

Perhaps this could be resolved by including a 'fallback language' field in config.php and having it auto-generate as 'en'. So Flarum could rightfully assume but allow it to be changed.

[franzliedke]

Trying to run this code, I kept getting "Maximum execution time exceeded" PHP errors. So I suppose maybe the current implementation was made that way because of performance concerns.

I'd appreciate if you could look into this a bit further. I assume it should not be a problem in production, because that code would only be run when compiling the JS assets (e.g. when enabling/disabling extensions).

Regarding fallback language:

• It's true that due to our language pack architecture, we don't really have a default fallback language.
• Falling back to the system-configured default makes sense.
• If that's easy to implement, we can also hardcode the English language pack as a last resort, if it is installed.

[clarkwinkelmann]

Could we split this into two different issues? The discussions about which language should be the default are a completely different matter than the broken fallback in the front-end. They can both be worked on separately.

[franzliedke]

Hmm, the fallback is indeed already configured and the fallback logic is implemented, so there is indeed only a bug in getting this to be triggered for the frontend assets.

[clarkwinkelmann]

@franzliedke the problem is specifically with the frontend. You can see the line I linked in the issue description in the file core/src/Frontend/AddTranslations.php. It's only fetching the catalog for the current locale, bypassing any fallback that might have been defined.

[jbrgr]

@clarkwinkelmann The example from the symfony documentation is actually the solution to this problem. There is just a small logical error in your code snipped causing an infinite while loop. 😉

This way it should work fine (at least it did in my test):

$catalogue = $this->locales->getTranslator()->getCatalogue($locale);
$translations = $catalogue->all('messages');
while ($catalogue = $catalogue->getFallbackCatalogue()) {
    $translations = array_replace($catalogue->all('messages'), $translations);
}

[clarkwinkelmann]

Nice spot! Thanks!

I'll review your PR right away.

[clarkwinkelmann]

I separated the discussion about setting the fallback locale so that we can close this one and not forget.

[jbrgr]

Thanks for taking care of that so quickly!

[bobmulder]

Can anybody give me any direction what happens? My fallback language isn't loaded either...

[clarkwinkelmann]

@bobmulder the fix has been merged into the master branch and will be released with beta 12.

[bobmulder]

Thank you for the quick response! Any release date of beta 12?

[clarkwinkelmann]

@bobmulder see https://discuss.flarum.org/d/22183-dev-diary-beta-12