Weblate not outputing Android <plurals> fields properly, causing app crashes

[eighthave]

With Hebrew, the user interface shows One and Other as the options:

• https://hosted.weblate.org/translate/f-droid/f-droid/he/?checksum=&offset=0&q=details_last_update_weeks&search=substring&context=on&type=all&ignored=

But the strings.xml then has One and Two filled out, then Many and Other blank:

<plurals name="details_last_update_weeks">
	<item quantity="one">עודכן לפני שבוע</item>
	<item quantity="two">עודכן לפני %1$d שבועות</item>
	<item quantity="many"></item>
	<item quantity="other"></item>
</plurals>

An empty Other string can cause crashes. Especially because we remove blank strings, since they cause other problems. Weblate should ensure that if any item of <plurals> has been translated, then <item quantity="other"> must be translated. Here's the Android source reference on the requirement for other:
https://android.googlesource.com/platform/frameworks/base/+/master/core/java/android/content/res/ResourcesImpl.java#268

Server configuration

https://hosted.weblate.org/translate/f-droid/f-droid

[eighthave]

MissingQuantity: Missing quantity translation

../../src/main/res/values-he/strings.xml:355: For locale "he" (Hebrew) the following quantities should also be defined: many, two

 352     <string name="updates__hide_updateable_apps">הסתרת יישומונים</string>
 353     <string name="updates__show_updateable_apps">הצגת יישומונים</string>
 354 
 355     <plurals name="updates__download_updates_for_apps">
 356 	<item quantity="one">הורדת עדכון ליישומון אחד.</item>
 357 	<item quantity="other">הורדת עדכון ל־%1$d יישומונים.</item>

../../src/main/res/values-iw/strings.xml:355: For locale "iw" (Hebrew) the following quantities should also be defined: many, two

 352     <string name="updates__hide_updateable_apps">הסתרת יישומונים</string>
 353     <string name="updates__show_updateable_apps">הצגת יישומונים</string>
 354 
 355     <plurals name="updates__download_updates_for_apps">
 356 	<item quantity="one">הורדת עדכון ליישומון אחד.</item>
 357 	<item quantity="other">הורדת עדכון ל־%1$d יישומונים.</item>

../../src/main/res/values-he/strings.xml:437: For locale "he" (Hebrew) the following quantities should also be defined: many, two

 434     <string name="notification_content_single_downloading_update">מתבצעת הורדת עדכון עבור „%1$s“…</string>
 435     <string name="notification_content_single_installing">ההתקנה של „%1$s“ מתבצעת כעת…</string>
 436     <string name="notification_content_single_installed">ההתקנה הצליחה</string>
 437     <plurals name="notification_summary_updates">
 438 	<item quantity="one">עדכון אחד</item>
 439 	<item quantity="other">%1$d עדכונים</item>

../../src/main/res/values-iw/strings.xml:437: For locale "iw" (Hebrew) the following quantities should also be defined: many, two

 434     <string name="notification_content_single_downloading_update">מתבצעת הורדת עדכון עבור „%1$s“…</string>
 435     <string name="notification_content_single_installing">ההתקנה של „%1$s“ מתבצעת כעת…</string>
 436     <string name="notification_content_single_installed">ההתקנה הצליחה</string>
 437     <plurals name="notification_summary_updates">
 438 	<item quantity="one">עדכון אחד</item>
 439 	<item quantity="other">%1$d עדכונים</item>

../../src/main/res/values-he/strings.xml:441: For locale "he" (Hebrew) the following quantities should also be defined: many, two

 438 	<item quantity="one">עדכון אחד</item>
 439 	<item quantity="other">%1$d עדכונים</item>
 440 </plurals>
 441     <plurals name="notification_summary_installed">
 442 	<item quantity="one">יישומון הותקן</item>
 443 	<item quantity="other">%1$d יישומונים הותקנו</item>

+ 11 More Occurrences...

../../src/main/res/values-iw/strings.xml:441: For locale "iw" (Hebrew) the following quantities should also be defined: many, two

 438 	<item quantity="one">עדכון אחד</item>
 439 	<item quantity="other">%1$d עדכונים</item>
 440 </plurals>
 441     <plurals name="notification_summary_installed">
 442 	<item quantity="one">יישומון הותקן</item>
 443 	<item quantity="other">%1$d יישומונים הותקנו</item>

../../src/main/res/values-he/strings.xml:454: For locale "he" (Hebrew) the following quantities should also be defined: many, two

 451     <string name="notification_title_summary_installed">הותקן בהצלחה</string>
 452     <string name="tts_category_name">קטגוריה %1$s</string>
 453 
 454     <plurals name="tts_view_all_in_category">
 455 	<item quantity="one">הצגת יישומון %1$d בקטגוריה %2$s</item>
 456 	<item quantity="other">הצגת כל %1$d היישומונים בקטגוריה %2$s</item>

../../src/main/res/values-iw/strings.xml:454: For locale "iw" (Hebrew) the following quantities should also be defined: many, two

 451     <string name="notification_title_summary_installed">הותקן בהצלחה</string>
 452     <string name="tts_category_name">קטגוריה %1$s</string>
 453 
 454     <plurals name="tts_view_all_in_category">
 455 	<item quantity="one">הצגת יישומון %1$d בקטגוריה %2$s</item>
 456 	<item quantity="other">הצגת כל %1$d היישומונים בקטגוריה %2$s</item>

../../src/main/res/values-he/strings.xml:460: For locale "he" (Hebrew) the following quantities should also be defined: many, two

 457 </plurals>
 458 
 459     <string name="details_last_updated_today">עודכן היום</string>
 460     <plurals name="details_last_update_days">
 461 	<item quantity="one">עודכן לפני יום</item>
 462 	<item quantity="other">עודכן לפני %1$d ימים</item>

../../src/main/res/values-iw/strings.xml:460: For locale "iw" (Hebrew) the following quantities should also be defined: many, two

 457 </plurals>
 458 
 459     <string name="details_last_updated_today">עודכן היום</string>
 460     <plurals name="details_last_update_days">
 461 	<item quantity="one">עודכן לפני יום</item>
 462 	<item quantity="other">עודכן לפני %1$d ימים</item>

../../src/main/res/values-he/strings.xml:464: For locale "he" (Hebrew) the following quantities should also be defined: many, two

 461 	<item quantity="one">עודכן לפני יום</item>
 462 	<item quantity="other">עודכן לפני %1$d ימים</item>
 463 </plurals>
 464     <plurals name="details_last_update_weeks">
 465 	<item quantity="one">עודכן לפני שבוע</item>
 466 	<item quantity="other">עודכן לפני %1$d שבועות</item>

../../src/main/res/values-iw/strings.xml:464: For locale "iw" (Hebrew) the following quantities should also be defined: many, two

 461 	<item quantity="one">עודכן לפני יום</item>
 462 	<item quantity="other">עודכן לפני %1$d ימים</item>
 463 </plurals>
 464     <plurals name="details_last_update_weeks">
 465 	<item quantity="one">עודכן לפני שבוע</item>
 466 	<item quantity="other">עודכן לפני %1$d שבועות</item>

../../src/main/res/values-he/strings.xml:468: For locale "he" (Hebrew) the following quantities should also be defined: many, two

 465 	<item quantity="one">עודכן לפני שבוע</item>
 466 	<item quantity="other">עודכן לפני %1$d שבועות</item>
 467 </plurals>
 468     <plurals name="details_last_update_months">
 469 	<item quantity="one">עודכן לפני חודש</item>
 470 	<item quantity="other">עודכן לפני %1$d חודשים</item>

../../src/main/res/values-iw/strings.xml:468: For locale "iw" (Hebrew) the following quantities should also be defined: many, two

 465 	<item quantity="one">עודכן לפני שבוע</item>
 466 	<item quantity="other">עודכן לפני %1$d שבועות</item>
 467 </plurals>
 468     <plurals name="details_last_update_months">
 469 	<item quantity="one">עודכן לפני חודש</item>
 470 	<item quantity="other">עודכן לפני %1$d חודשים</item>

../../src/main/res/values-he/strings.xml:472: For locale "he" (Hebrew) the following quantities should also be defined: many, two

 469 	<item quantity="one">עודכן לפני חודש</item>
 470 	<item quantity="other">עודכן לפני %1$d חודשים</item>
 471 </plurals>
 472     <plurals name="details_last_update_years">
 473 	<item quantity="one">עודכן לפני שנה</item>
 474 	<item quantity="other">עודכן לפני %1$d שנים</item>

../../src/main/res/values-iw/strings.xml:472: For locale "iw" (Hebrew) the following quantities should also be defined: many, two

 469 	<item quantity="one">עודכן לפני חודש</item>
 470 	<item quantity="other">עודכן לפני %1$d חודשים</item>
 471 </plurals>
 472     <plurals name="details_last_update_years">
 473 	<item quantity="one">עודכן לפני שנה</item>
 474 	<item quantity="other">עודכן לפני %1$d שנים</item>

Priority: 8 / 10
Category: Correctness:Messages

Severity: Error

Explanation: Missing quantity translation.

Different languages have different rules for grammatical agreement with quantity. In English, for example, the quantity 1 is a special case. We write "1 book", but for any other quantity we'd write "n books". This distinction between singular and plural is very common, but other languages make finer distinctions.

This lint check looks at each translation of a <plural> and makes sure that all the quantity strings considered by the given language are provided by this translation.

For example, an English translation must provide a string for quantity="one". Similarly, a Czech translation must provide a string for quantity="few".

More info: http://developer.android.com/guide/topics/resources/string-resource.html#Plurals

To suppress this error, use the issue id "MissingQuantity" as explained in the Suppressing Warnings and Errors section.

[nijel]

Apparently Gettext is not following CLDR plurals here and it causes problems in this case. Weblate is based on Gettext ones, where there are just two plural forms (n != 1), while CLDR (and Android) use four (one, two, many, other). This will be tricky to address...

[eighthave]

As a temporary workaround, maybe you could just stick the Other value into Two and Many?

[eighthave]

It seems that some languages are including more than One/Other. For example, Russian has One, Few, and Many, but is missing Other:

<plurals name="details_last_update_days">
	<item quantity="one">Обновлено %1$d день назад</item>
	<item quantity="few">Обновлено %1$d дня назад</item>
	<item quantity="many">Обновлено %1$d дней назад</item>
</plurals>

And Slovak includes One, Few, Many, and Other, but Many is apparently not supposed to be there:

<plurals name="updates__download_updates_for_apps">
	<item quantity="one">Stiahnuť aktualizáciu pre %1$d aplikáciu.</item>
	<item quantity="few">Stiahnuť aktualizácie pre %1$d aplikácie.</item>
	<item quantity="many">Stiahnuť aktualizácie pre %1$d aplikácií.</item>
	<item quantity="other">Stiahnuť aktualizácie pre %1$d aplikácií.</item>
</plurals>

[eighthave]

Serbian seems to have the correct plurals (One, Few, Other):

<plurals name="tts_view_all_in_category">
	<item quantity="one">Прикажи %1$d апликацију у категорији %2$s</item>
	<item quantity="few">Прикажи све %1$d апликације у категорији %2$s</item>
	<item quantity="other">Прикажи свих %1$d апликација у категорији %2$s</item>
</plurals>

[nijel]

This is problem only for languages where CLDR plurals do not match the ones commonly used by Gettext. I guess there won't be much of such languages...

PS: Slovak should have one/few/many/other, see http://www.unicode.org/cldr/charts/29/supplemental/language_plural_rules.html#sk.

[nijel]

Thinking more about this we really have to add plural definitions per translation, which would be based on file format and fallback to language definition if nothing is defined for file format or within the file. This way we could also address different plural forms being used for one language (such as #901).

[nijel]

Some formats will be easy to support properly:

• Use CLDR rules for Android (they can be probably extracted from Babel)
• Use Plural-Forms header for Gettext PO

Others might be a bit tricky and we will probably stick to current language defaults:

• XLIFF plurals are usually based on Gettext ones without mentioning the file format
• Qt TS rules themselves are not documented

Note: The Babel rules are not same as Gettext ones we're currently using, so it might need some more tweaks:

>>> import babel
>>> l = babel.Locale('cs')
>>> l.plural_form.tags
frozenset(['few', 'one', 'many'])
>>> l.plural_form.rules
{'few': 'i in 2..4 and v in 0', 'one': 'i in 1 and v in 0', 'many': 'v not in 0'}
>>> l.plural_form.abstract
[('few', ('and', (('relation', ('in', (u'i', ()), ('range_list', [(('value', (2,)), ('value', (4,)))]))), ('relation', ('in', (u'v', ()), ('range_list', [(('value', (0,)), ('value', (0,)))])))))), ('many', ('not', (('relation', ('in', (u'v', ()), ('range_list', [(('value', (0,)), ('value', (0,)))]))),))), ('one', ('and', (('relation', ('in', (u'i', ()), ('range_list', [(('value', (1,)), ('value', (1,)))]))), ('relation', ('in', (u'v', ()), ('range_list', [(('value', (0,)), ('value', (0,)))]))))))]

The variables are defined in the specification, there is also tool to convert them (written in PHP): https://github.com/mlocati/cldr-to-gettext-plural-rules

[nijel]

After digging into this for few hours (trying to add CLDR plurals to Weblate), now I finally understand where the problem is - it's not that the CLDR plural rules would be that different (the differences are there, but they are limited to few languages).

The problem lies a bit deeper:

Translate-toolkit always adds other to the plurals:

https://github.com/translate/translate/blob/441fcd1e40e2902e6d88616e8fd4f5dd0103bf44/translate/storage/aresource.py#L309-L310

It might sound weird, but apparently the root cause is in Babel not returning other for most of the languages it should. There is even bug reported for similar thing since April 21, see python-babel/babel#495