New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Better handling of InvalidPluralizationData #123

Open
nikosd opened this Issue Nov 11, 2011 · 11 comments

Comments

Projects
None yet
10 participants
@nikosd
Contributor

nikosd commented Nov 11, 2011

I'm having some hard times trying to handle errors related to missing translations for languages with non 'germanic' pluralization forms (for example Polish, Russian, etc).

The problem / current behaviour

The problem comes up when the following scenario is true :

# Given the locale is `ru`
I18n.locale = :ru

# And the pluralization key for `2` is :few
pluralizer.call(2) # => :few

# And the ru messages are missing (ru.po)
#
# msgid "singular text"
# msgid_plural "plural text"
# msgstr[0] ""
# msgstr[1] ""
# msgstr[2] ""

# Asking for translations with count => 2 raises exception
n_('singular text', 'plural text', 2) # => raises InvalidPluralizationData

Wanted behaviour

Instead I would like the following :

# Asking for translations with count => 2 should return the default plural form
n_('singular text', 'plural text', 2) # => 'plural text'

Current (hack) workaround

To actually make this work in our own applications I have made the following monkey patch on the pluralization module :

module I18n
  module Backend
    module Pluralization
      # Overriding the pluralization method so if the proper plural form is missing we will try
      # to fallback to the default gettext plural form (which is the `germanic` one).
      def pluralize(locale, entry, count)
        return entry unless entry.is_a?(Hash) and count

        pluralizer = pluralizer(locale)
        if pluralizer.respond_to?(:call)
          return entry[:zero] if count == 0 && entry.has_key?(:zero)

          plural_key = pluralizer.call(count)
          return entry[plural_key] if entry.has_key?(plural_key)

          # fallback to the default gettext plural forms if real entry is missing (for example :few)
          default_gettext_key = count == 1 ? :one : :other
          return entry[default_gettext_key] if entry.has_key?(default_gettext_key)

          # If nothing is found throw the classic exception
          raise InvalidPluralizationData.new(entry, count)
        else
          super
        end
      end
    end
  end
end

but this is not the right way to do it since it's way too opinionated.

Possible approches / Ideal solutions

Ideally, any of the following would be good solutions :

A. Using the generic I18n.exception_handler

# config/initializers/i18n.rb
module I18n
  def self.handle_invalid_pluralization_data(*args)
    exception = args.first
    if exception.is_a?(InvalidPluralizationData)
      # do your thing here ...
    else
      super
    end
  end
end

I18n.exception_handler = :handle_invalid_pluralization_data

to make this work something InvalidPluralizationData should become a MissingTranslation which makes sense to me also in terms of semantics, since it's not the case that the localization data are invalid, it's that a specialized localization datum is missing.

alternatively :

B. Using a specialized I18n::Backend::Pluralization.invalid_pluralization_handler

# config/initializers/i18n.rb
I18n::Backend::Pluralization.invalid_pluralization_handler = Proc.new do |locale, key, count|
  # to the handling here
end

which means that the pluralization backend should be changed to something like :

module I18n
  module Backend
    module Pluralization
      # ...
      def pluralize(locale, entry, count)
        return entry unless entry.is_a?(Hash) and count

        pluralizer = pluralizer(locale)
        if pluralizer.respond_to?(:call)
          key = count == 0 && entry.has_key?(:zero) ? :zero : pluralizer.call(count)
          entry.has_key?(key) ? entry[key] : pluralization_error_proc.call(locale, entry, count)
        else
          super
        end
      end

      # ...
    end
  end
end
@nikosd

This comment has been minimized.

Show comment
Hide comment
@nikosd

nikosd Nov 11, 2011

Contributor

For the record this is how C# Gettext is handling this issue by default (yikes) :

    public virtual String GetPluralString (String msgid, String msgidPlural, long n) {
      Object value = GetObject(msgid);
      if (value == null || value is String)
        return (String)value;
      else if (value is String[]) {
        String[] choices = (String[]) value;
        long index = PluralEval(n);
        return choices[index >= 0 && index < choices.Length ? index : 0];
      } else
        throw new InvalidOperationException("resource for \""+msgid+"\" in "+GetType().FullName+" is not a string");
    }
Contributor

nikosd commented Nov 11, 2011

For the record this is how C# Gettext is handling this issue by default (yikes) :

    public virtual String GetPluralString (String msgid, String msgidPlural, long n) {
      Object value = GetObject(msgid);
      if (value == null || value is String)
        return (String)value;
      else if (value is String[]) {
        String[] choices = (String[]) value;
        long index = PluralEval(n);
        return choices[index >= 0 && index < choices.Length ? index : 0];
      } else
        throw new InvalidOperationException("resource for \""+msgid+"\" in "+GetType().FullName+" is not a string");
    }
@svenfuchs

This comment has been minimized.

Show comment
Hide comment
@svenfuchs

svenfuchs Nov 11, 2011

Owner

Interesting.

Lemme first see if I get the issue right.

So, you have pluralization data that does not fit the current pluralization algorithm and you want to pick a default, but there's no way to handle that.

Does the default not depend on the current locale as well? I could imagine it's different for different locales?

What exactly would you do in handle_invalid_pluralization_data where you say "do your thing here"?

Owner

svenfuchs commented Nov 11, 2011

Interesting.

Lemme first see if I get the issue right.

So, you have pluralization data that does not fit the current pluralization algorithm and you want to pick a default, but there's no way to handle that.

Does the default not depend on the current locale as well? I could imagine it's different for different locales?

What exactly would you do in handle_invalid_pluralization_data where you say "do your thing here"?

@nikosd

This comment has been minimized.

Show comment
Hide comment
@nikosd

nikosd Nov 11, 2011

Contributor

A simple example of a graceful fallback :

# config/initializers/i18n.rb
module I18n
  def self.handle_invalid_pluralization_data(*args)
    exception = args.first
    if exception.is_a?(InvalidPluralizationData)
      key = exception.count == 1 ? :one : :other
      return exception.entry[key] if exception.entry.has_key?(key)
    else
      super
    end
  end
end

I18n.exception_handler = :handle_invalid_pluralization_data

where the super actually means call the default exception handler (I'm not sure that this example would work)

Does this make it any clearer or should add an exact scenario with english default values (singular/plural) and polish as the requested language (I18n.locale)?

Contributor

nikosd commented Nov 11, 2011

A simple example of a graceful fallback :

# config/initializers/i18n.rb
module I18n
  def self.handle_invalid_pluralization_data(*args)
    exception = args.first
    if exception.is_a?(InvalidPluralizationData)
      key = exception.count == 1 ? :one : :other
      return exception.entry[key] if exception.entry.has_key?(key)
    else
      super
    end
  end
end

I18n.exception_handler = :handle_invalid_pluralization_data

where the super actually means call the default exception handler (I'm not sure that this example would work)

Does this make it any clearer or should add an exact scenario with english default values (singular/plural) and polish as the requested language (I18n.locale)?

@marcusg

This comment has been minimized.

Show comment
Hide comment
@marcusg

marcusg Dec 22, 2012

+1

any updates here?

marcusg commented Dec 22, 2012

+1

any updates here?

@merlagautham

This comment has been minimized.

Show comment
Hide comment
@merlagautham

merlagautham Jan 31, 2013

+1

In our case if the translator does not give all the plural form I just want to fall back to the default language (English in our case) instead of showing an exception to end users.

merlagautham commented Jan 31, 2013

+1

In our case if the translator does not give all the plural form I just want to fall back to the default language (English in our case) instead of showing an exception to end users.

@tigrish

This comment has been minimized.

Show comment
Hide comment
@tigrish

tigrish Feb 1, 2013

Collaborator

The way we handle this when we export data on Locale is to use the :other form. AFAIK, this form is defined by all pluralization rules.

Collaborator

tigrish commented Feb 1, 2013

The way we handle this when we export data on Locale is to use the :other form. AFAIK, this form is defined by all pluralization rules.

@mixmix

This comment has been minimized.

Show comment
Hide comment
@mixmix

mixmix Feb 26, 2014

I think i'm striking an issue similar to this with some asian languages (japanese, indonesian, vietnamese) where there is no difference between singular and plural.
In this case Transifex records only generate the :other translation, and our app is crashing because it is unnecessarily looking for the singular translation :one

Is there a tidy solution to this?

mixmix commented Feb 26, 2014

I think i'm striking an issue similar to this with some asian languages (japanese, indonesian, vietnamese) where there is no difference between singular and plural.
In this case Transifex records only generate the :other translation, and our app is crashing because it is unnecessarily looking for the singular translation :one

Is there a tidy solution to this?

@JasonBarnabe JasonBarnabe referenced this issue Jun 29, 2014

Closed

Add i18n #87

@jsilland

This comment has been minimized.

Show comment
Hide comment
@jsilland

jsilland Aug 12, 2014

+1 to resolving this issue. As it currently stands, there are basically no ways to properly recover from a missing plural form. An acceptable fall back would be to switch back the locale to English in this case, as is the case for when a translation is simply missing.

jsilland commented Aug 12, 2014

+1 to resolving this issue. As it currently stands, there are basically no ways to properly recover from a missing plural form. An acceptable fall back would be to switch back the locale to English in this case, as is the case for when a translation is simply missing.

@Startouf

This comment has been minimized.

Show comment
Hide comment
@Startouf

Startouf Feb 18, 2015

I also run into frequent translations errors for French, mainly related to using :many instead of :other, etc. I believe there are already some merge requests to fix this issue,

Startouf commented Feb 18, 2015

I also run into frequent translations errors for French, mainly related to using :many instead of :other, etc. I believe there are already some merge requests to fix this issue,

@radar

This comment has been minimized.

Show comment
Hide comment
@radar

radar Nov 20, 2016

Collaborator

I would accept a PR here to fix this issue.

Collaborator

radar commented Nov 20, 2016

I would accept a PR here to fix this issue.

@radar radar added the pr-required label Nov 20, 2016

@radar radar added this to the 0.9.0 milestone Nov 20, 2016

@sandstrom

This comment has been minimized.

Show comment
Hide comment
@sandstrom

sandstrom Dec 4, 2016

How widespread is gettext for translation keys? Anyone know what the most popular methods for storing translations are these days?

My guess would be that most are yml, json or some database (redis, sql, etc). If anyone know I'd be very happy to hear!

sandstrom commented Dec 4, 2016

How widespread is gettext for translation keys? Anyone know what the most popular methods for storing translations are these days?

My guess would be that most are yml, json or some database (redis, sql, etc). If anyone know I'd be very happy to hear!

@radar radar modified the milestones: 0.9.0, 0.10.0 Oct 1, 2017

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment