Permalink
Browse files

Fix I18n.transliterate for non utf-8 keys (occurs e.g. in jruby 1.7.0…

… which have US-ASCII encoded symbols)
  • Loading branch information...
1 parent d9dbe2e commit df79fa5eb7752154da9e390879c8646b854b87a0 @knapo knapo committed Dec 5, 2012
Showing with 23 additions and 9 deletions.
  1. +11 −8 lib/i18n/backend/transliterator.rb
  2. +10 −0 lib/i18n/core_ext/string/encoding.rb
  3. +2 −1 test/backend/transliterator_test.rb
@@ -83,16 +83,19 @@ def transliterate(string, replacement = nil)
private
- def approximations
- @approximations ||= {}
- end
+ def approximations
+ @approximations ||= {}
+ end
- # Add transliteration rules to the approximations hash.
- def add(hash)
- hash.keys.each {|key| hash[key.to_s] = hash.delete(key).to_s}
- approximations.merge! hash
+ # Add transliteration rules to the approximations hash.
+ def add(hash)
+ hash.keys.each do |key|
+ utf8_key = key.to_s.dup.force_encoding('UTF-8')
+ hash[utf8_key] = hash.delete(key).to_s
end
+ approximations.merge! hash
+ end
end
end
end
-end
+end
@@ -0,0 +1,10 @@
+# Makes String#force_encoding working with ruby 1.8.7
+unless defined?(Encoding)
+ require 'iconv'
+
+ class String
+ def force_encoding(encoding)
+ ::Iconv.conv('UTF-8//IGNORE', encoding.upcase, self)
+ end
+ end
+end
@@ -1,5 +1,6 @@
-# :coding: utf-8
+# encoding: utf-8
require 'test_helper'
+require 'i18n/core_ext/string/encoding'
class I18nBackendTransliterator < Test::Unit::TestCase
def setup

4 comments on commit df79fa5

@josevalim
Collaborator

Do we have a test case or a way to reproduce this issue? Just install jruby 1.7.0?

@knapo
Collaborator
knapo commented on df79fa5 Feb 27, 2013

yes, install jruby and run tests.

@josevalim
Collaborator

I think the solution is to not have symbols as keys. YAML keys are not transformed into symbols by default but they keep their proper encoding. Am I missing something? Can any of the default ways that data is loaded into transliterate be passing symbols?

@knapo
Collaborator
knapo commented on df79fa5 Feb 27, 2013

I don't know any, transliterating symbols doesn't have much sense. Thanks Jose!

Please sign in to comment.