Permalink
Browse files

Fix #192 - resolve problems with encoding and transliterator between …

…various ruby versions.
  • Loading branch information...
1 parent 35b4490 commit ea81ca0486c9e37fa543a238c0e18a7db969f27d @knapo knapo committed Feb 27, 2013
Showing with 12 additions and 9 deletions.
  1. +1 −1 lib/i18n/backend/transliterator.rb
  2. +10 −7 lib/i18n/core_ext/string/encoding.rb
  3. +1 −1 lib/i18n/tests/interpolation.rb
@@ -92,7 +92,7 @@ def approximations
# Add transliteration rules to the approximations hash.
def add(hash)
hash.keys.each do |key|
- utf8_key = key.to_s.dup.force_encoding('UTF-8')
+ utf8_key = ::String.force_utf8(key)
hash[utf8_key] = hash.delete(key).to_s
end
approximations.merge! hash
@@ -1,10 +1,13 @@
-# Makes String#force_encoding working with ruby 1.8.7
-unless defined?(Encoding)
- require 'iconv'
-
- class String
- def force_encoding(encoding)
- ::Iconv.conv('UTF-8//IGNORE', encoding.upcase, self)
+class String
+ if Object.const_defined?(:Encoding)
@josevalim

josevalim Feb 27, 2013

Collaborator

if "utf-8".respond_to?(:force_encoding) would be more appropriate, no?

@knapo

knapo Feb 27, 2013

Collaborator

yes, it would be better here.

+ def self.force_utf8(obj)
+ obj.to_s.dup.force_encoding('UTF-8')
+ end
+ else
+ require 'iconv'
+
+ def self.force_utf8(obj)
+ ::Iconv.conv('UTF-8//IGNORE', 'UTF-8', obj.to_s.dup)
end
end
end
@@ -66,7 +66,7 @@ module Interpolation
assert_equal 'こんにちは、ゆきひろさん!', interpolate(:default => 'こんにちは、%{name}さん!', :name => 'ゆきひろ')
end
- if Kernel.const_defined?(:Encoding)
+ if Object.const_defined?(:Encoding)
test "interpolation: given a euc-jp translation and a utf-8 value it raises Encoding::CompatibilityError" do
assert_raise(Encoding::CompatibilityError) do
interpolate(:default => euc_jp('こんにちは、%{name}さん!'), :name => 'ゆきひろ')

4 comments on commit ea81ca0

Collaborator

josevalim replied Feb 27, 2013

Since when do we use iconv? That would be super slow for 1.8. :(

Collaborator

knapo replied Feb 27, 2013

Since df79fa5 what solve the transliterator issue for jruby. We could leave key as it is for 1.8 and force when String#force_encoding defined. What to you think?

Collaborator

josevalim replied Feb 27, 2013

Honestly, I don't think we should do anything at all. Calling force_encoding also seems a bad idea, what if someone sends a string in ISO? The calling code should be responsible to send valid UTF-8 strings or, at least, Strings that can be converted to UTF-8 (namely ASCII).

Collaborator

knapo replied Feb 27, 2013

Good point. Agree - calling code side should handle it, if it uses/needs encoding other than UTF-8.

Please sign in to comment.