Skip to content
Browse files

Revert "Remove unicode character encoding from ActiveSupport::JSON.en…

…code"

This reverts commit 815a943.

Conflicts:
	activesupport/test/json/encoding_test.rb

Reason: This was causing a regression where the resulting string is always
returning UTF-8. Also it changes the behavior of this method on a stable release.

Fixes #9498.
  • Loading branch information...
1 parent 8218a46 commit c910388587220e962682b0b9187e79b8f1641c17 @rafaelfranca rafaelfranca committed May 8, 2013
View
14 activesupport/CHANGELOG.md
@@ -1,5 +1,12 @@
## unreleased ##
+* Revert the changes on unicode character encoding from `ActiveSupport::JSON.encode`.
+ This was causing a regression where the resulting string is always returning UTF-8.
+ Also it changes the behavior of this method on a stable release.
+ Fixes #9498.
+
+ *Rafael Mendonça França*
+
* Fix `ActiveSupport::TimeZone.parse` when time is at a local DST jump.
Fixes #9678.
@@ -13,13 +20,6 @@
*Dan Kubb*
-* Remove surrogate unicode character encoding from ActiveSupport::JSON.encode
- The encoding scheme was broken for unicode characters outside the basic
- multilingual plane; since json is assumed to be UTF-8, and we already force the
- encoding to UTF-8 simply pass through the un-encoded characters.
-
- *Brett Carter*
-
* Fix mocha v0.13.0 compatibility. *James Mead*
* `#as_json` isolates options when encoding a hash. [Backport #8185]
View
8 activesupport/lib/active_support/json/encoding.rb
@@ -122,7 +122,13 @@ def escape(string)
if string.respond_to?(:force_encoding)
string = string.encode(::Encoding::UTF_8, :undef => :replace).force_encoding(::Encoding::BINARY)
end
- json = string.gsub(escape_regex) { |s| ESCAPED_CHARS[s] }
+ json = string.
+ gsub(escape_regex) { |s| ESCAPED_CHARS[s] }.
+ gsub(/([\xC0-\xDF][\x80-\xBF]|
+ [\xE0-\xEF][\x80-\xBF]{2}|
+ [\xF0-\xF7][\x80-\xBF]{3})+/nx) { |s|
+ s.unpack("U*").pack("n*").unpack("H*")[0].gsub(/.{4}/n, '\\\\u\&')
+ }
json = %("#{json}")
json.force_encoding(::Encoding::UTF_8) if json.respond_to?(:force_encoding)
json
View
24 activesupport/test/json/encoding_test.rb
@@ -100,11 +100,11 @@ def test_hash_encoding
def test_utf8_string_encoded_properly_when_kcode_is_utf8
with_kcode 'UTF8' do
result = ActiveSupport::JSON.encode('€2.99')
- assert_equal '"€2.99"', result
+ assert_equal '"\\u20ac2.99"', result
assert_equal(Encoding::UTF_8, result.encoding) if result.respond_to?(:encoding)
result = ActiveSupport::JSON.encode('✎☺')
- assert_equal '"✎☺"', result
+ assert_equal '"\\u270e\\u263a"', result
assert_equal(Encoding::UTF_8, result.encoding) if result.respond_to?(:encoding)
end
end
@@ -113,22 +113,16 @@ def test_utf8_string_encoded_properly_when_kcode_is_utf8
def test_non_utf8_string_transcodes
s = ''.encode('Shift_JIS')
result = ActiveSupport::JSON.encode(s)
- assert_equal '""', result
+ assert_equal '"\\u4e8c"', result
assert_equal Encoding::UTF_8, result.encoding
end
- end
-
- def test_wide_utf8_chars
- w = '𠜎'
- result = ActiveSupport::JSON.encode(w)
- assert_equal '"𠜎"', result
- end
- def test_wide_utf8_roundtrip
- hash = { :string => "𐒑" }
- json = ActiveSupport::JSON.encode(hash)
- decoded_hash = ActiveSupport::JSON.decode(json)
- assert_equal "𐒑", decoded_hash['string']
+ def test_utf8_hash_key
+ w = { '𠜎' => 'a' }
+ result = ActiveSupport::JSON.encode(w)
+ assert_equal '{"\\u070e":"a"}', result
+ assert_equal Encoding::US_ASCII, result.encoding
+ end
end
def test_exception_raised_when_encoding_circular_reference_in_array

0 comments on commit c910388

Please sign in to comment.
Something went wrong with that request. Please try again.