Skip to content

HTTPS clone URL

Subversion checkout URL

You can clone with HTTPS or Subversion.

Download ZIP

Loading…

Eagerload active_support/json/encoding in active_support/core_ext/object/to_json #12203

Merged
merged 1 commit into from

2 participants

@chancancode
Owner

This is related to #12106. (The root cause for that ticket is that json/add defines Regexp#to_json among others, but here I'll reproduce the problem without json/add.)

Before:

>> require 'active_support/core_ext'
=> true
>> //.as_json
NoMethodError: undefined method `as_json' for //:Regexp
  from (irb):3
  from /Users/godfrey/.rvm/rubies/ruby-2.0.0-p195/bin/irb:16:in `<main>'
>> //.to_json
=> "\"(?-mix:)\""
>> //.as_json
=> "(?-mix:)"

After:

>> require 'active_support/core_ext'
=> true
>> //.as_json
=> "(?-mix:)"

When someone require 'active_support/core_ext', the expectation is that it would add certain methods to the core classes NOW. The previous behaviour causes additional methods to be loaded the first time you call to_json, which could cause nasty surprises.

@chancancode
Owner

By the way, I can squash the commits if preferred, but since they are not really related I think it's better to keep the isolated.

@chancancode
Owner

/cc @steveklabnik in case you have any objections

...support/lib/active_support/core_ext/object/to_json.rb
@@ -1,8 +1,8 @@
+# The core extentions for JSON encoding are definied here.
@jeremy Owner
jeremy added a note

typo: defined

@chancancode Owner

Oops, fixed, thanks!

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
@jeremy
Owner
When someone require 'active_support/core_ext', the expectation is that it would
add certain methods to the core classes NOW. The previous behaviour causes
additional methods to be loaded the first time you call to_json, which could cause
nasty surprises.

Any way to test this?

@chancancode
Owner

I was thinking really hard about this and I couldn't come up with any good ideas, any hints? :P

I have some ideas that might work when the test file is run in isolation, but I'm not sure if there are any good ways to test this as part of the suite without relying the order the tests are run/loaded.

@chancancode chancancode Moved all JSON core extensions into core_ext/object/json
TL;DR The primary driver is to remove autoload surprise.

This is related to #12106. (The root cause for that ticket is that
json/add defines Regexp#to_json among others, but here I'll reproduce
the problem without json/add.)

Before:

   >> require 'active_support/core_ext/to_json'
   => true
   >> //.as_json
   NoMethodError: undefined method `as_json' for //:Regexp
     from (irb):3
     from /Users/godfrey/.rvm/rubies/ruby-2.0.0-p195/bin/irb:16:in `<main>'
   >> //.to_json
   => "\"(?-mix:)\""
   >> //.as_json
   => "(?-mix:)"

After:

   >> require 'active_support/core_ext/to_json'
   => true
   >> //.as_json
   => "(?-mix:)"

This is because ActiveSupport::JSON is autoloaded the first time
Object#to_json is called, which causes additional core extentions
(previously defined in active_support/json/encoding.rb) to be loaded.

When someone require 'active_support/core_ext', the expectation is
that it would add certain methods to the core classes NOW. The
previous behaviour causes additional methods to be loaded the first
time you call `to_json`, which could cause nasty surprises and other
unplesant side-effects.

This change moves all core extensions in to core_ext/json. AS::JSON is
still autoloaded on first #to_json call, but since it nolonger
include the core extensions, it should address the aforementioned bug.

*Requiring core_ext/object/to_json now causes a deprecation warnning*
64c88fb
@chancancode
Owner

Sorry @jeremy, I changed my mind again – the final solution I proposed in the inline notes seems like the safest and cleanest solution to fix this, and on second thought the distinction between core_ext/object/to_json and core_ext/object/as_json seemed unnecessary, so I went ahead and moved everything to core_ext/object/json and deprecated core_ext/object/to_json.

With this change, chance of regressing on this same issue seems pretty slim, so I removed the test.

If we want to backport this (I think the bug might be worth fixing in other branches), we can probably just do the same thing minus the deprecation warning. I just didn't feel too good about silencing the circular require warning as I'm unsure of the impact.

This should be ready for merge barring any extra feedback :+1:

@chancancode chancancode referenced this pull request
Merged

JSON encoder refactor #12183

13 of 21 tasks complete
@jeremy jeremy merged commit dae66a0 into from
@chancancode chancancode referenced this pull request from a commit
Commit has since been removed from the repository and is no longer available.
@chancancode chancancode referenced this pull request from a commit in chancancode/rails
@chancancode chancancode Move the JSON extension require statements to the right place.
In #12203, the JSON core extensions were moved into the `core_ext`
folder. Unfortunately, there are some corresponding requires that
were left behind. The problem is partially addressed in #12710, this
commit fixes the rest.
0b7c6d5
@chancancode chancancode referenced this pull request from a commit
@chancancode chancancode Fixed a compatibility issue with the `Oj` gem
`Time#as_json`, `Date#as_json` and `DateTime#as_json` incorrectly depends on a
delegation that is set up in `active_support/json/encoding`. We cannot simply
require that file in `core_ext/object/json` because it would cause a circular
dependency problem (see #12203 for background). We should instead rely on AS's
autoload to load that file for us on-demand.

To trigger autoload correctly, we need to reference the `AS::JSON::Encoding`
constant instead of using the delegated version.

Fixes #16131.
9f489a8
@chancancode chancancode referenced this pull request from a commit
@chancancode chancancode Fixed a compatibility issue with the `Oj` gem
`Time#as_json`, `Date#as_json` and `DateTime#as_json` incorrectly depends on a
delegation that is set up in `active_support/json/encoding`. We cannot simply
require that file in `core_ext/object/json` because it would cause a circular
dependency problem (see #12203 for background). We should instead rely on AS's
autoload to load that file for us on-demand.

To trigger autoload correctly, we need to reference the `AS::JSON::Encoding`
constant instead of using the delegated version.

Fixes #16131.
2ae87ad
@chancancode chancancode referenced this pull request from a commit
@chancancode chancancode Fixed a compatibility issue with the `Oj` gem
`Time#as_json`, `Date#as_json` and `DateTime#as_json` incorrectly depends on a
delegation that is set up in `active_support/json/encoding`. We cannot simply
require that file in `core_ext/object/json` because it would cause a circular
dependency problem (see #12203 for background). We should instead rely on AS's
autoload to load that file for us on-demand.

To trigger autoload correctly, we need to reference the `AS::JSON::Encoding`
constant instead of using the delegated version.

Fixes #16131.
37792d3
@chancancode chancancode referenced this pull request from a commit
@chancancode chancancode Fixed a compatibility issue with the `Oj` gem
`Time#as_json`, `Date#as_json` and `DateTime#as_json` incorrectly depends on a
delegation that is set up in `active_support/json/encoding`. We cannot simply
require that file in `core_ext/object/json` because it would cause a circular
dependency problem (see #12203 for background). We should instead rely on AS's
autoload to load that file for us on-demand.

To trigger autoload correctly, we need to reference the `AS::JSON::Encoding`
constant instead of using the delegated version.

Fixes #16131.
bf7fbe6
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Commits on Sep 13, 2013
  1. @chancancode

    Moved all JSON core extensions into core_ext/object/json

    chancancode authored
    TL;DR The primary driver is to remove autoload surprise.
    
    This is related to #12106. (The root cause for that ticket is that
    json/add defines Regexp#to_json among others, but here I'll reproduce
    the problem without json/add.)
    
    Before:
    
       >> require 'active_support/core_ext/to_json'
       => true
       >> //.as_json
       NoMethodError: undefined method `as_json' for //:Regexp
         from (irb):3
         from /Users/godfrey/.rvm/rubies/ruby-2.0.0-p195/bin/irb:16:in `<main>'
       >> //.to_json
       => "\"(?-mix:)\""
       >> //.as_json
       => "(?-mix:)"
    
    After:
    
       >> require 'active_support/core_ext/to_json'
       => true
       >> //.as_json
       => "(?-mix:)"
    
    This is because ActiveSupport::JSON is autoloaded the first time
    Object#to_json is called, which causes additional core extentions
    (previously defined in active_support/json/encoding.rb) to be loaded.
    
    When someone require 'active_support/core_ext', the expectation is
    that it would add certain methods to the core classes NOW. The
    previous behaviour causes additional methods to be loaded the first
    time you call `to_json`, which could cause nasty surprises and other
    unplesant side-effects.
    
    This change moves all core extensions in to core_ext/json. AS::JSON is
    still autoloaded on first #to_json call, but since it nolonger
    include the core extensions, it should address the aforementioned bug.
    
    *Requiring core_ext/object/to_json now causes a deprecation warnning*
This page is out of date. Refresh to see the latest.
View
2  activesupport/lib/active_support/core_ext/object.rb
@@ -8,7 +8,7 @@
require 'active_support/core_ext/object/conversions'
require 'active_support/core_ext/object/instance_variables'
-require 'active_support/core_ext/object/to_json'
+require 'active_support/core_ext/object/json'
require 'active_support/core_ext/object/to_param'
require 'active_support/core_ext/object/to_query'
require 'active_support/core_ext/object/with_options'
View
216 activesupport/lib/active_support/core_ext/object/json.rb
@@ -0,0 +1,216 @@
+# Hack to load json gem first so we can overwrite its to_json.
+require 'json'
+
+# The JSON gem adds a few modules to Ruby core classes containing :to_json definition, overwriting
+# their default behavior. That said, we need to define the basic to_json method in all of them,
+# otherwise they will always use to_json gem implementation, which is backwards incompatible in
+# several cases (for instance, the JSON implementation for Hash does not work) with inheritance
+# and consequently classes as ActiveSupport::OrderedHash cannot be serialized to json.
+[Object, Array, FalseClass, Float, Hash, Integer, NilClass, String, TrueClass].each do |klass|
+ klass.class_eval do
+ # Dumps object in JSON (JavaScript Object Notation). See www.json.org for more info.
+ def to_json(options = nil)
+ ActiveSupport::JSON.encode(self, options)
+ end
+ end
+end
+
+class Object
+ def as_json(options = nil) #:nodoc:
+ if respond_to?(:to_hash)
+ to_hash
+ else
+ instance_values
+ end
+ end
+end
+
+class Struct #:nodoc:
+ def as_json(options = nil)
+ Hash[members.zip(values)]
+ end
+end
+
+class TrueClass
+ def as_json(options = nil) #:nodoc:
+ self
+ end
+
+ def encode_json(encoder) #:nodoc:
+ to_s
+ end
+end
+
+class FalseClass
+ def as_json(options = nil) #:nodoc:
+ self
+ end
+
+ def encode_json(encoder) #:nodoc:
+ to_s
+ end
+end
+
+class NilClass
+ def as_json(options = nil) #:nodoc:
+ self
+ end
+
+ def encode_json(encoder) #:nodoc:
+ 'null'
+ end
+end
+
+class String
+ def as_json(options = nil) #:nodoc:
+ self
+ end
+
+ def encode_json(encoder) #:nodoc:
+ encoder.escape(self)
+ end
+end
+
+class Symbol
+ def as_json(options = nil) #:nodoc:
+ to_s
+ end
+end
+
+class Numeric
+ def as_json(options = nil) #:nodoc:
+ self
+ end
+
+ def encode_json(encoder) #:nodoc:
+ to_s
+ end
+end
+
+class Float
+ # Encoding Infinity or NaN to JSON should return "null". The default returns
+ # "Infinity" or "NaN" which breaks parsing the JSON. E.g. JSON.parse('[NaN]').
+ def as_json(options = nil) #:nodoc:
+ finite? ? self : nil
+ end
+end
+
+class BigDecimal
+ # A BigDecimal would be naturally represented as a JSON number. Most libraries,
+ # however, parse non-integer JSON numbers directly as floats. Clients using
+ # those libraries would get in general a wrong number and no way to recover
+ # other than manually inspecting the string with the JSON code itself.
+ #
+ # That's why a JSON string is returned. The JSON literal is not numeric, but
+ # if the other end knows by contract that the data is supposed to be a
+ # BigDecimal, it still has the chance to post-process the string and get the
+ # real value.
+ #
+ # Use <tt>ActiveSupport.use_standard_json_big_decimal_format = true</tt> to
+ # override this behavior.
+ def as_json(options = nil) #:nodoc:
+ if finite?
+ ActiveSupport.encode_big_decimal_as_string ? to_s : self
+ else
+ nil
+ end
+ end
+end
+
+class Regexp
+ def as_json(options = nil) #:nodoc:
+ to_s
+ end
+end
+
+module Enumerable
+ def as_json(options = nil) #:nodoc:
+ to_a.as_json(options)
+ end
+end
+
+class Range
+ def as_json(options = nil) #:nodoc:
+ to_s
+ end
+end
+
+class Array
+ def as_json(options = nil) #:nodoc:
+ # use encoder as a proxy to call as_json on all elements, to protect from circular references
+ encoder = options && options[:encoder] || ActiveSupport::JSON::Encoding::Encoder.new(options)
+ map { |v| encoder.as_json(v, options) }
+ end
+
+ def encode_json(encoder) #:nodoc:
+ # we assume here that the encoder has already run as_json on self and the elements, so we run encode_json directly
+ "[#{map { |v| v.encode_json(encoder) } * ','}]"
+ end
+end
+
+class Hash
+ def as_json(options = nil) #:nodoc:
+ # create a subset of the hash by applying :only or :except
+ subset = if options
+ if attrs = options[:only]
+ slice(*Array(attrs))
+ elsif attrs = options[:except]
+ except(*Array(attrs))
+ else
+ self
+ end
+ else
+ self
+ end
+
+ # use encoder as a proxy to call as_json on all values in the subset, to protect from circular references
+ encoder = options && options[:encoder] || ActiveSupport::JSON::Encoding::Encoder.new(options)
+ Hash[subset.map { |k, v| [k.to_s, encoder.as_json(v, options)] }]
+ end
+
+ def encode_json(encoder) #:nodoc:
+ # values are encoded with use_options = false, because we don't want hash representations from ActiveModel to be
+ # processed once again with as_json with options, as this could cause unexpected results (i.e. missing fields);
+
+ # on the other hand, we need to run as_json on the elements, because the model representation may contain fields
+ # like Time/Date in their original (not jsonified) form, etc.
+
+ "{#{map { |k,v| "#{encoder.encode(k.to_s)}:#{encoder.encode(v, false)}" } * ','}}"
+ end
+end
+
+class Time
+ def as_json(options = nil) #:nodoc:
+ if ActiveSupport.use_standard_json_time_format
+ xmlschema
+ else
+ %(#{strftime("%Y/%m/%d %H:%M:%S")} #{formatted_offset(false)})
+ end
+ end
+end
+
+class Date
+ def as_json(options = nil) #:nodoc:
+ if ActiveSupport.use_standard_json_time_format
+ strftime("%Y-%m-%d")
+ else
+ strftime("%Y/%m/%d")
+ end
+ end
+end
+
+class DateTime
+ def as_json(options = nil) #:nodoc:
+ if ActiveSupport.use_standard_json_time_format
+ xmlschema
+ else
+ strftime('%Y/%m/%d %H:%M:%S %z')
+ end
+ end
+end
+
+class Process::Status
+ def as_json(options = nil)
+ { :exitstatus => exitstatus, :pid => pid }
+ end
+end
View
30 activesupport/lib/active_support/core_ext/object/to_json.rb
@@ -1,27 +1,5 @@
-# Hack to load json gem first so we can overwrite its to_json.
-begin
- require 'json'
-rescue LoadError
-end
+ActiveSupport::Deprecation.warn 'You have required `active_support/core_ext/object/to_json`. ' \
+ 'This file will be removed in Rails 4.2. You should require `active_support/core_ext/object/json` ' \
+ 'instead.'
-# The JSON gem adds a few modules to Ruby core classes containing :to_json definition, overwriting
-# their default behavior. That said, we need to define the basic to_json method in all of them,
-# otherwise they will always use to_json gem implementation, which is backwards incompatible in
-# several cases (for instance, the JSON implementation for Hash does not work) with inheritance
-# and consequently classes as ActiveSupport::OrderedHash cannot be serialized to json.
-[Object, Array, FalseClass, Float, Hash, Integer, NilClass, String, TrueClass].each do |klass|
- klass.class_eval do
- # Dumps object in JSON (JavaScript Object Notation). See www.json.org for more info.
- def to_json(options = nil)
- ActiveSupport::JSON.encode(self, options)
- end
- end
-end
-
-module Process
- class Status
- def as_json(options = nil)
- { :exitstatus => exitstatus, :pid => pid }
- end
- end
-end
+require 'active_support/core_ext/object/json'
View
196 activesupport/lib/active_support/json/encoding.rb
@@ -1,6 +1,6 @@
#encoding: us-ascii
-require 'active_support/core_ext/object/to_json'
+require 'active_support/core_ext/object/json'
require 'active_support/core_ext/module/delegation'
require 'bigdecimal'
@@ -148,197 +148,3 @@ def escape(string)
end
end
end
-
-class Object
- def as_json(options = nil) #:nodoc:
- if respond_to?(:to_hash)
- to_hash
- else
- instance_values
- end
- end
-end
-
-class Struct #:nodoc:
- def as_json(options = nil)
- Hash[members.zip(values)]
- end
-end
-
-class TrueClass
- def as_json(options = nil) #:nodoc:
- self
- end
-
- def encode_json(encoder) #:nodoc:
- to_s
- end
-end
-
-class FalseClass
- def as_json(options = nil) #:nodoc:
- self
- end
-
- def encode_json(encoder) #:nodoc:
- to_s
- end
-end
-
-class NilClass
- def as_json(options = nil) #:nodoc:
- self
- end
-
- def encode_json(encoder) #:nodoc:
- 'null'
- end
-end
-
-class String
- def as_json(options = nil) #:nodoc:
- self
- end
-
- def encode_json(encoder) #:nodoc:
- encoder.escape(self)
- end
-end
-
-class Symbol
- def as_json(options = nil) #:nodoc:
- to_s
- end
-end
-
-class Numeric
- def as_json(options = nil) #:nodoc:
- self
- end
-
- def encode_json(encoder) #:nodoc:
- to_s
- end
-end
-
-class Float
- # Encoding Infinity or NaN to JSON should return "null". The default returns
- # "Infinity" or "NaN" which breaks parsing the JSON. E.g. JSON.parse('[NaN]').
- def as_json(options = nil) #:nodoc:
- finite? ? self : nil
- end
-end
-
-class BigDecimal
- # A BigDecimal would be naturally represented as a JSON number. Most libraries,
- # however, parse non-integer JSON numbers directly as floats. Clients using
- # those libraries would get in general a wrong number and no way to recover
- # other than manually inspecting the string with the JSON code itself.
- #
- # That's why a JSON string is returned. The JSON literal is not numeric, but
- # if the other end knows by contract that the data is supposed to be a
- # BigDecimal, it still has the chance to post-process the string and get the
- # real value.
- #
- # Use <tt>ActiveSupport.use_standard_json_big_decimal_format = true</tt> to
- # override this behavior.
- def as_json(options = nil) #:nodoc:
- if finite?
- ActiveSupport.encode_big_decimal_as_string ? to_s : self
- else
- nil
- end
- end
-end
-
-class Regexp
- def as_json(options = nil) #:nodoc:
- to_s
- end
-end
-
-module Enumerable
- def as_json(options = nil) #:nodoc:
- to_a.as_json(options)
- end
-end
-
-class Range
- def as_json(options = nil) #:nodoc:
- to_s
- end
-end
-
-class Array
- def as_json(options = nil) #:nodoc:
- # use encoder as a proxy to call as_json on all elements, to protect from circular references
- encoder = options && options[:encoder] || ActiveSupport::JSON::Encoding::Encoder.new(options)
- map { |v| encoder.as_json(v, options) }
- end
-
- def encode_json(encoder) #:nodoc:
- # we assume here that the encoder has already run as_json on self and the elements, so we run encode_json directly
- "[#{map { |v| v.encode_json(encoder) } * ','}]"
- end
-end
-
-class Hash
- def as_json(options = nil) #:nodoc:
- # create a subset of the hash by applying :only or :except
- subset = if options
- if attrs = options[:only]
- slice(*Array(attrs))
- elsif attrs = options[:except]
- except(*Array(attrs))
- else
- self
- end
- else
- self
- end
-
- # use encoder as a proxy to call as_json on all values in the subset, to protect from circular references
- encoder = options && options[:encoder] || ActiveSupport::JSON::Encoding::Encoder.new(options)
- Hash[subset.map { |k, v| [k.to_s, encoder.as_json(v, options)] }]
- end
-
- def encode_json(encoder) #:nodoc:
- # values are encoded with use_options = false, because we don't want hash representations from ActiveModel to be
- # processed once again with as_json with options, as this could cause unexpected results (i.e. missing fields);
-
- # on the other hand, we need to run as_json on the elements, because the model representation may contain fields
- # like Time/Date in their original (not jsonified) form, etc.
-
- "{#{map { |k,v| "#{encoder.encode(k.to_s)}:#{encoder.encode(v, false)}" } * ','}}"
- end
-end
-
-class Time
- def as_json(options = nil) #:nodoc:
- if ActiveSupport.use_standard_json_time_format
- xmlschema
- else
- %(#{strftime("%Y/%m/%d %H:%M:%S")} #{formatted_offset(false)})
- end
- end
-end
-
-class Date
- def as_json(options = nil) #:nodoc:
- if ActiveSupport.use_standard_json_time_format
- strftime("%Y-%m-%d")
- else
- strftime("%Y/%m/%d")
- end
- end
-end
-
-class DateTime
- def as_json(options = nil) #:nodoc:
- if ActiveSupport.use_standard_json_time_format
- xmlschema
- else
- strftime('%Y/%m/%d %H:%M:%S %z')
- end
- end
-end
View
9 activesupport/test/core_ext/object/json_test.rb
@@ -0,0 +1,9 @@
+require 'abstract_unit'
+
+class JsonTest < ActiveSupport::TestCase
+ # See activesupport/test/json/encoding_test.rb for JSON encoding tests
+
+ def test_deprecated_require_to_json_rb
+ assert_deprecated { require 'active_support/core_ext/object/to_json' }
+ end
+end
View
8 activesupport/test/json/encoding_test.rb
@@ -1,4 +1,5 @@
# encoding: utf-8
+require 'securerandom'
require 'abstract_unit'
require 'active_support/core_ext/string/inflections'
require 'active_support/json'
@@ -96,6 +97,13 @@ def sorted_json(json)
end
end
+ def test_process_status
+ # There doesn't seem to be a good way to get a handle on a Process::Status object without actually
+ # creating a child process, hence this to populate $?
+ system("not_a_real_program_#{SecureRandom.hex}")
+ assert_equal %({"exitstatus":#{$?.exitstatus},"pid":#{$?.pid}}), ActiveSupport::JSON.encode($?)
+ end
+
def test_hash_encoding
assert_equal %({\"a\":\"b\"}), ActiveSupport::JSON.encode(:a => :b)
assert_equal %({\"a\":1}), ActiveSupport::JSON.encode('a' => 1)
View
2  activesupport/test/ordered_hash_test.rb
@@ -1,6 +1,6 @@
require 'abstract_unit'
require 'active_support/json'
-require 'active_support/core_ext/object/to_json'
+require 'active_support/core_ext/object/json'
require 'active_support/core_ext/hash/indifferent_access'
require 'active_support/core_ext/array/extract_options'
View
6 guides/source/active_support_core_extensions.md
@@ -420,11 +420,9 @@ NOTE: Defined in `active_support/core_ext/object/with_options.rb`.
### JSON support
-Active Support provides a better implementation of `to_json` than the +json+ gem ordinarily provides for Ruby objects. This is because some classes, like +Hash+ and +OrderedHash+ needs special handling in order to provide a proper JSON representation.
+Active Support provides a better implementation of `to_json` than the +json+ gem ordinarily provides for Ruby objects. This is because some classes, like +Hash+, +OrderedHash+ and +Process::Status+ needs special handling in order to provide a proper JSON representation.
-Active Support also provides an implementation of `as_json` for the <tt>Process::Status</tt> class.
-
-NOTE: Defined in `active_support/core_ext/object/to_json.rb`.
+NOTE: Defined in `active_support/core_ext/object/json.rb`.
### Instance Variables
Something went wrong with that request. Please try again.