ActiveRecord::Base.extend kills JSON performance #9212
Comments
|
The root cause is # Hack to load json gem first so we can overwrite its to_json.
begin
require 'json'
rescue LoadError
end
# The JSON gem adds a few modules to Ruby core classes containing :to_json definition, overwriting
# their default behavior. That said, we need to define the basic to_json method in all of them,
# otherwise they will always use to_json gem implementation, which is backwards incompatible in
# several cases (for instance, the JSON implementation for Hash does not work) with inheritance
# and consequently classes as ActiveSupport::OrderedHash cannot be serialized to json.
[Object, Array, FalseClass, Float, Hash, Integer, NilClass, String, TrueClass].each do |klass|
klass.class_eval do
# Dumps object in JSON (JavaScript Object Notation). See www.json.org for more info.
def to_json(options = nil)
ActiveSupport::JSON.encode(self, options)
end
end
endA slightly rewritten test demonstrates this: #!/usr/bin/env ruby
require 'benchmark'
require 'json'
def profile
blob = File.read(File.join('big_file.json'))
time = Benchmark.realtime do
JSON.parse(JSON.parse(blob).to_json).to_json
end
time * 1000
end
NUM = 10
puts NUM.times.map { profile }.reduce(:+)/NUM.to_f
require 'active_support/json'
require 'active_support/core_ext/object/to_json'
puts NUM.times.map { profile }.reduce(:+)/NUM.to_fIs there any way of preventing |
|
Looks like this code was introduced in 7bd85a8 to fix https://rails.lighthouseapp.com/projects/8994/tickets/4890 (man, pity you couldn't maintain issue numbers between Lighthouse and GitHub). |
|
@josevalim can you comment on this issue? |
|
Cold this possibly be a consequence of extend blowing away the method cache? Hence to_json being called recursively a lot of times incurring expensive costs causing the speed delay. Reloading it possibly recreates the cache (all assumptions) hence the speed up. |
|
I think the slowness we're seeing is merely due to the fact that The intent of the rails code is obvious though; deliberately prevent the json gem from being used. This smells wrong to me, especially since the bug this code was written to fix was so specific. |
It only extends once, the cache should be fine. |
|
Has there been any progress on this? |
|
No progress, I was waiting for @josevalim to comment. In practice we've minimised our use of |
|
That makes sense, I'm using @josevalim 's |
|
As I mentioned in your email, I recently merged a performance-related patch into AMS; I think your issues probably had more to do with our excessive instrumentation and lack of caching. Or at least, that's some of your issue. |
|
I started looking into this issue this morning, here's the problem: diff --git a/activesupport/lib/active_support/core_ext/object/to_json.rb b/activesupport/lib/active_support/core_ext/object/to_json.rb
index 83cc806..336f6ff 100644
--- a/activesupport/lib/active_support/core_ext/object/to_json.rb
+++ b/activesupport/lib/active_support/core_ext/object/to_json.rb
@@ -1,20 +1,6 @@
-# Hack to load json gem first so we can overwrite its to_json.
-begin
- require 'json'
-rescue LoadError
-end
-
-# The JSON gem adds a few modules to Ruby core classes containing :to_json definition, overwriting
-# their default behavior. That said, we need to define the basic to_json method in all of them,
-# otherwise they will always use to_json gem implementation, which is backwards incompatible in
-# several cases (for instance, the JSON implementation for Hash does not work) with inheritance
-# and consequently classes as ActiveSupport::OrderedHash cannot be serialized to json.
-[Object, Array, FalseClass, Float, Hash, Integer, NilClass, String, TrueClass].each do |klass|
- klass.class_eval do
- # Dumps object in JSON (JavaScript Object Notation). See www.json.org for more info.
- def to_json(options = nil)
+class Object
+ def to_json(options = nil)
ActiveSupport::JSON.encode(self, options)
- end
end
end
diff --git a/activesupport/test/json/encoding_test.rb b/activesupport/test/json/encoding_test.rb
index 12ce250..fdfe794 100644
--- a/activesupport/test/json/encoding_test.rb
+++ b/activesupport/test/json/encoding_test.rb
@@ -3,6 +3,8 @@ require 'abstract_unit'
require 'active_support/core_ext/string/inflections'
require 'active_support/json'
+require 'json'
+
class TestJSONEncoding < ActiveSupport::TestCase
class Foo
def initialize(a, b)Yields this: So that will all need to be dealt with somehow in the process of dealing with this. |
|
bump -- @josevalim care to take a look again? this seems like a problem that needs a high-level call |
|
This is still an issue in rails 4.0, as of now. |
|
Wait, so rails doesn't use a C extension for generating JSON at the moment? |
|
@andrewvc yes, that's right. |
|
Yep. I've switched to RABL for my JSON and they've just hardcoded using Oj On Fri, Jul 19, 2013 at 11:39 AM, Jason Hutchens
Christopher Bull 0276308358 |
|
Rails does not use any C extensions at all. It makes Rails easier to install, and easier for platforms like JRuby. |
|
Understood, I was just validating that what I read was right, that at
|
|
The reason |
|
@aq1018 I don't think that's true. I reckon ActionSupport::JSON::Encode is itself really slow. |
|
See this explanation and also #12183. |
|
Here's a working link to @chancancode's explanation: intridea/multi_json#138 (comment) Great stuff! |
|
Are there any before/after benchmarks for @chancancode PR #12183 |
* We use the Oj and multi_json gems, which makes Oj the default JSON parser. However, Rails' ActiveRecord::Base overrides this and uses the native JSON parser, which is slow. In our case we have two render() calls that represent nearly all cases where we ask ActiveRecord to serialize for us. In both cases we already have a hash (not a model object), and we always want JSON responses. So we can fix the performance problem simply by calling Oj.dump() ourselves, and passing the resulting JSON (instead of the hash) to render(). More gory details: * "ActiveRecord::Base.extend kills JSON performance": rails/rails#9212 * "when freedom patches fight, nobody wins": intridea/multi_json#138 (comment)
Due to rails/rails#9212 rendering json caused performance issues and had to be replaced in OBS. We started to use yajl-ruby instead. Supposedly this has been solved in rails meanwhile and we can revert the workaround. See rails/rails#12183 for details Partial revert of 148b7a9
We have a large Rails app, and our JSON performance is abysmal. We've verified JSON.parser is using the C implementation and not the Ruby implementation. We've tried YAJL and OJ to no avail. When we test these things outside of Rails everything's really fast. But running the same tests from the Rails console tells another story.
Today I narrowed this down to authlogic (binarylogic/authlogic#344), but a bit of further tinkering reveals the problem may be in active_record.
Here's my Gemfile:
And here's my test:
On my machine the output of bundle-exec'ing that test is as follows:
Crazypants, right? That little innocuous
Foomodule kills JSON performance. Anyone else seeing this kind of problem? What the freakazoid is going on?The text was updated successfully, but these errors were encountered: