Skip to content

HTTPS clone URL

Subversion checkout URL

You can clone with HTTPS or Subversion.

Download ZIP

Loading…

Cannot use cached json value #140

Closed
kookster opened this Issue · 6 comments

3 participants

Andrew Kuklewicz Peter Ohler Shaokun Wu
Andrew Kuklewicz

Hello,

I am working on speeding up our JSON serializing in a Rails app by caching the resulting String of a dump in an instance of CompiledJson.

My purpose is to be able to cache parts of a larger JSON obj as String rather than as a Hash, which will be faster. Imagine you have a parent object aggregating 3 children that are aggregated in its JSON representation. If one of the children is updated, the parent JSON could be recreated without having to re-serialize the other 2 children.

Anyway, here's kind of an example to show how it might work:

require File.expand_path("../config/environment", __FILE__) if ARGV.first == 'rails'
require 'oj'

class CompiledJson
  def initialize(s); @s = s; end
  def to_json(*args); @s; end
  def to_s; @s; end
end

h = {"a" => "1", "b" => { "c" => "2" } }
control = ::Oj.dump(h, mode: :compat)
puts "control:\n#{control}\n"

# cache the "b" child of h
h["b"] = CompiledJson.new(::Oj.dump(h["b"], mode: :compat))
test = ::Oj.dump(h, mode: :compat)
puts "test:\n#{test}\n"

puts (control == test) ? "Success!" : "Fail!"

And this actually works pretty well:

]> bundle exec ruby to_json_test.rb
control:
{"a":"1","b":{"c":"2"}}
test:
{"a":"1","b":{"c":"2"}}
Success!

This is working b/c in :compat mode, dump checks to see that to_json is available for the CompiledJson instance, calls it, and copies the result into out:
https://github.com/ohler55/oj/blob/master/ext/oj/dump.c#L1251-L1268

BUT (there is always a but), if I load my Rails env first, then the oj_rails_hack gets set, and instead of using the to_json method on CompiledJson, it dumps the object attributes (just @s in this case) instead:

]> bundle exec ruby to_json_test.rb rails
control:
{"a":"1","b":{"c":"2"}}
test:
{"a":"1","b":{"s":"{\"c\":\"2\"}"}}
Fail!

Any advice how I can work around this issue?

My current thought is I'll have to fork and add some logic to detect when the object to be dumped has some is_cached_json method and use that even when oj_rails_hack is > 0.

Andrew Kuklewicz

Ugh, the problem is actually that ActiveSupport defines as_json on Object, so that is getting called instead of the to_json. So this works:

class CompiledJson
  def initialize(s); @s = s; end
  def to_json(*args); @s; end
  def to_s; @s; end

  undef_method :as_json

end
Peter Ohler
Owner

The ActiveSupport additions to core objects has been tough to work around. I'm still not happy with what I've done in Oj but have not found a better approach yet.

Is there still an issue I need to look at here?

Andrew Kuklewicz

Thanks, not much to be done here, just my own inability to read code ;)

The only change I feel like is needed is to document that as_json is checked before to_json in the :compat mode description in the README?

It says to_hash now, but that isn't accurate.

https://github.com/ohler55/oj/blob/master/README.md#description

as it really checks as_json

https://github.com/ohler55/oj/blob/master/ext/oj/dump.c#L1157
https://github.com/ohler55/oj/blob/master/ext/oj/dump.c#L1249
https://github.com/ohler55/oj/blob/master/ext/oj/oj.c#L1760

FWIW - I also don't have any ideas for dealing with ActiveSupport/Rails, but they are changing all this in 4.1 anyway, to get rid of MultiJson, and change how ActiveSupport behaves. That means we'll have another be bite at that apple once those are updated, and maybe it will be better/easier/clearer:

http://edgeguides.rubyonrails.org/4_1_release_notes.html#active-support

Peter Ohler
Owner

I will update the docs.

The real problem with the active to_json that it call the JSON serializer with itself that causes a loop since the serialiser is supposed toe call to_json again.

Andrew Kuklewicz

added this pull request with updated README, so this should be closed.
#141

Andrew Kuklewicz kookster closed this
Shaokun Wu

in Oj 2.10.2, you also need to specify this option {use_to_json: true}.

::Oj.dump(h["b"], mode: :compat, use_to_json: true)

# or, specify them in the default_options as below:
Oj.default_options = {:mode => :compat, use_to_json: true}
Shaokun Wu shaokun referenced this issue in rails/jbuilder
Closed

Question: rendering raw JSON #204

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Something went wrong with that request. Please try again.