Problem when using oj as JSON library.

[seban]

oj is A fast JSON parser and Object marshaller as a Ruby gem. I tried to used it in my Rails application but looks like pact tests are failing when oj is used as default parser/marshaller.
I have posted an inssue in oj gem already - ohler55/oj#475. Now JSON representation looks ok, but rspec is still failing and it looks like more work than that is required.
I created also sample https://github.com/seban/oj-pact-problem rails (4.2.10) app to reproduce conditions.
Can some one advise me on this one?

[bethesque]

Firstly, thank you for creating a repository for recreating the issue! It makes life so much easier, and it's so rarely done. I really appreciate it.

So, as you've observed, the problem is caused by the fact that OJ serialises Regexp differently to standard JSON.

"(?-mix:application\\/json)"}}}}

I've had this problem before with ActiveSupport, and had to resort to some dodgey dodgey hacks to make it work. I'll see if I can do the same to make it work with OJ.

[seban]

@ohler55 make some branch where he tries to fix it on oj side also. Probably worth checking

[seban]

Any update on this one? Can I help you somehow?

[ohler55]

I think it was resolved. Oj was using the default for Rails instead of for the JSON gem. Now it does both according to the mode.

[bethesque]

Sorry, I got confused and thought the other issue that got closed (#169) was this one. Is it still an issue?

[ohler55]

I guess I jumped the gun. One issue solved but I think there is another one in that Pact::SomethingLike is expected to be encoded as the value of the @content attribute and it is being encoded like any other class of object. Maybe @seban can weigh in on this.

[seban]

I still have a the same problem. Test passes as long as I don't use Pact.like.
I attached the log:
zoo_mock_service.log

[ohler55]

@bethesque can you explain? Is it in the Pact code?

[bethesque]

I'm so sorry, I forgot about this.

The problem is occurring at this line of code, where the incoming JSON should be loaded into Ruby classes.

# lib/pact/mock_service/request_handlers/interaction_post.rb

interaction = Interaction.from_hash(JSON.load(request_body))

The relevant part of the JSON is:

[{"json_class":"Pact::SomethingLike","contents":"gator"}]

Without OJ

A Pact::SomethingLike is correctly made.

I, [2018-07-10T07:27:25.723163 #70020]  INFO -- : [#<Pact::SomethingLike:0x007ffb3ab6e698 @contents="gator">]

With OJ

The hash stays as a hash, and does not get loaded into a Pact::SomethingLike.

I, [2018-07-10T07:42:21.529658 #70049]  INFO -- : [{"json_class"=>"Pact::SomethingLike", "contents"=>"gator"}]>

I'm guessing OJ uses a different technique for recognising and loading embedded Ruby classes in JSON.

[bethesque]

I've narrowed down the problem in this failing test: bethesque/oj-pact-problem@63cfd07

If you run it with OJ_ENABLED=true, it fails.

  1) Pact::SomethingLike JSON.load creates a SomethingLike object from json
     Failure/Error: expect(subject).to eq(SomethingLike.new({"thing" => "blah"}))

       expected: #<Pact::SomethingLike:0x007faf5215ecb8 @contents={"thing"=>"blah"}>
            got: {"json_class"=>"Pact::SomethingLike", "contents"=>{"thing"=>"blah"}}

       (compared using ==)

       Diff:
       @@ -1,2 +1,3 @@
       -#<Pact::SomethingLike:0x007faf5215ecb8 @contents={"thing"=>"blah"}>
       +"contents" => {"thing"=>"blah"},
       +"json_class" => "Pact::SomethingLike",

     # ./spec/models/something_like_spec.rb:23:in `block (3 levels) in <module:Pact>'

[ohler55]

That's helpful. Oj will honor the json_class with the appropriate options set. If Oj.mimic_JSON was called the option should already be set. If not then setting the create_id to json_class should do the trick. Do you know what the options are?

puts Oj.default_options

[bethesque]

{:indent=>0, :second_precision=>9, :circular=>false, :class_cache=>true, :auto_define=>false, :symbol_keys=>false, :bigdecimal_as_decimal=>nil, :use_to_json=>false, :use_to_hash=>false, :use_as_json=>false, :nilnil=>false, :empty_string=>true, :allow_gc=>true, :quirks_mode=>true, :allow_invalid_unicode=>false, :allow_nan=>true, :trace=>false, :float_precision=>16, :mode=>:object, :escape_mode=>:json, :time_format=>:unix, :bigdecimal_load=>:auto, :create_id=>"json_class", :space=>nil, :space_before=>nil, :object_nl=>nil, :array_nl=>nil, :nan=>:auto, :omit_nil=>false, :hash_class=>nil, :array_class=>nil, :ignore=>nil}

[bethesque]

It looks ok to me 🤔

[ohler55]

Oj is in :object mode. That's great for serializing and de-serializing as long as no other components are involved but in this case I think either the :compat mode or the :custom mode would be better. If you want Oj to be like the JSON gem then call Oj.mimic_JSON. I noticed that :use_to_json is also false which is probably not what you want.

[bethesque]

I cannot fix the error with any value of mode that I've tried. eg. Oj.default_options = {:mode => :compat }

[ohler55]

Can you call Oj.mimic_JSON?

[bethesque]

It doesn't fix it. The only thing that makes it work is taking away the Oj.optimize_rails call, and then I don't need to do anything else.

[ohler55]

That is odd although rails and the JSON do fight over encoding. In then end, the setting that Oj.mimic_JSON sets up are probably what you want. There are a few options that are changed in order to mimic the JSON gem. Calling Oj.optimize_rails after that will change some of them and also change the way to_json works so that it is like rails instead of the JSON gem.

[ohler55]

Do you have a test setup I can use to put together a set of options that will do what you want?

[bethesque]

Yes, you can clone this: https://github.com/bethesque/oj-pact-problem

[ohler55]

Okay, thanks. Will look into it tomorrow.

[ohler55]

Adding create_additions: true to the default options fixed the issue but I noticed that does not show up in the default options so that needs to be fixed. The kicker is that in the older JSON gem the create_additions defaulted to true. In the more recent that Oj is mimicking it is set to false. To be on the safe side you should explicitly set that since the default can not be counted on being the same from one version to the next.

[bethesque]

Thanks @ohler55.

As a summary for future users with the same problem, if you're using Pact with OJ and you call Oj.optimize_rails, you also need to call Oj.default_options = { create_additions: true }.

@seban this is a change that I think belongs in your codebase rather than in the Pact code, so I'm going to close this issue now.

[ohler55]

I was nice working with you on this. Thanks.

[bethesque]

Thanks for your help @ohler55

[bheemreddy181-zz]

@bethesque I am still facing the same issue even after adding Oj.default_options = { create_additions: true }

I am using gem 'oj', '~> 3.6', require: false

if I comment out the initializer it works as expected

below is the initializer

require 'oj'
Oj.optimize_rails()
# DecNum#as_json raises SystemStackError, as ActiveSupport sets Numeric#as_json
# to just return self.
require 'flt'
class Flt::DecNum
  def as_json(*)
    to_s
  end
end

Default options on Oj

{:indent=>0, :second_precision=>3, :circular=>false, :class_cache=>true, :auto_define=>false, :symbol_keys=>false, :bigdecimal_as_decimal=>nil, :use_to_json=>false, :use_to_hash=>false, :use_as_json=>false, :nilnil=>false, :empty_string=>true, :allow_gc=>true, :quirks_mode=>true, :allow_invalid_unicode=>false, :allow_nan=>true, :trace=>false, :float_precision=>16, :mode=>:object, :escape_mode=>:json, :time_format=>:unix, :bigdecimal_load=>:auto, :create_id=>"json_class", :space=>nil, :space_before=>nil, :object_nl=>nil, :array_nl=>nil, :nan=>:auto, :omit_nil=>false, :hash_class=>nil, :array_class=>nil, :ignore=>nil}

i don't see after adding create_additions: true adds to default options

[bheemreddy181-zz]

@bethesque you can ignore my comment as per ohler55/oj#532