Skip to content

HTTPS clone URL

Subversion checkout URL

You can clone with HTTPS or Subversion.

Download ZIP

Loading…

Poor encoding performance #75

Closed
dim opened this Issue · 9 comments

2 participants

@dim
dim commented

It's not necessarily an "issue", but oj was created with the goal to be fast. While this is true for decoding, I have seen real performance issues with encoding, compared to other implementations. Please see below:

> Oj::VERSION
=> "2.0.9"
> MultiJson.engine = :oj
=> :oj
> Benchmark.realtime { 1_000_000.times { MultiJson.decode(%({"k1":"val1","k2":"val2","k3":"val3"})) }}
=> 5.100778548
> Benchmark.realtime { 1_000_000.times { MultiJson.encode(k1: "val1", k2: "val2", k3: "val3") }}
=> 11.810473809
> MultiJson.engine = :yajl
=> :yajl
> Benchmark.realtime { 1_000_000.times { MultiJson.decode(%({"k1":"val1","k2":"val2","k3":"val3"})) }}
=> 5.527885379
> Benchmark.realtime { 1_000_000.times { MultiJson.encode(k1: "val1", k2: "val2", k3: "val3") }}
=> 6.986225712
> MultiJson.engine = :json_gem
=> :json_gem
> Benchmark.realtime { 1_000_000.times { MultiJson.decode(%({"k1":"val1","k2":"val2","k3":"val3"})) }}
=> 8.616023528
> Benchmark.realtime { 1_000_000.times { MultiJson.encode(k1: "val1", k2: "val2", k3: "val3") }}
=> 7.957204574
@ohler55
Owner

I'll look into it. The benchmark of Oj on its own show it to be 30% faster than Yajl and 60% faster than the json gem. The dump options might not be set right.

@ohler55
Owner

I ran this test file:

#!/usr/bin/env ruby

encoding: UTF-8

require 'multi_json'
require 'benchmark'
require 'yajl'
require 'json'
require 'oj'

iter = 1_000_000

puts "Benchmarks for different JSON handlers with MultiJson."
puts " Ruby #{RUBY_VERSION}"
puts " #{iter} iterations"

MultiJson.engine = :oj
dt = Benchmark.realtime { iter.times { MultiJson.decode(%({"k1":"val1","k2":"val2","k3":"val3"})) }}
et = Benchmark.realtime { iter.times { MultiJson.encode(k1: "val1", k2: "val2", k3: "val3") }}
puts " Oj decode: #{dt} encode: #{et}"

MultiJson.engine = :yajl
dt = Benchmark.realtime { iter.times { MultiJson.decode(%({"k1":"val1","k2":"val2","k3":"val3"})) }}
et = Benchmark.realtime { iter.times { MultiJson.encode(k1: "val1", k2: "val2", k3: "val3") }}
puts " Yajl decode: #{dt} encode: #{et}"

MultiJson.engine = :json_gem
dt = Benchmark.realtime { iter.times { MultiJson.decode(%({"k1":"val1","k2":"val2","k3":"val3"})) }}
et = Benchmark.realtime { iter.times { MultiJson.encode(k1: "val1", k2: "val2", k3: "val3") }}
puts " Json decode: #{dt} encode: #{et}"

Oj.default_options = { :mode => :compat, :time_format => :ruby }
dt = Benchmark.realtime { iter.times { Oj.load(%({"k1":"val1","k2":"val2","k3":"val3"})) }}
et = Benchmark.realtime { iter.times { Oj.dump(k1: "val1", k2: "val2", k3: "val3") }}

puts "Raw Oj decode: #{dt} encode: #{et}"

The output on a Mac was:

mj.rb
Benchmarks for different JSON handlers with MultiJson.
Ruby 2.0.0
1000000 iterations
Oj decode: 6.568211 encode: 8.292665
Yajl decode: 6.526439 encode: 8.672024
Json decode: 7.075647 encode: 7.62152
Raw Oj decode: 1.973925 encode: 5.629778

Oj is on pare with others for encoding with MultiJson and a bit faster on its own. Way faster on its own for decoding. I ran my own benchmarks on the other Json parsers and it looks like I need to update the numbers. Oj used to be almost twice as fast as Yajl but now Oj is only 30% faster.

What OS and Ruby version were you using?

@dim

Just tried a different, older machine (Laptop, Core i3, Ubuntu 12.10) + added the following lines to your script:

ye = Yajl::Encoder.new
dt = Benchmark.realtime { iter.times { Yajl::Parser.parse(%({"k1":"val1","k2":"val2","k3":"val3"})) }}
et = Benchmark.realtime { iter.times { Yajl::Encoder.encode(k1: "val1", k2: "val2", k3: "val3") }}
e2 = Benchmark.realtime { iter.times { ye.encode(k1: "val1", k2: "val2", k3: "val3") }}
puts "Raw Yajl decode: #{dt} encode: #{et}, encoder: #{e2}"

The results are pretty much the same:

$ uname -a
Linux six 3.5.0-25-generic #39-Ubuntu SMP Mon Feb 25 18:26:58 UTC 2013 x86_64 x86_64 x86_64 GNU/Linux
$ ruby -v
ruby 1.9.3p392 (2013-02-22 revision 39386) [x86_64-linux]
$ ./json_test.rb
Benchmarks for different JSON handlers with MultiJson.
 Ruby 1.9.3
 1000000 iterations
 Oj decode: 9.346176885 encode: 15.319761161
 Yajl decode: 10.082180069 encode: 11.206073485
 Json decode: 15.321923409 encode: 12.522993395
Raw Oj decode: 3.242786396 encode: 10.039278345
Raw Yajl decode: 7.39639453 encode: 7.093391354, encoder: 2.789767341
@ohler55
Owner

I check to see what is going on with Linux. That is a big difference between the Mac and Ubuntu. Give me a few days though.

@ohler55
Owner

I was being a bit too aggressive in preallocating. A more reasonable number really made a difference in speed. Pull and try out the current code. I think you will be happier.

On OS X

mj.rb
Benchmarks for different JSON handlers with MultiJson.
Ruby 2.0.0
1000000 iterations
Oj decode: 6.34794 encode: 3.954895
Yajl decode: 6.308113 encode: 7.589498
Json decode: 8.450363 encode: 6.775135
Raw Oj decode: 1.878968 encode: 0.907779
Raw Yajl decode: 3.875092 encode: 3.949765, encoder: 1.510184

On my Ubuntu

mj.rb
Benchmarks for different JSON handlers with MultiJson.
Ruby 2.0.0
1000000 iterations
Oj decode: 15.90058155 encode: 10.80698237
Yajl decode: 16.34628324 encode: 19.936725807
Json decode: 22.815444217 encode: 23.572341141
Raw Oj decode: 7.863133713 encode: 2.705808481
Raw Yajl decode: 13.897729925 encode: 13.322004708, encoder: 6.085867842

@ohler55
Owner

Given those results I have another tweak I would like to do before releasing.

@ohler55
Owner

The latest, ready for next release.

mj.rb
Benchmarks for different JSON handlers with MultiJson.
Ruby 2.0.0
1000000 iterations
Oj decode: 15.922938767 encode: 9.878166545
Yajl decode: 16.420218433 encode: 19.746729393
Json decode: 22.880924195 encode: 23.384747012
Raw Oj decode: 7.746233074 encode: 1.89902634
Raw Yajl decode: 13.949303248 encode: 13.270999184, encoder: 6.080515983

@ohler55
Owner

Release 2.0.10 just released.

@dim

Confirmed! Great work, much appreciated. Many, many thanks.

 Oj decode: 10.851628285 encode: 6.172948518
 Yajl decode: 11.989925035 encode: 13.175451555
 Json decode: 17.607883599 encode: 14.619682815
Raw Oj decode: 4.458702522 encode: 1.02548957
Raw Yajl decode: 8.800097524 encode: 8.300420872, encoder: 3.30696805
@dim dim closed this
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Something went wrong with that request. Please try again.