Why is toJSON slow in jsonlite?

[ramnathv]

I did a quick benchmark of speeds of toJSON conversions across packages. Here is the code

library(microbenchmark)
microbenchmark(
  jsonlite::toJSON(mtcars),
  RJSONIO::toJSON(mtcars),
  rjson::toJSON(mtcars),
  times = 10
)

The results seem to indicate that jsonlite is much slower as compared to the other two packages. Is this because of the more verbose serialization?

Unit: microseconds
                     expr        min         lq      median         uq        max neval
 jsonlite::toJSON(mtcars) 137884.341 142062.506 145352.0815 150982.395 177053.330    10
  RJSONIO::toJSON(mtcars)   1187.170   1280.926   1301.4975   1487.285   2240.849    10
    rjson::toJSON(mtcars)    226.272    230.432    271.4605    275.248    325.430    10

[jeroen]

Great question. Several reasons:

The initial focus in jsonlite is in correctness rather than performance. I have updated the readme a bit to emphasize this. We can tweak performance, but only after we know exactly what it needs to do. In my experience rjson and RJSONIO are very fast at doing something, but you're never sure what's going to happen (slightly exaggerated) :) The package vignette has some more examples.

The behavior of jsonlite can be very different from rjson and RJSONIO, especially for data frames. The fact that jsonlite uses row-based encoding, checks for missing values, etc just makes that more work is involved. I'm sure we can optimize this eventually though. If you really want to compare them you could do:

library(microbenchmark)

microbenchmark(
  jsonlite::toJSON(mtcars, dataframe="columns"),
  RJSONIO::toJSON(mtcars),
  rjson::toJSON(mtcars),
  times = 10
)

Which at least gives you similar output. In this case jsonlite is still slower, but the difference is less extreme. It's pretty much the same reason that lubridate is slower than as.posixct. There are just much more options, and checks being performed to achieve the desired behavior.

[ramnathv]

Thanks @jeroenooms for a detailed answer. It makes sense now.

[jeroen]

I'm working on performance. Could you rerun benchmarks on the latest dev version

library(devtools)
install_github("jsonlite", "jeroenooms")

[ramnathv]

@jeroenooms I ran the benchmarking tests. I can see significant speedup.

For the first test, I get

                     expr      min       lq    median       uq      max neval
 jsonlite::toJSON(mtcars) 6651.857 6787.771 6987.6110 8260.953 8350.356    10
  RJSONIO::toJSON(mtcars) 1166.622 1198.494 1246.8295 1298.283 1543.357    10
    rjson::toJSON(mtcars)  224.388  227.175  243.3265  256.114  279.849    10

For the second one, with dataframe = "columns", I get

Unit: microseconds
                                            expr      min       lq   median       uq      max
 jsonlite::toJSON(mtcars, dataframe = "columns") 4521.350 4593.469 4744.664 5688.685 7502.369
                         RJSONIO::toJSON(mtcars) 1165.328 1185.811 1259.521 1359.632 1546.591
                           rjson::toJSON(mtcars)  226.732  229.432  247.691  260.196  265.969
 neval
    10
    10

[jeroen]

Ok that's getting better. The remaining difference turns out to be because formatC is slow. Let's see if we can fix that too.

[ramnathv]

Yes, it is getting better. I am wondering if it would be possible to exploit rjson, since it already has C code for json conversion.

[jeroen]

Not really, rjson implements a very different mapping between R objects and json. Also most methods in jsonlite are vectorized using base R functions now, so I doubt you'll get a lot of speedup from using C; the main reason why rjson is faster is because it doesn't have any options, and doesn't check for consistency.

[ramnathv]

Makes a lot of sense. Thanks for the answer.

[jeroen]

FYI a new version with the performance enhancements was released on CRAN: http://cran.r-project.org/web/packages/jsonlite/. Thanks for your help. Let me know how the performance is for actual applications, and if you find problems.

I'll continue to fine-tune and optimize the package.