Use NSDictionary instead of swift types when dealing with JSON

[cmyr]

Some profiling suggests that copying from NSDictionary (which is the type produced by the builtin JSONSerialization class) into swift types like [String: AnyObject] is a significant performance bottleneck.

We've also played around a bit with the new Decodable protocol for handling JSON, and it doesn't have particularly impressive performance yet either; so staying in NSDictionary will be an immediate near-term improvement.

I'll get around to this in the next little bit, but if someone is interested it's a good intro issue.

[raphlinus]

After a little more investigation, while this would be a short-term improvement, I'm not sure this specifically is the direction we want to go. The Decodable route is cleaner code and potentially more performant, the only problem is that the implementation in Swift 4 is very slow.

I think the best case here is to write a new JSON parser from scratch that implements Decoder, and also in parallel try to get that upstreamed to Swift. To work in the Swift standard library requires a lot more options (dates, base64, etc) so I think it makes sense to land a simpler version in xi first.

[vlovich]

FWIW there's quite a number of available JSON parsing alternatives for Swift that focus on speed.

https://github.com/delba/JASON
https://github.com/bwhiteley/JSONShootout

Maybe using an existing one might be a better attempt than rolling our own?

[raphlinus]

I saw those. These are very rough numbers because they're different structs and running on different hardware, but let's put it in terms of MB/s of JSON processed. According to the shootout, "Swift 4" is at about 10, and SwiftyJSON about 5. If JASON is twice as fast as SwiftyJSON, that puts it also at 10.

The tests I did (see irc log for more details) show 0.89MB/s for Swift 4, 1.3 for the current approach, and 5.6 for using raw NSJSONSerialization. There are two reasons these numbers are much lower. One is that my test file has lots of arrays of numbers, where there are usually only 2 bytes per number (so the ratio of JSON objects per byte is high). The other is that my benchmarks were measured for a single iteration on the command line, and I've observed that in a tight loop it's roughly 2x faster. I'm curious why. A reasonable guess is that the allocator "warms up" as it learns the allocation patterns. It's a pretty allocation-heavy workload.

serde benchmarks clock in at 128MB/s. I actually adapted the update benchmark to serde (code not published) and measured 65MB/s. See above for why I'm not surprised.

Lastly, my very rough prototype in Swift, using the Codable interface, gets 10.7MB/s for a single pass and 25.9 for a tight loop. It's not doing things like string escaping, so those numbers will probably change, but I wanted to get a rough estimate. That puts it at roughly 2x the speed of using raw NSJSONSerialization, over 10x the speed of Swift 4 using Codable, and roughly 0.4x the speed of Rust.

Conclusion: I haven't seen evidence of any other performant JSON parser in Swift. Using NS JSON directly would be ok performance but is not beautiful code. Writing our own would be pretty good performance-wise and would make for elegant code, and could benefit the much broader Swift community if upstreamed.

[vlovich]

Thinking about this some more, what about just using the Rust JSON library for the parsing & just have a single rust dictionary <> swift dictionary conversion routine? Benefits of having a safe + fast decoder with the hopefully minimal cost of transforming a data struct, especially since all the allocations can be exactly sized & reserved up-front (presumably a transformation wouldn't eat up the 10x speed difference between serde & Swift). For encoding it's probably not as big a deal & we can just continue to use the Swift encoder as-is.

[jansol]

Issue cleanup bump. What's the status on this?

[cmyr]

This would likely still be a performance improvement, but doesn't seem particularly pressing.

[mmatoszko]

I think it might be worth to just wait with this one, it looks like there is some work done in the String area of Swift:
String’s ABI and UTF-8

[raphlinus]

I think you're right. In any case, the original title (use NS types) is clearly not a good way forward, though it would buy us a performance bump in the short term.

[mmatoszko]

Here are some benchmarks for the actual state of things:
https://flight.school/articles/benchmarking-codable/

I think we should give Codable a try just around Swift 5 ;)

[jeremywiebe]

I've done some work to re-test with Swift 5. Here's the code with results in the README. It doesn't look to have improved too much, but for a true comparison it'd probably be best to run all of the benchmarks on the same machine.

https://github.com/jeremywiebe/json-performance

[raphlinus]

Since the IRC log linked above is no longer online, here are the links to the code gists: Decodable.swift ObjCLike.swift OldJson.swift update.json