Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Use NSDictionary instead of swift types when dealing with JSON #102

Open
cmyr opened this issue Dec 31, 2017 · 11 comments

Comments

@cmyr
Copy link
Member

commented Dec 31, 2017

Some profiling suggests that copying from NSDictionary (which is the type produced by the builtin JSONSerialization class) into swift types like [String: AnyObject] is a significant performance bottleneck.

We've also played around a bit with the new Decodable protocol for handling JSON, and it doesn't have particularly impressive performance yet either; so staying in NSDictionary will be an immediate near-term improvement.

I'll get around to this in the next little bit, but if someone is interested it's a good intro issue.

@raphlinus

This comment has been minimized.

Copy link
Member

commented Jan 6, 2018

After a little more investigation, while this would be a short-term improvement, I'm not sure this specifically is the direction we want to go. The Decodable route is cleaner code and potentially more performant, the only problem is that the implementation in Swift 4 is very slow.

I think the best case here is to write a new JSON parser from scratch that implements Decoder, and also in parallel try to get that upstreamed to Swift. To work in the Swift standard library requires a lot more options (dates, base64, etc) so I think it makes sense to land a simpler version in xi first.

@vlovich

This comment has been minimized.

Copy link
Collaborator

commented Jan 31, 2018

FWIW there's quite a number of available JSON parsing alternatives for Swift that focus on speed.

https://github.com/delba/JASON
https://github.com/bwhiteley/JSONShootout

Maybe using an existing one might be a better attempt than rolling our own?

@raphlinus

This comment has been minimized.

Copy link
Member

commented Jan 31, 2018

I saw those. These are very rough numbers because they're different structs and running on different hardware, but let's put it in terms of MB/s of JSON processed. According to the shootout, "Swift 4" is at about 10, and SwiftyJSON about 5. If JASON is twice as fast as SwiftyJSON, that puts it also at 10.

The tests I did (see irc log for more details) show 0.89MB/s for Swift 4, 1.3 for the current approach, and 5.6 for using raw NSJSONSerialization. There are two reasons these numbers are much lower. One is that my test file has lots of arrays of numbers, where there are usually only 2 bytes per number (so the ratio of JSON objects per byte is high). The other is that my benchmarks were measured for a single iteration on the command line, and I've observed that in a tight loop it's roughly 2x faster. I'm curious why. A reasonable guess is that the allocator "warms up" as it learns the allocation patterns. It's a pretty allocation-heavy workload.

serde benchmarks clock in at 128MB/s. I actually adapted the update benchmark to serde (code not published) and measured 65MB/s. See above for why I'm not surprised.

Lastly, my very rough prototype in Swift, using the Codable interface, gets 10.7MB/s for a single pass and 25.9 for a tight loop. It's not doing things like string escaping, so those numbers will probably change, but I wanted to get a rough estimate. That puts it at roughly 2x the speed of using raw NSJSONSerialization, over 10x the speed of Swift 4 using Codable, and roughly 0.4x the speed of Rust.

Conclusion: I haven't seen evidence of any other performant JSON parser in Swift. Using NS JSON directly would be ok performance but is not beautiful code. Writing our own would be pretty good performance-wise and would make for elegant code, and could benefit the much broader Swift community if upstreamed.

@vlovich

This comment has been minimized.

Copy link
Collaborator

commented Mar 7, 2018

Thinking about this some more, what about just using the Rust JSON library for the parsing & just have a single rust dictionary <> swift dictionary conversion routine? Benefits of having a safe + fast decoder with the hopefully minimal cost of transforming a data struct, especially since all the allocations can be exactly sized & reserved up-front (presumably a transformation wouldn't eat up the 10x speed difference between serde & Swift). For encoding it's probably not as big a deal & we can just continue to use the Swift encoder as-is.

@jansol

This comment has been minimized.

Copy link
Contributor

commented Jul 27, 2018

Issue cleanup bump. What's the status on this?

@cmyr

This comment has been minimized.

Copy link
Member Author

commented Jul 28, 2018

This would likely still be a performance improvement, but doesn't seem particularly pressing.

@mmatoszko

This comment has been minimized.

Copy link
Member

commented Nov 7, 2018

I think it might be worth to just wait with this one, it looks like there is some work done in the String area of Swift:
String’s ABI and UTF-8

@raphlinus

This comment has been minimized.

Copy link
Member

commented Nov 7, 2018

I think you're right. In any case, the original title (use NS types) is clearly not a good way forward, though it would buy us a performance bump in the short term.

@mmatoszko

This comment has been minimized.

Copy link
Member

commented Dec 17, 2018

Here are some benchmarks for the actual state of things:
https://flight.school/articles/benchmarking-codable/

I think we should give Codable a try just around Swift 5 ;)

@jeremywiebe

This comment has been minimized.

Copy link
Contributor

commented Feb 19, 2019

I've done some work to re-test with Swift 5. Here's the code with results in the README. It doesn't look to have improved too much, but for a true comparison it'd probably be best to run all of the benchmarks on the same machine.

https://github.com/jeremywiebe/json-performance

@raphlinus

This comment has been minimized.

Copy link
Member

commented Feb 20, 2019

Since the IRC log linked above is no longer online, here are the links to the code gists: Decodable.swift ObjCLike.swift OldJson.swift update.json

@jeremywiebe jeremywiebe referenced this issue Mar 6, 2019
2 of 6 tasks complete
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
6 participants
You can’t perform that action at this time.