JSON.parse() performance

[scarlac]

Bug Description

TL;DR: JSON.parse() is 3x slower on Hermes than JSC.

| Engine                  | Method                 | Result |
|-------------------------+------------------------+--------|
| Hermes v0.12.0          | parseJson median       |   50.0 |
| Hermes v0.12.0          | parseJson avg          |   54.5 |
| JSC                     | parseJson median       |   15.0 |
| JSC                     | parseJson avg          |   16.6 |
| NodeJS v16.13.2         | parseJson median       |   15.0 |
| NodeJS v16.13.2         | parseJson avg          |   15.4 |

Porting a React Native app to use Hermes can cause performance regressions depending on use case.
I have narrowed it down to large JSON payloads being loaded. Specifically, JSON.parse() is much slower in Hermes than JSC. JSC is 70% faster than Hermes / Hermes is ~3x slower than JSC. This is significant for our use case and blocks us from migrating.

It's probably fair to say that parsing JSON data from a string is a very common use case. The performance regression (compared to JSC) is severe enough that we cannot use Hermes in production. We have tried 3 times and gotten complaints from users.

• I have run gradle clean and confirmed this bug does not occur with JSC

Hermes version: v0.12.0
React Native version (if any): 0.68.2
OS version (if any): Android 12.0
Platform (most likely one of arm64-v8a, armeabi-v7a, x86, x86_64): arm64-v8a

Steps To Reproduce

1. Call JSON.parse(largePayloadString)
2. Compare with JavaScriptCore and NodeJS

Code Example:

https://github.com/scarlac/benchmark-fs#command-line-test-cli-testjs

The repo contains several performance tests but for the purpose of this ticket, we should focus on the cli-test.js file which can be run with the Hermes CLI tool:
hermes cli-test.js

NodeJS performs similar to JSC. It'd be amazing to have Hermes at that level as well.

The Expected Behavior

Performance should be similar (🤞 or better) than JSC.

[tmikov]

I was able to repro this with the file at the link.

[fbmal7]

Hi @scarlac, just wanted to update you that the team has started an investigation into this issue. We are optimistic that we can bring perf improvements to JSON.parse, to bring it on par to JSC/V8. We will be entering the experimentation phase soon to see what changes help. We will update this thread as we make progress.

I wanted to ask more about the kinds of JSON data you are actually processing. In the benchmark you provided, objects with the same keys & values are being constructed many times. Is that representative of your actual production workload? Do you expect that a lot of the data you are processing will have the same keys or values? If there's any other useful benchmarking tests for JSON parsing you can share, that would be helpful as well.

[scarlac]

@fbmal7 love to hear it. The data we typically parse is more varied and deeper nested but I wanted to make the test case clear and simple to begin with.

in reality we see a lot of variety in the values. Eg GUIDs, long lists of photos with local file paths (formatted as a URI), metadata with numbers and strings, product names, etc.

[billouboq]

@fbmal7 One big use case for this is the usage of redux-persist which save and load data from JS to any database we want. This is done a lot of times even if it's throttled

[fbmal7]

@scarlac We just landed 40de6b5 which did bring a decent perf win. When I ran it locally against the benchmark you provided, it improved the speed by 33%. We still have a long way to go to match the perf of V8/JSC, but this was an easier optimization to land first.

[scarlac]

Thanks @fbmal7 ! Any improvement we can make to JSON handling is a giant win for almost all apps. I can't think of any that don't use JSON heavily. From API responses to persistence layers, to wonky deep object cloning.
Will there be a binary with this I can play around with?

[fbmal7]

We don't tend to cut releases for Hermes very often. Building Hermes from source is always an option, and it's not too complicated. If you just want to build hermes and run it on the command line, you can use these instructions.

[scarlac]

For those wondering, it looks like RN 0.71.0 is currently targeting Hermes tag hermes-2022-11-03-RNv0.71.0-85613e1f9d1216f2cce7e54604be46057092939d which include the 33% improvement(s):
1900bad
40de6b5