Parsing JSON data takes a long time. How can I reduce parsing time?

[jasperhabicht]

Parsing large amounts of JSON data takes a very long time up to several minutes. Especially when the file needs to compiled multiple times, this leads to time consuming compilation. How can I reduce parsing and compilation time?

[jasperhabicht]

Parsing large or complex data can indeed take a while which is why I track parsing speed with a medium-sized example file (you can see the results in the benchmark folder in the main tree; the file has about 4k6 chars and parsing currently takes about 1 second).

Here are some ideas to improve the parsing speed:

1. Make sure that you update jsonparse to the most recent version. Parsing time was reduced a lot with version 1.4.0 and again a bit with version 1.5.0.
2. You can try to add skip structures as option to the \JSONParse(FromFile) command. So, for example, \JSONParse[skip structures]{\myjson}{ { "foo" : "bar" } } should be considerably faster than \JSONParse{\myjson}{ { "foo" : "bar" } }. But this will only make sense if you don't need to access the full objects or arrays later on in your code.
3. You can try to add validate numbers=false as option to the \JSONParse(FromFile) command. So, for example, \JSONParse[validate numbers=false]{\myjson}{ { "foo" : 1.23 } } should be a tad faster than \JSONParse{\myjson}{ { "foo" : 1.23 } }, but the effect will only be noticeable if the data actually contains a larger amount of numbers.
4. If the JSON data does not change often, you can externalise the parsed data which will reduce compilation time as the data is only parsed once. This is done by adding the externalize key to the \JSONParse(FromFile) command, such as: \JSONParse[externalize]{\myjson}{ { "foo" : "bar" } }.
5. If nothing helps, you may want to use Lua as parser. I did not yet implement it to the package, but you can make use of the JSON parser implemented in Lua provided by ConTeXt.