Swap JSX for Euneus

[williamthome]

This PR proposes swapping JSX for Euneus for performance and memory reasons.

Motivation

Zotonic largely does JSON parsing and generation, so it's extremely important to be performant.

What is Euneus?

Euneus it's a JSON parser and generator. It is a rewrite of Thoas, but it's more flexible and performant.

Benchmarks

Setup

Operating System: Linux
CPU Information: Intel(R) Core(TM) i7-7700HQ CPU @ 2.80GHz
Number of Available Cores: 8
Available memory: 15.54 GB
Elixir 1.16.0-dev
Erlang 26.1

Benchmark suite executing with the following configuration:
warmup: 5 s
time: 30 s
memory time: 1 s
reduction time: 0 ns
parallel: 1

Results

Encode

##### With input Blockchain #####
Name           ips        average  deviation         median         99th %
euneus      7.92 K      126.30 μs    ±14.02%      118.46 μs      191.87 μs
jsx         2.06 K      484.71 μs    ±14.31%      467.80 μs      813.39 μs

Comparison: 
euneus      7.92 K
jsx         2.06 K - 3.84x slower +358.41 μs

Memory usage statistics:

Name         Memory usage
euneus      90.78 KB
jsx        438.98 KB - 4.84x memory usage +348.20 KB

##### With input GitHub #####
Name           ips        average  deviation         median         99th %
euneus      2.55 K        0.39 ms     ±7.63%        0.39 ms        0.51 ms
jsx         0.55 K        1.80 ms    ±20.10%        1.64 ms        2.88 ms

Comparison: 
euneus      2.55 K
jsx         0.55 K - 4.61x slower +1.41 ms

Memory usage statistics:

Name         Memory usage
euneus       0.24 MB
jsx          1.46 MB - 6.10x memory usage +1.22 MB

##### With input GovTrack #####
Name           ips        average  deviation         median         99th %
euneus       14.70       68.02 ms    ±12.42%       67.34 ms       88.63 ms
jsx           2.88      347.35 ms     ±9.34%      352.89 ms      383.57 ms

Comparison: 
euneus       14.70
jsx           2.88 - 5.11x slower +279.33 ms

Memory usage statistics:

Name         Memory usage
euneus      22.17 MB
jsx        105.88 MB - 4.78x memory usage +83.71 MB

##### With input Issue 90 #####
Name           ips        average  deviation         median         99th %
euneus       27.89       35.86 ms     ±3.28%       35.76 ms       41.76 ms
jsx           8.31      120.33 ms     ±2.16%      119.84 ms      130.93 ms

Comparison: 
euneus       27.89
jsx           8.31 - 3.36x slower +84.47 ms

Memory usage statistics:

Name         Memory usage
euneus       1.33 MB
jsx          4.59 MB - 3.45x memory usage +3.26 MB

Decode

##### With input Blockchain #####
Name           ips        average  deviation         median         99th %
euneus      6.64 K      150.54 μs    ±11.10%      144.54 μs      215.65 μs
jsx         2.56 K      391.16 μs    ±11.90%      374.40 μs      631.59 μs

Comparison: 
euneus      6.64 K
jsx         2.56 K - 2.60x slower +240.62 μs

Memory usage statistics:

Name         Memory usage
euneus      51.78 KB
jsx        354.02 KB - 6.84x memory usage +302.24 KB

##### With input GitHub #####
Name           ips        average  deviation         median         99th %
euneus      2.16 K        0.46 ms     ±7.44%        0.45 ms        0.59 ms
jsx         0.73 K        1.37 ms    ±15.31%        1.29 ms        2.25 ms

Comparison: 
euneus      2.16 K
jsx         0.73 K - 2.96x slower +0.91 ms

Memory usage statistics:

Name         Memory usage
euneus      0.126 MB
jsx          1.28 MB - 10.22x memory usage +1.16 MB

##### With input GovTrack #####
Name           ips        average  deviation         median         99th %
euneus       16.02       62.43 ms     ±2.41%       62.08 ms       71.97 ms
jsx           3.84      260.50 ms     ±1.98%      259.64 ms      278.84 ms

Comparison: 
euneus       16.02
jsx           3.84 - 4.17x slower +198.07 ms

Memory usage statistics:

Name         Memory usage
euneus      11.98 MB
jsx         91.50 MB - 7.64x memory usage +79.53 MB

##### With input Issue 90 #####
Name           ips        average  deviation         median         99th %
euneus       23.97       41.72 ms     ±3.72%       41.64 ms       47.44 ms
jsx           9.88      101.25 ms     ±2.09%      100.77 ms      110.61 ms

Comparison: 
euneus       23.97
jsx           9.88 - 2.43x slower +59.53 ms

Memory usage statistics:

Name         Memory usage
euneus       1.56 MB
jsx          4.29 MB - 2.74x memory usage +2.73 MB

[mmzeeman]

It looks like dialyzer does not see your json parser. Is it in the app.src? In other instances I found the same sort of issue.

[williamthome]

Thanks for the tip @mmzeeman, but there was an issue in Euneus spec.
All is looking fine now.

I will wait for feedback until I clean up the PR removing the duplicated files and the benchmark project.

[mworrell]

This looks like an impressive improvement! Great work.

[williamthome]

The PR is ready to be reviewed.
I have removed jsx completely and reorganized the code that I have changed.
Feel free to do any kind of consideration 🙂

[mworrell]

Maybe add euneus here?

[williamthome]

Euneus does not need to be started as an application. I think it's not necessary to add it there.

[mmzeeman]

A couple of months ago I had to add dependencies there because dialyzer couldn't find types if I didn't. But reading from erlang documentation, the applications entry is indeed intended for applications which must be started before this application.

[williamthome]

@mmzeeman I think I had the same issue with Dialyzer by using cowboy, but adding the plt_extra_apps option as below solved it:

% rebar.config
{dialyzer, [
    {plt_extra_apps, [cowboy]}
]}.

[mmzeeman]

Ah... but cowboy is an application which must be started. It's a bit strange that as a user of an application you have to know how it works internally.

[williamthome]

I don't remember correctly, maybe I have made a mistake, but I was using ErlangLS extension for VSCode and cowboy behaviors were not being recognized.

[mmzeeman]

We had a similar problem. The CI scripts suddenly all failed. I didn't know about the dialyzer config at the time.

Question @mworrell. What about other deps we use which uses jsx. For core zotonic that is probably mqtt_sessions. Should we change the default json encoder there too? There are also a couple of other places where it is used directly.

[mmzeeman]

Ah.. @mworrell here you add the dependency for dialyzer. If it is listed in applications in the app.src file it is automatically picked up.

[mmzeeman]

Excellent... 🎉

This almost means a rename of this project. Maybe it should become json_record at some point.