Graal JS Script Engine is very slow compared to Rhino

[nmondal]

We have this https://github.com/nmondal/cowj where we use js engines.
We have ran some tests and in summary, when we use Graal Script Engine + GraalVM, it is very, very slow compared to Rhino + Temurin 21 VM.

To reproduce:

1. Clone the cowj repo
2. Switch branch to graal
3. Build the project gradle clean build -x test
4. Go to the app/build/libs folder and
5. Run the command java -jar cowj-0.1-SNAPSHOT.jar ../../samples/hello/hello.yaml true
6. Or, if running in graal - gava -Dpolyglot.js.nashorn-compat=true -jar cowj-0.1-SNAPSHOT.jar ../../samples/hello/hello.yaml true
7. Then run the following wrk --latency -t12 -c400 -d30s http://127.0.0.1:5003/hello/j

We used wrk tool from here ( https://github.com/wg/wrk )
The script file hello.js contains the following:

// here is the return value
// here is a require
//let add = require( "./demo.js")
//_log.info( "10 + 20 is {}", add(10,20) )
//Test.print(req + "\n")
//Test.printe(resp + "\n")
//Test.panic(true,"Boom!")
"hello, world!"

It just returns "hello, world!"

We found out that graal is very slow compared to Rhino.

Attached are the benchmark results.
In summary, the best performance for these two:

GRAAL

wrk git:(master) ./wrk --latency  -t12 -c400 -d30s http://127.0.0.1:5003/hello/j
Running 30s test @ http://127.0.0.1:5003/hello/j
  12 threads and 400 connections
  Thread Stats   Avg      Stdev     Max   +/- Stdev
    Latency   151.87ms  229.33ms   1.41s    91.82%
    Req/Sec   456.94    296.54     1.97k    59.53%
  Latency Distribution
     50%   65.13ms
     75%  177.84ms
     90%  323.05ms
     99%    1.21s 
  132949 requests in 30.10s, 21.55MB read
Requests/sec:   4417.54
Transfer/sec:    733.39KB

Rhino

  wrk git:(master) ./wrk --latency  -t12 -c400 -d30s http://127.0.0.1:5003/hello/j
Running 30s test @ http://127.0.0.1:5003/hello/j
  12 threads and 400 connections
  Thread Stats   Avg      Stdev     Max   +/- Stdev
    Latency     5.74ms    2.64ms 215.97ms   85.20%
    Req/Sec     5.72k   455.54    14.28k    92.53%
  Latency Distribution
     50%    6.39ms
     75%    6.76ms
     90%    7.24ms
     99%    9.06ms
  2050856 requests in 30.05s, 332.49MB read
Requests/sec:  68246.99
Transfer/sec:     11.06MB

This is a HUGE gap, order of magnitude high gap.

Notes

Build was done using temurin open jdk 21

graalvm.perf.txt
rhino.perf.txt

[nmondal]

Please note that this is not about graalvm being slower ( which it is - around 12+% with respect to Temurin 21.).
We also ran another end point /hello/z and the comparison is as follows:

Graal VM

Running 30s test @ http://127.0.0.1:5003/hello/z
  12 threads and 400 connections
  Thread Stats   Avg      Stdev     Max   +/- Stdev
    Latency     7.56ms   35.67ms 889.61ms   97.87%
    Req/Sec    10.06k     1.95k   47.71k    90.00%
  Latency Distribution
     50%    3.61ms
     75%    3.79ms
     90%    4.28ms
     99%  148.73ms
  3585205 requests in 30.09s, 581.25MB read
Requests/sec: 119146.64
Transfer/sec:     19.32MB

Temurin VM

Running 30s test @ http://127.0.0.1:5003/hello/z
  12 threads and 400 connections
  Thread Stats   Avg      Stdev     Max   +/- Stdev
    Latency     3.82ms   13.06ms 394.50ms   99.37%
    Req/Sec    11.04k     1.63k   36.14k    94.03%
  Latency Distribution
     50%    3.34ms
     75%    3.49ms
     90%    3.68ms
     99%    8.55ms
  3939720 requests in 30.10s, 638.73MB read
Requests/sec: 130887.98
Transfer/sec:     21.22MB

Thus the slowness in the fist comment is entirely due to graalvm's way pf handling script engines and loading and running compiled scripts.

[ecki]

Did you test GraalJS with Temurin as well?

[nmondal]

Did you test GraalJS with Temurin as well?

No.. I ran Graal against Graal and.. Rhino in Temurin! Sure I can try running Graal in Temurin.

[ecki]

Did you test GraalJS with Temurin as well?

No.. I ran Graal against Graal and.. Rhino in Temurin! Sure I can try running Graal in Temurin.

Or maybe just change the title to “GraalJS on GraalVm slower than Rhino on Temurin”. I do suspect it is caused be the scaffolding, maybe creating engine and context is the slow part, not the execution. Maybe you can measure that separately? (Or is that what your /z test is saying?)

[nmondal]

Hi, Sorry to be so late.
The reason Graal was slow is because it was fully interpreting the code.
If we stop interpreting - via providing the compile time options -

https://github.com/nmondal/cowj/blob/main/app/graal.sh

In that case, our performance benchmark shows Graal is way, way faster than Rhino. It is probably the fastest engine.

See here:
nmondal/cowj#136 (comment)

• Graal has 3x throughput than Rhino.
• Graal is faster in the long tailed section, in every percentile over 75.
• At 99% Rhino is 20% slower than Graal ( 1020 ms vs 790 ms )

The benchmark is this script:

https://github.com/nmondal/cowj/blob/main/app/samples/js-benchmark/lib/common-js/factorial.js

Thank you, you can close it!