Looking for feedback on new just-in-time compilation mode

[nickg]

For the last several years NVC has performed all code generation at elaboration time, generating a single native shared library containing all the processes and subprograms used by the design. This gives excellent simulation performance, but at the expense of relatively long elaboration times. This is particularly a problem for modern frameworks like OSVVM or VUnit where elaboration might take several seconds but the test only runs for a fraction of a second.

The latest development version of NVC contains a new elaboration flag --jit which skips ahead-of-time code generation and instead defers code generation until the simulation is running. Initially all processes and subprograms are executed in a bytecode interpreter (the same one that's currently used for evaluating static expressions during elaboration). Once a piece of code has been executed often enough for compilation to be worthwhile, it is compiled to native code in memory on a background thread. By default "often enough" is 100 invocations but you can change that with the NVC_JIT_THRESHOLD environment variable.

For short-running simulations this gives a huge speed-up when measuring the total test time including elaboration. For simulations that run for longer than 30 seconds or so there is probably not much benefit, although the total test time shouldn't be significantly different.

There is currently one limitation that will be removed in the future: this feature only works in combination with the --no-save elaboration option, so you must elaborate and run in the same command. E.g. instead of

$ nvc -e toplevel
$ nvc -r toplevel

Do

$ nvc -e --no-save --jit toplevel -r

I am really keen to get feedback on how well this works for you (or not) since I am planning to make this the default in some future release.

[bpadalino]

This really did save a ton of time when doing the OSVVM quick tests that are part of the test-osvvm.tcl script. I'm pretty happy with the performance increase.

[mariuselv]

Hi Nick.
This sounds like a nice feature. We will incorporate this testing approach not only in our design projects but also in the UVVM development.

Br,
Marius

[avelure]

I tested a bit with the new JIT functionality.
I have about 80 tests in a regression of a medium complex design, with test runtimes ranging from a few seconds to 15min.
I run 10 tests in parallel at the same time, this makes the runtime primarily dominated by the longer tests.
I already was using --no-save, which I guess by it self saves a bit of time.

Overall the runtime is 22m:35s pre and 22m:49s post, plus an additional 5s of analysis time for the whole regression.
Tests shorter than 4m seem to reduce the runtime by 4-8s consistently, while the longer one gains about 10-40s consistently, however some of this gain could be due to the load balancing between the tests.

All in all, not a significant change on my setup, but it still beats GHDL which does 41m:10s with 25s additional analysis time on the same setup.

[nickg]

Thanks for testing. It's a bit disappointing that it slows down your longer tests. That might be because NVC's default behaviour is to create N-1 background compilation threads when system has N cores. But if you are running 10 tests in parallel this might be too many. There's an environment variable NVC_MAX_THREADS that controls this. Could you try setting this to e.g. 4 and see if that has any effect?