jit: Introduce background compilation #457

qwe661234 · 2024-06-13T05:25:57Z

Given the significant runtime compilation overhead associated with performing aggressive optimizations, we have implemented a background compilation mechanism to mitigate this issue. When the runtime profiler identifies a strong hotspot, it adds a T2C compilation request to the wait queue. A background thread, which continuously monitors this queue, triggers T2C to process the requests and notifies the main thread upon completion by updating a flag.

This mechanism defers the execution of T2C-generated machine code, leading to more frequent use of T1C-generated code. Despite this, the approach effectively minimizes runtime compilation delays by eliminating the need for the main thread to wait for T2C compilation to complete, thereby improving overall performance.

The workflow of background compilation.
The performance comparison with and without the background compialtion.

Close: #239

jserv

Benchmarks

Benchmark suite	Current: `51ce53b`	Previous: `6b462c0`	Ratio
`Dhrystone`	`5.22` Average DMIPS over 10 runs	`4.88` Average DMIPS over 10 runs	`0.93`
`Coremark`	`0.004` Average iterations/sec over 10 runs	`0.004` Average iterations/sec over 10 runs	`1`

This comment was automatically generated by workflow using github-action-benchmark.

src/riscv.c

src/riscv_private.h

src/riscv.c

jserv · 2024-06-13T07:14:17Z

I defer to @vacantron for confirmation.

src/riscv.c

src/feature.h

jserv · 2024-06-18T05:04:30Z

src/riscv.c

+                        (uint64_t) ((memory_t *) PRIV(rv)->mem)->mem_base);
+            free(entry);
+        }
+        /* Instead of writing while(rv->quit), placing the code here prevents


Does volatile help for such purpose?

Yes, it works.

Given the significant runtime compilation overhead associated with performing aggressive optimizations, we have implemented a background compilation mechanism to mitigate this issue. When the runtime profiler identifies a strong hotspot, it adds a T2C compilation request to the wait queue. A background thread, which continuously monitors this queue, triggers T2C to process the requests and notifies the main thread upon completion by updating a flag.

jserv · 2024-06-18T05:30:30Z

Finally, T2C is landed!