Netty branch/module for early access Loom compatibility and Loom benchmark update. #12348
Description
A lot has changed since #8439, so I created an new issue for this.
Based on my experiments on loom, I found current reactor mode could benefit a lot from loom.
Under which configuration current Netty performs best with loom?
NOTICE I updated few places in loom so that epoll wait could yield out , but this change could be done outside of loom.
So if you are using the docker I provided, it should shows similar improvements.
git clone https://github.com/joeyleeeeeee97/FrameworkBenchmarks.git
cd FrameworkBenchmarks
./tfb --test netty-optimized --duration 60
Numbers:
- now I am using the
“best”configuration that I know, if you have some suggestions I will be glad to try it out, current benchmark is open and reproducible. - Summary, I didn't observe an obvious improvement from current usage... but similar configuration brings improvement to spring as in spring experiments
What did each column mean?
● netty(no suffix) -> the default platform thread mode
● netty-virtual -> all IO threads and executor threads are converted to virtual threads
● netty-virtualoptimized -> all IO threads and executor threads are converted to virtual threads and enable this patch
● netty-preferoio -> all IO threads and executor threads are converted to virtual threads and use blocking OioChannel
When IO threads are not likely to block:
case "/plaintext":
writePlainResponse(ctx, Unpooled.wrappedBuffer(STATIC_PLAINTEXT));
return;
case "/json":
byte[] json = serializeMsg(newMsg());
writeJsonResponse(ctx, Unpooled.wrappedBuffer(json));
return;
+-------------------------------------------------------------------------+
| Type: plaintext, Result: totalRequests |
+----------+-----------------+----------+---------------+-----------------+
| pipeline | netty-preferoio | netty | netty-virtual | netty-optimized |
+----------+-----------------+----------+---------------+-----------------+
| 4 | 10530667 | 14691760 | 15738128 | 14393872 |
| 8 | 13949919 | 26950480 | 25710512 | 22943440 |
| 16 | 14489186 | 32762208 | 32044880 | 27417568 |
| 32 | 14828997 | 32855888 | 32049648 | 28064912 |
| 256 | 15706426 | 32584896 | 31897920 | 30612608 |
| 1024 | 14663735 | 31866864 | 31305904 | 31894016 |
+----------+-----------------+----------+---------------+-----------------+
+---------------------------------------------------------------------------------+
| Type: json, Result: totalRequests |
+------------------+-----------------+----------+---------------+-----------------+
| concurrencyLevel | netty-preferoio | netty | netty-virtual | netty-optimized |
+------------------+-----------------+----------+---------------+-----------------+
| 4 | 1643654 | 1864622 | 1861855 | 1724663 |
| 8 | 2774330 | 3279037 | 3100116 | 2835602 |
| 16 | 4469149 | 5372280 | 5064919 | 4666069 |
| 32 | 5351567 | 8813844 | 7940714 | 7176427 |
| 64 | 5358308 | 10963798 | 10777658 | 8739648 |
| 128 | 5558293 | 10929623 | 10887536 | 9191461 |
| 256 | 5736596 | 10749314 | 10735618 | 9459791 |
| 512 | 5671719 | 10504008 | 10585946 | 9437229 |
+------------------+-----------------+----------+---------------+-----------------+
When IO threads blocks on executor future
case "/json-async":
byte[] jsonAsync = ASYNC_EXECUTOR.submit(() -> serializeMsg(newMsg())).get();
writeJsonResponse(ctx, Unpooled.wrappedBuffer(jsonAsync));
return;
case "/plaintext-async":
writePlainResponse(ctx, Unpooled.wrappedBuffer(ASYNC_EXECUTOR.submit(() -> STATIC_PLAINTEXT).get()));
return;
+----------------------------------------------------------------------+
| Type: plaintext, Result: totalRequests |
+----------+---------------------+-------------+-----------------------+
| pipeline | netty-async-virtual | netty-async | netty-async-optimized |
+----------+---------------------+-------------+-----------------------+
| 4 | 9820688 | 6248112 | 9218720 |
| 8 | 14545936 | 7950528 | 13054288 |
| 16 | 14807040 | 7988448 | 13954288 |
| 32 | 14943360 | 7971392 | 14451552 |
| 256 | 14944096 | 7875424 | 14703376 |
| 1024 | 14697152 | 7770560 | 14621920 |
+----------+---------------------+-------------+-----------------------+
+------------------------------------------------------------------------------+
| Type: json, Result: totalRequests |
+------------------+---------------------+-------------+-----------------------+
| concurrencyLevel | netty-async-virtual | netty-async | netty-async-optimized |
+------------------+---------------------+-------------+-----------------------+
| 4 | 1759783 | 1658822 | 1598679 |
| 8 | 2795169 | 2576804 | 2622889 |
| 16 | 4655580 | 4113309 | 4328865 |
| 32 | 7117640 | 4661847 | 5979519 |
| 64 | 7440990 | 4641424 | 6612982 |
| 128 | 7414462 | 4581665 | 7008762 |
| 256 | 7401979 | 4553923 | 7234946 |
| 512 | 7421222 | 4530526 | 7406681 |
+------------------+---------------------+-------------+-----------------------+
Question:
- Theoretically, oio(blocking read/write and ThreadPerTaskExecutor) with virtual thread is the best. But benchmark results disagree...
- Also, now I am using the
“best”configuration that I know, if you have some suggestions I will be glad to try it out, current benchmark is open and reproducible.