Make HikariCP loom friendly

[cogman]

Currently, HikariCP uses the synchronized keyword in a few places. Unfortunately, loom may (or may not) support the synchronized keyword pausing a fiber, instead it may end up pausing the fiber carrier thread (it is an open question).

See: https://cr.openjdk.java.net/~rpressler/loom/Loom-Proposal.html

And where those happen: https://github.com/brettwooldridge/HikariCP/search?q=synchronized&unscoped_q=synchronized

Fortunately there is an easy solution here, instead of using the synchronized keyword, use j.u.c locks such as https://docs.oracle.com/javase/7/docs/api/java/util/concurrent/locks/ReentrantLock.html

These APIs will be guaranteed to pause fibers when they are hit.

[Glusk]

IIUC, there will be no need for a thread pool anymore.

In one of the podcasts introducing the technology, Ron Pressler suggests using a combination of a semaphore and virtual threads instead of a connection pool.

---

🤦 I just realised there's a difference between a Thread pool and a Connection pool. I'm sorry for polluting the issue.

[cogman]

@Glusk different problems.

Hikari doesn't manage a thread pool (sort of... not around connections at least). Instead, Hikari is managing network connections or JDBC connections to the database. You'll still need libraries like hikari to ensure connections are distributed quickly.

The two expensive parts of connections are the establishment cost and the overhead they introduce onto the server. Pooling connections puts a hard limit on both. A semaphore would ensure you don't overwhelm the database server with a bunch of connections but wouldn't solve the expense of connection establishment.

Hikari solves both problems.

What loom solves is if you need a bunch of concurrent requests talking to the DB you don't need to manage the number of threads needed to make those requests. Loom keeps the OS thread count down which ultimately helps with context switching times and memory requirements.

In other words, you could have a million threads waiting on 20 database connections and that'd be fine.

[Glusk]

@cogman

A semaphore would ensure you don't overwhelm the database server with a bunch of connections but wouldn't solve the expense of connection establishment.

Because opening a new database connection is a bit more than simply opening a new TCP/IP connection to the server. Fair enough. :)

[cogman]

@Glusk

Pretty much, but also remember that TCP/IP connections also aren't cheap. Absolutely DB connections are more expensive because they usually have some handshake aspect and auth can be somewhat expensive. But even still, spinning up a bunch of TCP connections can be costly.

There's a reason browsers limit the number of Http1.1 connections to ~6 rather than just doing a new connection per resource request. The main innovation of Http2 was multiplexing over a single TCP connection.

I think the only place where a more unbounded access of a constrained resource would make sense is file IO. Particularly with spinning disks, sending a bunch of concurrent file IO requests would allow the OS to better schedule where to put the disk head.

When talking about loom's benefit for an Http server, the main advantage is you cut down the context switching and memory requirements for the server. So servicing 1 million concurrent requests can ultimately process them based on when resources become available without hogging the system. You'd still want to limit the number of concurrently processed requests to avoid blowing out the heap, but you could make that whole process be constrained on the network IO rather than the number of threads that can be reasonably spun up.

[Glusk]

@cogman

Aside:

A server can handle upward of a million concurrent open sockets, yet the operating system cannot efficiently handle more than a few thousand active (non-idle) threads.
Source: State of Loom

[cogman]

Yup, that's all correct.

Socket churn can be a bigger issue, however, than the raw number of opened sockets.

This problem has actually struck my company a couple of times.

http://www.serverframework.com/asynchronousevents/2011/01/time-wait-and-its-design-implications-for-protocols-and-scalable-servers.html

Long story short, high TCP connection churn can exhaust the number of available ports in an OS due to the timed wait state.

So while handling a bunch of open sockets isn't really a problem, creating and destroying a bunch of sockets is.

Further, because of TCP slow start, you pay a time penalty every time a new TCP connection is established (The protocol tries to establish how congested the network to attempt to avoid overwhelming it).

https://blog.stackpath.com/tcp-slow-start/

All this is to say, the most efficient use of hardware and network resources is long lived TCP connections that are reused. There's not necessarily a problem having a million of those connections, but there is a problem if you churn through those connections quickly. Ideally, the protocol you are using is one that multiplexes over a single TCP connection (such as HTTP2). That'll give you the highest throughput with the least amount of resources on both the client and server.

Most database connection protocols get this wrong, IMO, primarily because it's simpler for them to just have a bunch of concurrent connections rather than multiplexing requests over a single connection.

[Blquinn]

Hello, I wanted to revive this thread now that Java 19 has been release with loom in preview.

There is pretty extensive explanations of the implications of using synchronized blocks from virtual threads in https://openjdk.org/jeps/425.

@cogman's original points about synchronized blocks pinning carrier threads and the use of ReentrantLock to alleviate the issue are still accurate in the current state of loom. However it does mention that if synchronized is just being used for short lived operations on a region of memory then it's probably okay. Where it would be really bad is if you did some longer running, blocking operation in a synchronized block.