alibaba/druid pool analysis

[manikantag]

Hi, I find your analysis on Java DB pools very informative. I happen to come across this 'druid' pool from Alibaba: https://github.com/alibaba/druid/wiki/FAQ (claims as the fastest DB pool in Java!).

From my very quick glance, seems it has some cool features. Any thoughts on this. Thanks.

[brettwooldridge]

A quick run of the benchmark on my desktop PC yielded the following:

Benchmark                                 (pool)   Mode  Samples       Score  Units
c.z.h.b.ConnectionBench.cycleCnnection    hikari  thrpt       16   21206.330  ops/ms
c.z.h.b.ConnectionBench.cycleCnnection    bone    thrpt       16   10389.139  ops/ms
c.z.h.b.ConnectionBench.cycleCnnection    vibur   thrpt       16    6764.233  ops/ms
c.z.h.b.ConnectionBench.cycleCnnection    tomcat  thrpt       16    2117.792  ops/ms
c.z.h.b.ConnectionBench.cycleCnnection    c3p0    thrpt       16     128.447  ops/ms
c.z.h.b.ConnectionBench.cycleCnnection    druid   thrpt       16     110.370  ops/ms

c.z.h.b.StatementBench.cycleStatement     hikari  thrpt       16  130426.003  ops/ms
c.z.h.b.StatementBench.cycleStatement     tomcat  thrpt       16   62071.180  ops/ms
c.z.h.b.StatementBench.cycleStatement     druid   thrpt       16   54845.335  ops/ms
c.z.h.b.StatementBench.cycleStatement     vibur   thrpt       16   33414.774  ops/ms
c.z.h.b.StatementBench.cycleStatement     bone    thrpt       16   26551.919  ops/ms
c.z.h.b.StatementBench.cycleStatement     c3p0    thrpt       16   15956.412  ops/ms

So, at least in our benchmark, Druid was the slowest for obtaining and returning connections, and the third fastest for creating and closing statements.

Druid was configured with similar settings to other pools in the benchmark:

druid.setInitialSize(0);
druid.setMinIdle(0);
druid.setMaxActive(32);
druid.setMaxWait(5000);
druid.setValidationQuery("SELECT 1");
druid.setTestOnBorrow(true);
druid.setDefaultAutoCommit(false);
druid.setDefaultTransactionIsolation(Connection.TRANSACTION_READ_COMMITTED);

The benchmark page in their wiki does not show what settings they ran with, but I suspect that they disabled test-on-borrow. While I won't say that is "cheating", it is not how I would run a production pool. They also do not provide the source code to their test, as far as I can tell.

[bobwenx]

druid was designed to focus on monitor and data access behavior enhance(like automatically database slice). it provides an SQL parser to analyze user's SQL query and delegate most JDBC class to collect metrics.
so if what you need is a JDBC monitor solution like newrelic, you may try Druid.

[brettwooldridge]

@bobwenx It is a valid point. I will point out that New Relic is also supported by HikariCP through the DropWizard Metrics integration, but the metrics provided are "pool-level" metrics and not specific to query execution time, etc.

[wenshao]

If you configure maxWait property, druid pool use "fair mode ReentrantLock", it's bad performance.

https://github.com/alibaba/druid/blob/master/src/main/java/com/alibaba/druid/pool/DruidAbstractDataSource.java

    public void setMaxWait(long maxWaitMillis) {
        if (maxWaitMillis == this.maxWait) {
            return;
        }

        if (maxWaitMillis > 0 && useUnfairLock == null && !this.inited) {
            final ReentrantLock lock = this.lock;
            lock.lock();
            try {
                if ((!this.inited) && (!lock.isFair())) {
                    this.lock = new ReentrantLock(true);
                    this.notEmpty = this.lock.newCondition();
                    this.empty = this.lock.newCondition();
                }
            } finally {
                lock.unlock();
            }
        }

        if (inited) {
            LOG.error("maxWait changed : " + this.maxWait + " -> " + maxWaitMillis);
        }

        this.maxWait = maxWaitMillis;
    }

This is by design, because we had some problems in the production environment. Here are the relevant documents, but I'm sorry, it is in Chinese.
https://github.com/alibaba/druid/wiki/Druid%E9%94%81%E7%9A%84%E5%85%AC%E5%B9%B3%E6%A8%A1%E5%BC%8F%E9%97%AE%E9%A2%98

Druid of PreparedStatementCache optimized, this is very important to enhance the mysql 5.5 & oracle & sqlserver & db2 performance.

Druid has very stable ExceptionSorter, including Oracle / MySql / Alibaba Oceanbase etc.

In Taobao large-scale high concurrency environment, only two connection pools to work very well, druid and jboss connection pool.

Druid is not just a connection pool, he can do a very good extension, similar to Filter-Chain extension. Built-in Filter include WallFilter, StatFilter, LogFilter. Which can WallFilter defense SQL injection, StatFilter can do performance monitoring, LogFilter can output SQL logs.

Druid built very strong monitoring support. demo: http://198.11.179.104/druid/index.html

[DR-YangLong]

温少不能忍了，哈哈哈

[hrchu]

Hmmm I only see it claims as "the best" DB pool in Java, instead of the fast... 😕

Anyway I am glad to see more positive comparisons between HikariCP and Druid...

[p0mp0k0]

Does wenshao or brettwooldridge can do the benchmark test according to druid characteristic and brettwooldridge's HikariCP benchmark source code? I think most people want to get a conclusion who is interesting in the performance of connection pool.

[nelly0571]

I choose durid. the best and faster and the filters ....

[ulfsar]

final result was???

[brettwooldridge]

Are you asking HikariCP? Benchmark results were in the first reply.

[JackFish]

Benchmark (jdbcUrl) (maxPoolSize) (pool) Mode Cnt Score Error Units
ConnectionBench.cycleCnnection jdbc:stub 32 druid thrpt 16 3144.688 ± 216.803 ops/ms
ConnectionBench.cycleCnnection jdbc:stub 32 hikari thrpt 16 42262.816 ± 1954.972 ops/ms
ConnectionBench.cycleCnnection jdbc:stub 32 dbcp2 thrpt 16 1988.912 ± 29.686 ops/ms
ConnectionBench.cycleCnnection jdbc:stub 32 tomcat thrpt 16 1849.525 ± 213.140 ops/ms
ConnectionBench.cycleCnnection jdbc:stub 32 c3p0 thrpt 16 642.396 ± 11.501 ops/ms
ConnectionBench.cycleCnnection jdbc:stub 32 vibur thrpt 16 4913.682 ± 400.545 ops/ms
StatementBench.cycleStatement jdbc:stub 32 druid thrpt 16 43187.450 ± 617.061 ops/ms
StatementBench.cycleStatement jdbc:stub 32 hikari thrpt 16 58916.066 ± 4954.865 ops/ms
StatementBench.cycleStatement jdbc:stub 32 dbcp2 thrpt 14 540.265 ± 645.336 ops/ms
StatementBench.cycleStatement jdbc:stub 32 tomcat thrpt 16 78930.496 ± 2561.000 ops/ms
StatementBench.cycleStatement jdbc:stub 32 c3p0 thrpt 16 13162.639 ± 728.747 ops/ms
StatementBench.cycleStatement jdbc:stub 32 vibur thrpt 16 26241.060 ± 1496.388 ops/ms

Seems tomcat got higher performace in StatementBench test
https://github.com/JackFish/HikariCP-benchmark.git

[brettwooldridge]

See the footnotes on the main project page. Tomcat only scores higher because we have to disable the StatementFinalizer, otherwise Tomcat fails to complete the benchmarks with an out of memory error.

To be a fair test the StatementFinalizer should be enabled.

[FutureElement]

druid有阿里大数据量的验证，而HikariCP没有

[brettwooldridge]

What validation?

[FutureElement]

长期的使用，有阿里长期的大数据量的使用的验证

[brettwooldridge]

HikariCP is one of the most widely used connection pools in the world, used by some of the largest companies, serving literally billions of users per day. Druid is rarely seen outside of China.

[FutureElement]

used by some of the largest companies？serving literally billions of users per day？
For example？

[brettwooldridge]

Wix.com by itself, for example hosts over 109 million websites and serves over 1 Billion requests per day. Atlassian has millions of customers for its products -- no published numbers exists for daily usage.

HikariCP is the default pool for every application built with the Play framework, Slick, JOOS, and is now the default pool for Spring Boot.

HikariCP is resolved from the central maven repository over 300,000 times per month.

Companies known to be using HikariCP.

[william8188]

To be honest, I prefer HikariCP 👍

[DR-YangLong]

我认为2个组件专注的方向不一样，各有特色和优点，不是组件如何，要看使用组件的人如何考虑，综合自己的需求选用，druid非常大一部分是为了监控和运维

[kid1412621]

I come here to learn tech instead of the difference of "capitalistic" tool and "communistic" tool.I think it's the performance and application scenario that count. And developers should help users to resolve that.

[pangding1987125]

it's very intersting druid and HikariCP are competing with each other ,and useful for users to use better tools in the future.

[syukoushin]

一个印度小哥引发的口水战？

[lfbayer]

This issue was marked as closed over 3 years ago. GitHub is not the correct place for general discussion. If there are specific other issues, please open a new issue.

[IceMimosa]

😂好无聊的事情。