Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Update HikariCP and other dependencies #1821

Merged
merged 3 commits into from Mar 26, 2018
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Jump to
Jump to file
Failed to load files.
Diff view
Diff view
2 changes: 1 addition & 1 deletion CONTRIBUTING.md
Expand Up @@ -7,7 +7,7 @@ Slick uses the standard Typesafe Project & Developer Guidelines below for contri
- We have the standard Github [issue tracker](https://github.com/slick/slick/issues?state=open) for bug reports. When you're referencing an issue number in a commit, please prefix it with "issue #".
- Branches starting with "tmp/" or "wip/" on the official repo are considered unstable and should not be used as a basis for your work. Their history may be rewritten at any time without notice.

# Typesafe Project & Developer Guidelines
## Typesafe Project & Developer Guidelines

These guidelines are meant to be a living document that should be changed and adapted as needed. We encourage changes that makes it easier to achieve our goals in an efficient way.

Expand Down
1 change: 0 additions & 1 deletion README.md
Expand Up @@ -26,7 +26,6 @@ These are the databases and driver versions that have explicit automated tests.
|HSQLDB/HyperSQL|hsqldb:2.2.8|[![Build Status](https://travis-ci.org/slick/slick.svg?branch=master)](https://travis-ci.org/slick/slick)|
|H2|com.h2database.h2:1.4.187|[![Build Status](https://travis-ci.org/slick/slick.svg?branch=master)](https://travis-ci.org/slick/slick)|


Accessing other database systems is possible, with a reduced feature set.

The [manual and scaladocs](http://slick.lightbend.com/docs/) for Slick can be
Expand Down
10 changes: 5 additions & 5 deletions project/Build.scala
Expand Up @@ -23,16 +23,16 @@ object SlickBuild extends Build {
"com.novocode" % "junit-interface" % "0.11"
)
def scalaTestFor(scalaVersion: String) = {
val v = "3.0.0"
val v = "3.0.4"
"org.scalatest" %% "scalatest" % v
}
val slf4j = "org.slf4j" % "slf4j-api" % "1.7.25"
val logback = "ch.qos.logback" % "logback-classic" % "1.1.6"
val typesafeConfig = "com.typesafe" % "config" % "1.3.1"
val reactiveStreamsVersion = "1.0.0"
val logback = "ch.qos.logback" % "logback-classic" % "1.2.3"
val typesafeConfig = "com.typesafe" % "config" % "1.3.2"
val reactiveStreamsVersion = "1.0.1"
val reactiveStreams = "org.reactivestreams" % "reactive-streams" % reactiveStreamsVersion
val reactiveStreamsTCK = "org.reactivestreams" % "reactive-streams-tck" % reactiveStreamsVersion
val hikariCP = "com.zaxxer" % "HikariCP" % "2.5.1"
val hikariCP = "com.zaxxer" % "HikariCP" % "2.7.4"
val mainDependencies = Seq(slf4j, typesafeConfig, reactiveStreams)
val h2 = "com.h2database" % "h2" % "1.4.191"
val testDBs = Seq(
Expand Down
Expand Up @@ -24,36 +24,67 @@ object HikariCPJdbcDataSource extends JdbcDataSourceFactory {
def forConfig(c: Config, driver: Driver, name: String, classLoader: ClassLoader): HikariCPJdbcDataSource = {
val hconf = new HikariConfig()

// Connection settings
if (c.hasPath("dataSourceClass")) {
hconf.setDataSourceClassName(c.getString("dataSourceClass"))
} else {
Option(c.getStringOr("driverClassName", c.getStringOr("driver"))).map(hconf.setDriverClassName _)
// Essential settings

// Use HikariCP `dataSourceClassName` as the main configuration and fallback to
// `dataSourceClass`, `driverClassName` and finally `drive`.
c.getStringOpt("dataSourceClassName").orElse(c.getStringOpt("dataSourceClass")) match {
case Some(className) => hconf.setDataSourceClassName(className)
case None => c.getStringOpt("driverClassName").orElse(c.getStringOpt("driver")).foreach(hconf.setDriverClassName)
}
hconf.setJdbcUrl(c.getStringOr("url", null))
c.getStringOpt("user").foreach(hconf.setUsername)

// Use `jdbcUrl` an then `url` to configure the pool. According to HikariCP docs, when
// using this property with "old" drivers, you may also need to set the driverClassName
// property.
c.getStringOpt("jdbcUrl").orElse(c.getStringOpt("url")).foreach(hconf.setJdbcUrl)
c.getStringOpt("username").orElse(c.getStringOpt("user")).foreach(hconf.setUsername)
c.getStringOpt("password").foreach(hconf.setPassword)

c.getPropertiesOpt("properties").foreach(hconf.setDataSourceProperties)

// Pool configuration
// Frequently used pool configuration
c.getBooleanOpt("autoCommit").foreach(hconf.setAutoCommit)

val numThreads = c.getIntOr("numThreads", 20)

hconf.setConnectionTimeout(c.getMillisecondsOr("connectionTimeout", 1000))
Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This is one of the places where Slick is not using HikariCP defaults. While Slick uses 1 second, Hikari defaults to 30 seconds.

Should Slick respect HikariCP default?

hconf.setValidationTimeout(c.getMillisecondsOr("validationTimeout", 1000))
hconf.setIdleTimeout(c.getMillisecondsOr("idleTimeout", 600000))
hconf.setMaxLifetime(c.getMillisecondsOr("maxLifetime", 1800000))
hconf.setLeakDetectionThreshold(c.getMillisecondsOr("leakDetectionThreshold", 0))
hconf.setInitializationFailFast(c.getBooleanOr("initializationFailFast", false))
c.getStringOpt("connectionTestQuery").foreach(hconf.setConnectionTestQuery)
c.getStringOpt("connectionInitSql").foreach(hconf.setConnectionInitSql)
val numThreads = c.getIntOr("numThreads", 20)
hconf.setMaximumPoolSize(c.getIntOr("maxConnections", numThreads))
hconf.setMinimumIdle(c.getIntOr("minConnections", numThreads))
hconf.setMaximumPoolSize(c.getIntOpt("maximumPoolSize").orElse(c.getIntOpt("maxConnections")).getOrElse(numThreads))
hconf.setMinimumIdle(c.getIntOpt("minimumIdle").orElse(c.getIntOpt("minConnections")).getOrElse(numThreads))
hconf.setPoolName(c.getStringOr("poolName", name))
hconf.setRegisterMbeans(c.getBooleanOr("registerMbeans", false))

// Equivalent of ConnectionPreparer
hconf.setReadOnly(c.getBooleanOr("readOnly", false))
c.getStringOpt("isolation").map("TRANSACTION_" + _).foreach(hconf.setTransactionIsolation)
hconf.setCatalog(c.getStringOr("catalog", null))
// Infrequently used

// `initializationFailFast` is deprecated and should be replaced by
// `initializationFailTimeout`. See HikariCP docs for more information:
// https://github.com/brettwooldridge/HikariCP#infrequently-used
// But we still respect the value if it configured.
c.getBooleanOpt("initializationFailFast").foreach(hconf.setInitializationFailFast)
Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Effectively avoids the log warning. Also, initializationFailTimeout overrides it if both are present.


// The default value for `initializationFailFast` was false, which means the pool
// will not fail to start if there is a problem when connecting to the database.
// To keep this behavior, we need to set `initializationFailTimeout` to -1 as
// documented by HikariCP.
hconf.setInitializationFailTimeout(c.getMillisecondsOr("initializationFailTimeout", -1))

c.getBooleanOpt("isolateInternalQueries").foreach(hconf.setIsolateInternalQueries)
c.getBooleanOpt("allowPoolSuspension").foreach(hconf.setAllowPoolSuspension)
c.getBooleanOpt("readOnly").foreach(hconf.setReadOnly)
c.getBooleanOpt("registerMbeans").foreach(hconf.setRegisterMbeans)
c.getStringOpt("catalog").foreach(hconf.setCatalog)
c.getStringOpt("connectionInitSql").foreach(hconf.setConnectionInitSql)
c.getStringOpt("transactionIsolation")
.orElse(c.getStringOpt("isolation"))
.map("TRANSACTION_" + _)
.foreach(hconf.setTransactionIsolation)

hconf.setValidationTimeout(c.getMillisecondsOr("validationTimeout", 1000))
Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This is another place where Slick is not using HikariCP defaults (5 seconds).

hconf.setLeakDetectionThreshold(c.getMillisecondsOr("leakDetectionThreshold", 0))

c.getStringOpt("schema").foreach(hconf.setSchema)

val ds = new HikariDataSource(hconf)
new HikariCPJdbcDataSource(ds, hconf)
Expand Down
136 changes: 87 additions & 49 deletions slick/src/main/scala/slick/jdbc/JdbcBackend.scala
Expand Up @@ -187,54 +187,92 @@ trait JdbcBackend extends RelationalBackend {
*
* The following config keys are supported for HikariCP:
* <ul>
* <li>`url` (String, required): JDBC URL</li>
* <li>`driver` or `driverClassName` (String, optional): JDBC driver class to load</li>
* <li>`user` (String, optional): User name</li>
* <li>`password` (String, optional): Password</li>
* <li>`isolation` (String, optional): Transaction isolation level for new connections.
* Allowed values are: `NONE`, `READ_COMMITTED`, `READ_UNCOMMITTED`, `REPEATABLE_READ`,
* `SERIALIZABLE`.</li>
* <li>`catalog` (String, optional): Default catalog for new connections.</li>
* <li>`readOnly` (Boolean, optional): Read Only flag for new connections.</li>
* <li>`properties` (Map, optional): Properties to pass to the driver or DataSource.</li>
* <li>`dataSourceClass` (String, optional): The name of the DataSource class provided by
* the JDBC driver. This is preferred over using `driver`. Note that `url` is ignored when
* this key is set (You have to use `properties` to configure the database
* connection instead).</li>
* <li>`maxConnections` (Int, optional, default: `numThreads`): The maximum number of
* connections in the pool.</li>
* <li>`minConnections` (Int, optional, default: same as `numThreads`): The minimum number
* of connections to keep in the pool.</li>
* <li>`connectionTimeout` (Duration, optional, default: 1s): The maximum time to wait
* before a call to getConnection is timed out. If this time is exceeded without a
* connection becoming available, a SQLException will be thrown. 1000ms is the minimum
* value.</li>
* <li>`validationTimeout` (Duration, optional, default: 1s): The maximum amount of time
* that a connection will be tested for aliveness. 1000ms is the minimum value.</li>
* <li>`idleTimeout` (Duration, optional, default: 10min): The maximum amount
* of time that a connection is allowed to sit idle in the pool. A value of 0 means that
* idle connections are never removed from the pool.</li>
* <li>`maxLifetime` (Duration, optional, default: 30min): The maximum lifetime of a
* connection in the pool. When an idle connection reaches this timeout, even if recently
* used, it will be retired from the pool. A value of 0 indicates no maximum
* lifetime.</li>
* <li>`connectionInitSql` (String, optional): A SQL statement that will be
* executed after every new connection creation before adding it to the pool. If this SQL
* is not valid or throws an exception, it will be treated as a connection failure and the
* standard retry logic will be followed.</li>
* <li>`initializationFailFast` (Boolean, optional, default: false): Controls whether the
* pool will "fail fast" if the pool cannot be seeded with initial connections
* successfully. If connections cannot be created at pool startup time, a RuntimeException
* will be thrown. This property has no effect if `minConnections` is 0.</li>
* <li>`leakDetectionThreshold` (Duration, optional, default: 0): The amount of time that a
* connection can be out of the pool before a message is logged indicating a possible
* connection leak. A value of 0 means leak detection is disabled. Lowest acceptable value
* for enabling leak detection is 10s.</li>
* <li>`connectionTestQuery` (String, optional): A statement that will be executed just
* before a connection is obtained from the pool to validate that the connection to the
* database is still alive. It is database dependent and should be a query that takes very
* little processing by the database (e.g. "VALUES 1"). When not set, the JDBC4
* `Connection.isValid()` method is used instead (which is usually preferable).</li>
* <li>Essentials:
Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Changes the docs organization and add docs to the HikariCP configurations added. But most of the content is untouched.

* <ul>
* <li>`dataSourceClass` (String, optional): The name of the DataSource class provided by
* the JDBC driver. This is preferred over using `driver`. Note that `url` is ignored when
* this key is set (You have to use `properties` to configure the database
* connection instead).</li>
* <li>`jdbcUrl` or `url` (String, required): JDBC URL</li>
* <li>`username` or `user` (String, optional): User name</li>
* <li>`password` (String, optional): Password</li>
* </ul>
* </li>
* <li>Frequently used:
* <ul>
* <li>`autoCommit` (Boolean, optional, default: true): controls the default auto-commit
* behavior of connections returned from the pool.</li>
* <li>`connectionTimeout` (Duration, optional, default: 1s): The maximum time to wait
* before a call to getConnection is timed out. If this time is exceeded without a
* connection becoming available, a SQLException will be thrown. 1000ms is the minimum
* value.</li>
* <li>`idleTimeout` (Duration, optional, default: 10min): The maximum amount
* of time that a connection is allowed to sit idle in the pool. A value of 0 means that
* idle connections are never removed from the pool.</li>
* <li>`maxLifetime` (Duration, optional, default: 30min): The maximum lifetime of a
* connection in the pool. When an idle connection reaches this timeout, even if recently
* used, it will be retired from the pool. A value of 0 indicates no maximum
* lifetime.</li>
* <li>`connectionTestQuery` (String, optional): A statement that will be executed just
* before a connection is obtained from the pool to validate that the connection to the
* database is still alive. It is database dependent and should be a query that takes very
* little processing by the database (e.g. "VALUES 1"). When not set, the JDBC4
* `Connection.isValid()` method is used instead (which is usually preferable).</li>
* <li>`minimumIdle` or `minConnections` (Int, optional, default: same as `numThreads`): The minimum number
* of connections to keep in the pool.</li>
* <li>`maximumPoolSize` or `maxConnections` (Int, optional, default: `numThreads` * 5): The maximum number of
* connections in the pool.</li>
* </ul>
* </li>
* <li>Infrequently used:
* <ul>
* <li>`initializationFailTimeout` (Long, optional, default: 1): controls whether the pool will
* "fail fast" if the pool cannot be seeded with an initial connection successfully. Any positive
* number is taken to be the number of milliseconds to attempt to acquire an initial connection;
* the application thread will be blocked during this period. If a connection cannot be acquired
* before this timeout occurs, an exception will be thrown. This timeout is applied after the
* connectionTimeout period. If the value is zero (0), HikariCP will attempt to obtain and validate
* a connection. If a connection is obtained, but fails validation, an exception will be thrown and
* the pool not started. However, if a connection cannot be obtained, the pool will start, but later
* efforts to obtain a connection may fail. A value less than zero will bypass any initial connection
* attempt, and the pool will start immediately while trying to obtain connections in the background.
* Consequently, later efforts to obtain a connection may fail.
* <li>DEPRECATED:`initializationFailFast` (Boolean, optional, default: false): Controls whether the
* pool will "fail fast" if the pool cannot be seeded with initial connections
* successfully. If connections cannot be created at pool startup time, a RuntimeException
* will be thrown. This property has no effect if `minConnections` is 0.</li>
* <li>`isolateInternalQueries` (Boolean, optional, default: false): determines whether HikariCP
* isolates internal pool queries, such as the connection alive test, in their own transaction.
* Since these are typically read-only queries, it is rarely necessary to encapsulate them in their
* own transaction. This property only applies if `autoCommit` is disabled.</li>
* <li>`allowPoolSuspension` (Boolean, optional, default: false): controls whether the pool can be
* suspended and resumed through JMX. This is useful for certain failover automation scenarios.
* When the pool is suspended, calls to getConnection() will not timeout and will be held until
* the pool is resumed.</li>
* <li>`readOnly` (Boolean, optional): Read Only flag for new connections.</li>
* <li>`catalog` (String, optional): Default catalog for new connections.</li>
* <li>`connectionInitSql` (String, optional): A SQL statement that will be
* executed after every new connection creation before adding it to the pool. If this SQL
* is not valid or throws an exception, it will be treated as a connection failure and the
* standard retry logic will be followed.</li>
* <li>`driver` or `driverClassName` (String, optional): JDBC driver class to load</li>
* <li>`transactionIsolation` or `isolation` (String, optional): Transaction isolation level for new connections.
* Allowed values are: `NONE`, `READ_COMMITTED`, `READ_UNCOMMITTED`, `REPEATABLE_READ`,
* `SERIALIZABLE`.</li>
* <li>`validationTimeout` (Duration, optional, default: 1s): The maximum amount of time
* that a connection will be tested for aliveness. 1000ms is the minimum value.</li>
* <li>`leakDetectionThreshold` (Duration, optional, default: 0): The amount of time that a
* connection can be out of the pool before a message is logged indicating a possible
* connection leak. A value of 0 means leak detection is disabled. Lowest acceptable value
* for enabling leak detection is 10s.</li>
* <li>`schema` (String, optional): Default catalog for new connections.</li>
* </ul>
* </li>
* <li>Driver or DataSource configuration:
* <ul>
* <li>`properties` (Map, optional): Properties to pass to the driver or DataSource.</li>
* </ul>
* </li>
* </ul>
*
* Direct connections are based on a `java.sql.DataSource` or a `java.sql.Driver`. This is
Expand Down Expand Up @@ -262,7 +300,7 @@ trait JdbcBackend extends RelationalBackend {
* the usual automatic registration process.</li>
* </ul>
*
* The following additional keys are supported for all direct connections:
* The following keys are supported for all direct connections:
* <ul>
* <li>`isolation` (String, optional): Transaction isolation level for new connections.
* Allowed values are: `NONE`, `READ_COMMITTED`, `READ_UNCOMMITTED`, `REPEATABLE_READ`,
Expand Down
2 changes: 1 addition & 1 deletion slick/src/main/scala/slick/jdbc/JdbcDataSource.scala
Expand Up @@ -85,7 +85,7 @@ class DataSourceJdbcDataSource(val ds: DataSource, val keepAliveConnection: Bool
object DataSourceJdbcDataSource extends JdbcDataSourceFactory {
def forConfig(c: Config, driver: Driver, name: String, classLoader: ClassLoader): DataSourceJdbcDataSource = {
val (ds, maxConnections) =
c.getStringOpt("dataSourceClass") match {
c.getStringOpt("dataSourceClassName").orElse(c.getStringOpt("dataSourceClass")) match {

case Some(dsClass) =>
val propsO = c.getPropertiesOpt("properties")
Expand Down
1 change: 1 addition & 0 deletions slick/src/main/scala/slick/util/GlobalConfig.scala
Expand Up @@ -80,6 +80,7 @@ class ConfigExtensionMethods(val c: Config) extends AnyVal {

def getBooleanOpt(path: String): Option[Boolean] = if(c.hasPath(path)) Some(c.getBoolean(path)) else None
def getIntOpt(path: String): Option[Int] = if(c.hasPath(path)) Some(c.getInt(path)) else None
def getLongOpt(path: String): Option[Long] = if(c.hasPath(path)) Some(c.getLong(path)) else None
def getStringOpt(path: String) = Option(getStringOr(path))
def getPropertiesOpt(path: String) = Option(getPropertiesOr(path))
}
Expand Down