diff --git a/README.md b/README.md index 45b09ec62..2f937bd10 100644 --- a/README.md +++ b/README.md @@ -49,12 +49,13 @@ Development snapshot are available on sonatype nexus repository ## Documentation For a Getting started guide, API docs, recipes, etc. see the -* [About MariaDB connector/J](/documentation/About-MariaDB-Connector-J.md) -* [Use MariaDB connector/j driver](/documentation/Use-MariaDB-Connector-j-driver.md) -* [Failover and high-availability](/documentation/Failover-and-high-availability.md) +* [About MariaDB connector/J](/documentation/about-mariadb-connector-j.creole) +* [Use MariaDB connector/j driver](/documentation/use-mariadb-connector-j-driver.creole) +* [Changelog](/documentation/changelog.creole) +* [Failover and high-availability](/documentation/failover-and-high-availability-with-mariadb-connector-j.creole) ## Contributing To get started with a development installation and learn more about contributing, please follow the instructions at our -[Developers Guide.](/documentation/Developers-Guide.md) +[Developers Guide.](/documentation/developers-guide.creole) diff --git a/documentation/Use-MariaDB-Connector-j-driver.md b/documentation/Use-MariaDB-Connector-j-driver.md deleted file mode 100644 index d3fe1f1a6..000000000 --- a/documentation/Use-MariaDB-Connector-j-driver.md +++ /dev/null @@ -1,156 +0,0 @@ - -# Using the driver - -The following subsections show the formatting of JDBC connection strings for -MariaDB, MySQL database servers. Additionally, sample code is provided that -demonstrates how to connect to one of these servers and create a table. - - -## Driver Manager - -Applications designed to use the driver manager to locate the entry point need -no further configuration, MariaDB Connector/J will -automatically be loaded and used in the way any previous MySQL driver would -have been. - -## Driver Class - -Please note that the driver class provided by MariaDB Connector/J **is not -`com.mysql.jdbc.Driver` but `org.mariadb.jdbc.Driver`**! - - -## Connection strings - -Format of the JDBC connection string is -```script -jdbc:(mysql|mariadb):[replication:|failover:]//[,...]/[database][?=[&=]] -``` - - HostDescription: -```script -[:] or address=(host=)[(port=)][(type=(master|slave))] -``` - -Host must be a DNS name or IP address. In case of ipv6 and simple host -description, the IP address must be written inside brackets. The default port -is ##3306##. The default type is ##master##. If ##replication## failover is -set, by default the first host is master, and the others are slaves. - -Examples : -* localhost:3306 -* [2001:0660:7401:0200:0000:0000:0edf:bdd7]:3306 -* somehost.com:3306 -* address=(host=localhost)(port=3306)(type=master) - - -### Failover parameters - -Failover was introduced in Connector/J 1.2.0. -See [failover and high availability documentation](./Failover-and-high-availability.md) for more informations. - - -| Failover option | Description| -| ------------ |:----------------| -| **failover** | High availability (random picking connection initialisation) with failover support for master replication cluster (for example Galera).
*Since 1.2.0*| -| **sequential** |Failover support for master replication cluster (for example Galera) **without** High availability.
the host will be connected in the order in which they were declared.

Example when using the jdbc url string "jdbc:mysql:replication:host1,host2,host3/test" :
When connecting, the driver will always try first host1, and if not available host2 and following. After a host fail, the driver will reconnect according to this order.
*Since 1.3.0*| -| **replication** | High availability (random picking connection initialisation) with failover support for master/slaves replication cluster (one or multiple master).
*Since 1.2.0*| -| **aurora** | High availability (random picking connection initialisation) with failover support for Amazon Aurora replication cluster.
*Since 1.2.0*| - - - - -### Optional URL parameters - -General remark: Unknown options accepted and are silently ignored. - -Following options are currently supported. - -| Option | Description| -| ------------ |:----------------| -|user|Database user name.
*since 1.0.0*| -|password|Password of database user.
*since 1.0.0*| -|useFractionalSeconds| Correctly handle subsecond precision in timestamps (feature available with MariaDB 5.3 and later).
May confuse 3rd party components (Hibernated).
*Default: true. Since 1.0.0*| -|allowMultiQueries| Allows multiple statements in single executeQuery.
example:
`insert into ab (i) values (1); insert into ab (i) values (2);`
will be rewritten
`insert into ab (i) values (1), (2);`
*Default: false. Since 1.0.0*| -|useCompression|allow compression in MySQL Protocol.
*Default: false. Since 1.0.0*| -|useSsl|Force SSL on connection.
Alias useSSL works too for mysql compatibility
*Default: false. Since 1.1.0*| -|trustServerCertificate|When using SSL, do not check server's certificate.
*Default: false. Since 1.1.1| -|serverSslCert|Server's certificatem in DER form, or server's CA certificate.
Can be used in one of 3 forms :
* sslServerCert=/path/to/cert.pem (full path to certificate)
* sslServerCert=classpath:relative/cert.pem (relative to current classpath)
* or as verbatim DER-encoded certificate string "------BEGING CERTIFICATE-----" .
*Since 1.1.3*| -|socketFactory| to use custom socket factory, set it to full name of the class that implements javax.net.SocketFactory.
*Since 1.0.0*| -|tcpNoDelay|Sets corresponding option on the connection socket.
*Default: true. Since 1.0.0*| -|tcpKeepAlive|Sets corresponding option on the connection socket.
*Since 1.0.0*| -|tcpAbortiveClose|Sets corresponding option on the connection socket.
*Since 1.1.1*| -|tcpRcvBuf| set buffer size for TCP buffer (SO_RCVBUF).
*Since 1.0.0*| -|tcpSndBuf| set buffer size for TCP buffer (SO_SNDBUF).
*Since 1.0.0*| -|pipe| On Windows, specify named pipe name to connect to mysqld.exe.
*Since 1.1.3*| -|tinyInt1isBit| Datatype mapping flag, handle MySQL Tiny as BIT(boolean).
Default: true.
*Since 1.0.0*| -|yearIsDateType|Year is date type, rather than numerical.

Default: true.
*Since 1.0.0*| -|sessionVariables|= pairs separated by comma, mysql session variables, set upon establishing successfull connection.
*Since 1.1.0*| -|localSocket|Allows to connect to database via Unix domain socket, if server allows it.
The value is the path of Unix domain socket (i.e "socket" database parameter : select @@socket) .
*Since 1.1.4*| -|sharedMemory|Allowed to connect database via shared memory, if server allows it.
The value is base name of the shared memory.
*Since 1.1.4*| -|localSocketAddress|Hostname or IP address to bind the connection socket to a local (UNIX domain) socket.
*Since 1.1.7*| -|socketTimeout|Defined the network socket timeout (SO_TIMEOUT) in milliseconds.
Default: 0 milliseconds(0 disable this timeout). Since 1.1.7*| -|interactiveClient|Session timeout is defined by the wait_timeout server variable. Setting interactiveClient to true will tell server to use the interactive_timeout server variable.
*Default: false. Since 1.1.7*| -|useOldAliasMetadataBehavior|Metadata ResultSetMetaData.getTableName() return the physical table name. "useOldAliasMetadataBehavior" permit to activate the legacy code that send the table alias if set.
*Default: false. Since 1.1.9*| -|createDatabaseIfNotExist|the specified database in url will be created if nonexistent.
Default: false. Since 1.1.7*| -|serverTimezone|Defined the server time zone.
to use only if jre server as a different time implementation of the server.
(best to have the same server time zone when possible).
Since 1.1.7*| -|rewriteBatchedStatements| rewrite batchedStatement to have only one server call.
*Default: false. Since 1.1.8*| -|useServerPrepStmts| if true, preparedStatement will be prepared on server side. If not, Prepared statements (parameter substitution) is handled by the driver, on the client side.
*Default: true. Since 1.3.0*| -|prepStmtCacheSize| if useServerPrepStmts = true, defined the prepared statement cache size.
*Default: 250. Since 1.3.0*| -|prepStmtCacheSqlLimit| if useServerPrepStmts = true, defined queries that size is more than this size will not be cached.
*Default: 2048. Since 1.3.0*| -|connectTimeout| The connect the timeout value, in milliseconds, or zero for no timeout.
*Default: 0. Since 1.1.8*| -|jdbcCompliantTruncation| Truncation error ("Data truncated for column '%' at row %", "Out of range value for column '%' at row %") will be thrown as error, and not as warning.
*Default: true. Since 1.4.0*| -|cacheCallableStmts| enable/disable callable Statement cache
*Default: true. Since 1.4.0*| -|callableStmtCacheSize| This sets the number of callable statements that the driver will cache per VM if "cacheCallableStmts" is enabled.
*Default: true. Since 1.4.0*| - -### Failover/High availability URL parameters - - -| Option | Description| -| ------------ |:----------------| -|autoReconnect|With basic failover: if true, will attempt to recreate connection after a failover.

With standard failover: if true, will attempt to recreate connection even if there is a temporary solution (like using a master connection temporary until reconnect to a slave connection)

Default is false.

since 1.1.7| -|retriesAllDown|When searching a valid host, maximum number of connection attempts before throwing an exception.

Default: 120 seconds.

since 1.2.0| -|failoverLoopRetries|When searching silently for a valid host, maximum number of connection attempts.

This differ from "retriesAllDown" parameter, because this silent search is for example used after a disconnection of a slave connection when using the master connection

Default: 120.

since 1.2.0| -|validConnectionTimeout|With multiple hosts, after this time in seconds has elapsed it’s verified that the connections haven’t been lost.

When 0, no verification will be done.

Default:120 seconds

since 1.2.0| -|loadBalanceBlacklistTimeout|When a connection fails, this host will be blacklisted during the "loadBalanceBlacklistTimeout" amount of time.

When connecting to a host, the driver will try to connect to a host in the list of not blacklisted hosts and after that only on blacklisted ones if none has been found before that.

This blacklist is shared inside the classloader.

Default: 50 seconds.

since 1.2.0| -|assureReadOnly|If true, in high availability, and switching to a read-only host, assure that this host is in read-only mode by setting session read-only.
alias "readOnlyPropagatesToServer" worked to for compatibility
Default to false.
*Default: 50 seconds. Since 1.3.0*| -
- -## JDBC API Implementation Notes -### Streaming result sets -By default, `Statement.executeQuery()` will read full result set -from server before returning. With large result sets, this will require large -amounts of memory. Better behavior in this case would be reading row-by-row, -with `ResultSet.next()`, so called "streaming" feature. It is -activated using `Statement.setFetchSize(Integer.MIN_VALUE)` - - -### CallableStatement -Callable statement implementation won't need to access stored procedure -metadata ([[mysqlproc-table|mysql.proc]]) table if both of following are true - -* CallableStatement.getMetadata() is not used -* Parameters are accessed by index, not by name - -When possible, following the two rules above provides both better speed and -eliminates concerns about SELECT privileges on the -[[mysqlproc-table|mysql.proc]] table. - -### Optional JDBC classes -Following optional interfaces are implemented by the -org.mariadb.jdbc.MariaDbDataSource class : javax.sql.DataSource, -javax.sql.ConnectionPoolDataSource, javax.sql.XADataSource - -## Usage examples - -The following code provides a basic example of how to connect to a MariaDB or -MySQL server and create a table. - -### Creating a table on a MariaDB or MySQL Server -```java -Connection connection = DriverManager.getConnection("jdbc:mysql://localhost:3306/test", "username", "password"); -Statement stmt = connection.createStatement(); -stmt.executeUpdate("CREATE TABLE a (id int not null primary key, value varchar(20))"); -stmt.close(); -connection.close(); -``` - diff --git a/documentation/About-MariaDB-Connector-J.md b/documentation/about-mariadb-connector-j.creole similarity index 64% rename from documentation/About-MariaDB-Connector-J.md rename to documentation/about-mariadb-connector-j.creole index 38aeee8b1..29ed2c803 100644 --- a/documentation/About-MariaDB-Connector-J.md +++ b/documentation/about-mariadb-connector-j.creole @@ -1,17 +1,17 @@ -# About MariaDB java connector += About MariaDB java connector MariaDB Connector/J is used to connect applications developed in Java to MariaDB and MySQL databases using the standard JDBC API. The library is LGPL licensed. -## Introduction +== Introduction MariaDB Connector/J is a Type 4 JDBC driver. It was developed specifically as a lightweight JDBC connector for use with MySQL and MariaDB database servers. It's originally based on the Drizzle JDBC code, and with a lot of additions and bug fixes. -## Obtaining the driver +== Obtaining the driver The driver source code can be downloaded from: https://downloads.mariadb.org/connector-java/ @@ -19,48 +19,53 @@ Pre-built .jar files can be downloaded from: https://code.mariadb.com/connectors/java/ -## Installing the driver +== Installing the driver Installation of the client library is very simple, the jar file should be saved in an appropriate place for your application and the classpath of your application altered to include MariaDB Connector/J rather than your current connector. Using maven : -```script +{{{ org.mariadb.jdbc mariadb-java-client xxx -``` +}}} -## Requirements +== Requirements -* Java 7 or 8 (Last compatible version with java 6 is [1.1.9](https://downloads.mariadb.org/connector-java/1.1.9/)) -* com.sun.JNA is used by some library functions and a jar is available at - https://github.com/twall/jna -** only needed when connecting to the server with unix sockets or windows - shared memory -* A MariaDB or MySQL Server -* maven (only if you want build from source) +* Java 7 or 8 (Last compatible version with java 6 is [[https://downloads.mariadb.org/connector-java/1.1.9/|1.1.9]]) -## Source code +Dependencies (not mandatory): +* [[https://maven-badges.herokuapp.com/maven-central/com.github.dblock.waffle/waffle-jna|waffle-jna 1.8.1]] +* [[https://maven-badges.herokuapp.com/maven-central/net.java.dev.jna/jna|jna 4.2.1]] +* [[https://maven-badges.herokuapp.com/maven-central/net.java.dev.jna/jna-platform|jna-platform 4.2.1]] +* [[https://maven-badges.herokuapp.com/maven-central/org.slf4j/jcl-over-slf4j|jcl-over-slf4j 1.7.14]] +* [[https://maven-badges.herokuapp.com/maven-central/org.slf4j/slf4j-api|slf4j-api 1.7.14]] +* [[https://maven-badges.herokuapp.com/com.google.guava/guava|guava 19.0]] + +jna is needed when when connecting to the server with unix sockets or windows shared memory +All of them are needed for native windows kerberos implementation. (see [[plugin/GSSAPI.creole|Gssapi documentation]]) + +== Source code The source code is available on GitHub: https://github.com/MariaDB/mariadb-connector-j and the most recent development version can be obtained using the following command: -```script +{{{ git clone https://github.com/MariaDB/mariadb-connector-j.git -``` +}}} -## License +== License GNU Lesser General Public License as published by the Free Software Foundation; either version 2.1 of the License, or (at your option) any later version. -## Building and testing the driver +== Building and testing the driver The section deals with building the connector from source and testing it. If you have downloaded a ready built connector, in a jar file, then this section @@ -74,15 +79,17 @@ To run the unit test, you'll need a MariaDB or MySQL server running on localhost (on default TCP port 3306) and a database called 'test', and user 'root' with empty password -```script -git clone https://github.com/MariaDB/mariadb-connector-j.git # Or, unpack the source distribution tarball +{{{ +git clone https://github.com/MariaDB/mariadb-connector-j.git = Or, unpack the source distribution tarball cd mariadb-connector-j -# For the unit test run, start local mysqld mysqld, + +# For the unit test run, start local mysqld mysqld, # ensure that user root with empty password can login mvn package + # If you want to build without running unit tests, use # mvn -Dmaven.test.skip=true package -``` +}}} After that, you should have JDBC jar mariadb-java-client-x.y.z.jar in the 'target' subdirectory diff --git a/documentation/Changelog.md b/documentation/changelog.creole similarity index 80% rename from documentation/Changelog.md rename to documentation/changelog.creole index b67b3c2d8..92cc341ea 100644 --- a/documentation/Changelog.md +++ b/documentation/changelog.creole @@ -1,35 +1,35 @@ -# Changelog -* [1.5.0](#1.5.0) Snapshot available, not released -* [1.4.6](#1.4.6) Released on 13 june 2016 -* [1.4.5](#1.4.5) Released on 18 mai 2016 -* [1.4.4](#1.4.4) Released on 04 mai 2016 -* [1.4.3](#1.4.3) Released on 22 april 2016 -* [1.4.2](#1.4.2) Released on 08 april 2016 -* [1.4.1](#1.4.1) Released on 07 april 2016 -* [1.4.0](#1.4.0) Released on 31 march 2016 += Changelog +* [[#150|1.5.0]] Snapshot available, not released +* [[#146|1.4.6]] Released on 13 june 2016 +* [[#146|1.4.5]] Released on 18 mai 2016 +* [[#144|1.4.4]] Released on 04 mai 2016 +* [[#143|1.4.3]] Released on 22 april 2016 +* [[#142|1.4.2]] Released on 08 april 2016 +* [[#141|1.4.1]] Released on 07 april 2016 +* [[#140|1.4.0]] Released on 31 march 2016 --- -## 1.5.0 +== 1.5.0 * [CONJ-291] Globally performance improvement * [CONJ-296] Support prepare + execute in one call (COM_MULTI protocol) -## 1.4.6 +== 1.4.6 * [CONJ-293] Permit named pipe connection without host * [CONJ-309] Possible NPE on aurora when failover occur during connection initialisation * [CONJ-312] NPE while loading a null from TIMESTAMP field using binary protocol * [misc] batch with one parameter correction (using rewriteBatchedStatements option) -## 1.4.5 +== 1.4.5 * [CONJ-297] Useless memory consumption when using Statement.setQueryTimeout * [CONJ-294] PrepareStatement on master reconnection after a failover * [CONJ-288] using SHOW VARIABLES to replace SELECT on connection to permit connection on a galera non primary node * [CONJ-290] Timestamps format error when using prepareStatement with options useFractionalSeconds and useServerPrepStmts -## 1.4.4 +== 1.4.4 * [CONJ-289] PrepareStatement on master reconnection after a failover * [CONJ-288] using SHOW VARIABLES to replace SELECT on connection to permit connection on a galera non primary node -## 1.4.3 +== 1.4.3 * [CONJ-284] Cannot read autoincremented IDs bigger than Short.MAX_VALUE * [CONJ-283] Parsing correction on MariaDbClientPreparedStatement - syntax error on insert values @@ -37,12 +37,12 @@ * [CONJ-281] Connector/J is incompatible with Google App Engine correction * [CONJ-278] Improve prepared statement on failover -## 1.4.2 +== 1.4.2 * [CONJ-275] Streaming result without result throw "Current position is before the first row" -## 1.4.1 +== 1.4.1 * [CONJ-274] correction to permit connection to MySQL 5.1 server @@ -52,39 +52,39 @@ * [misc] when option rewriteBatchedStatements is set to true, correction of packet separation when query size > max_allow_packet * [misc] performance improvement for select result. -## 1.4.0 +== 1.4.0 -### Complete implementation of fetch size. +=== Complete implementation of fetch size. CONJ-26 JDBC allows to specify the number of rows fetched for a query, and this number is referred to as the fetch size Before version 1.4.0, query were loading all results or row by row using Statement.setFetchSize(Integer.MIN_VALUE). Now it's possible to set fetch size according to your need. -Loading all results for large result sets is using a lot of memory. This functionnality permit to save memory without having performance decrease. +Loading all results for large result sets is using a lot of memory. This functionality permit to save memory without having performance decrease. -### Memory footprint improvement +=== Memory footprint improvement CONJ-125 Buffers have been optimized to reduced memory footprint -### CallableStatement performance improvement. +=== CallableStatement performance improvement. CONJ-209 Calling function / procedure performance is now optimized according to query. Depending on queries, difference can be up to 300%. -### Authentication evolution -CONJ-251 Permit now new authentication possibility : [PAM authentication](https://mariadb.com/kb/en/mariadb/pam-authentication-plugin/), and GSSAPI/SSPI authentication. +=== Authentication evolution +CONJ-251 Permit now new authentication possibility : [[https://mariadb.com/kb/en/mariadb/pam-authentication-plugin/|PAM authentication]], and GSSAPI/SSPI authentication. GSSAPI/SSPI authentication authentication plugin for MariaDB permit a passwordless login. On Unix systems, GSSAPI is usually synonymous with Kerberos authentication. Windows has slightly different but very similar API called SSPI, that along with Kerberos, also supports NTLM authentication. -See more detail in [GSSAPI/SSPI configuration](https://github.com/MariaDB/mariadb-connector-j/blob/master/documentation/plugin/GSSAPI.md) +See more detail in [[https://github.com/MariaDB/mariadb-connector-j/blob/master/documentation/plugin/GSSAPI|GSSAPI/SSPI configuration]] -### Connection attributes +=== Connection attributes CONJ-217 -Driver information informations are now send to [connection attributes tables](https://mariadb.com/kb/en/mariadb/performance-schema-session_connect_attrs-table/) (performance_schema must be activated). +Driver information informations are now send to [[https://mariadb.com/kb/en/mariadb/performance-schema-session_connect_attrs-table/|connection attributes tables]] (performance_schema must be activated). A new option "connectionAttributes" permit to add client specifics data. For example when connecting with the following connection string {{{"jdbc:mysql://localhost:3306/testj?user=root&connectionAttributes=myOption:1,mySecondOption:'jj'"}}}, if performance_schema is activated, information about this connection will be available during the time this connection is active : -``` java +{{{ select * from performance_schema.session_connect_attrs where processList_id = 5 +----------------+-----------------+---------------------+------------------+ | PROCESSLIST_ID | ATTR_NAME | ATTR_VALUE | ORDINAL_POSITION | @@ -99,15 +99,15 @@ select * from performance_schema.session_connect_attrs where processList_id = 5 |5 |myOption |1 |7 | |5 |mySecondOption |'jj' |8 | +----------------+-----------------+---------------------+------------------+ -``` +}}} -## Minor evolution +== Minor evolution * CONJ-210 : adding a "jdbcCompliantTruncation" option to force truncation warning as SQLException. * CONJ-211 : when in master/slave configuration, option "assureReadOnly" will ensure that slaves are in read-only mode ( forcing transaction by a query "SET SESSION TRANSACTION READ ONLY"). * CONJ-213 : new option "continueBatchOnError". Permit to continue batch when an exception occur : When executing a batch and an error occur, must the batch stop immediatly (default) or finish remaining batch before throwing exception. -## Bugfix +== Bugfix * CONJ-236 : Using a parametrized query with a smallint -1 does return the unsigned value * CONJ-250 : Tomcat doesn't stop when using Aurora failover configuration * CONJ-260 : Add jdbc nString, nCharacterStream, nClob implementation diff --git a/documentation/Developers-Guide.md b/documentation/developers-guide.creole similarity index 68% rename from documentation/Developers-Guide.md rename to documentation/developers-guide.creole index 7800386fc..44465b644 100644 --- a/documentation/Developers-Guide.md +++ b/documentation/developers-guide.creole @@ -1,51 +1,60 @@ -> **This guide will teach you:** -> * How to install a local version of connector/J -> * How to run tests locally and on travis CI -> * How to submit a request +//This guide will teach you:// -# Contributing + * How to install a local version of connector/J + * How to run tests locally and on travis CI + * How to submit a request + += Contributing Each pull request should address a single issue, and contain both the fix as well as a description of how the pull request and tests that validate that the PR fixes the issue in question. -For significant feature additions, we like to have an open issue in [MariaDB JIRA](https://mariadb.atlassian.net/secure/RapidBoard.jspa?projectKey=CONJ). It is expected that discussion will have taken place in the attached issue. +For significant feature additions, we like to have an open issue in [[https://mariadb.atlassian.net/secure/RapidBoard.jspa?projectKey=CONJ|MariaDB JIRA]]. It is expected that discussion will have taken place in the attached issue. -# Install Prerequisites += Install Prerequisites These are the set of tools which are required in order to complete any build. Follow the links to download and install them on your own before continuing. -1. Oracle JDK 8 (http://www.oracle.com/technetwork/java/javase/downloads/index.html) -2. IDE (eclipse / netbean / intelliJ) with maven and GIT plugins +* [[http://www.oracle.com/technetwork/java/javase/downloads/index.html|Oracle JDK 8]] ( with [[http://www.oracle.com/technetwork/java/javase/downloads/jce8-download-2133166.html|JCE policies]] if using TLS/SSL) +* IDE (eclipse / netbean / intelliJ) with maven and GIT plugins -# Fork source += Fork source Before downloading source, fork the project to your own repository, and use your repository as source. -# Run local test += Run local test Before any submission : Run the test locally : by default, you need to have a MySQL/MariaDB server on localhost:3306 with a database named "testj" and a user root without password. so you can run +{{{ mvn test +}}} You can change those parameter by adding -DdbUrl parameter. like : +{{{ mvn test -DdbUrl=jdbc:mariadb://127.0.0.1:3306/testj?user=root&password=***** +}}} You can launch a specific test by adding -Dtest +{{{ mvn test -Dtest=org.mariadb.jdbc.JdbcParserTest +}}} When all test are passing, you can package project. Additional tests , like javadoc formatting, code style validation will be done : +{{{ mvn package -Dmaven.test.skip=true +}}} If operation succeed, a new mariadb-java-client jar will be on the target folder. -# Run travis test += Run travis test You can activate travis to validate your repository. The advantage of travis compare to running test locally is that it will launch tests for a combination of those parameters : @@ -53,12 +62,12 @@ The advantage of travis compare to running test locally is that it will launch t jdk: * oraclejdk8 * oraclejdk7 -* openjdk7 server : * MariaDB 5.5 * MariaDB 10.0 * MariaDB 10.1 +* MariaDB 10.2 * MySQL 5.6 * MySQL 5.7 @@ -67,14 +76,14 @@ max_allowed_packet : * 16M * 32M -For that, you have to go on [travis website](https://travis-ci.org), connect with your github account, and activate your mariadb-connector-j repository. +For that, you have to go on [[https://travis-ci.org|travis website]], connect with your github account, and activate your mariadb-connector-j repository. After this step, every push to your repository will launch a travis test. -## Submitting a request +== Submitting a request When your repository has the correction/change done, you can submit a pull request by clicking the "Pull request" button on github. Please detail the operation done in your request. -## License +== License Distributed under the terms of the GNU Library or "Lesser" General Public License (LGPL). diff --git a/documentation/Failover-and-high-availability.md b/documentation/failover-and-high-availability-with-mariadb-connector-j.creole similarity index 71% rename from documentation/Failover-and-high-availability.md rename to documentation/failover-and-high-availability-with-mariadb-connector-j.creole index 4656a58b3..e2a11b8d7 100644 --- a/documentation/Failover-and-high-availability.md +++ b/documentation/failover-and-high-availability-with-mariadb-connector-j.creole @@ -1,18 +1,19 @@ -> **This guide will teach you:** -> * The load balancing and high availability concepts in Mariadb java connector -> * the different options +//**This guide will teach you:**// + + * The load balancing and high availability concepts in Mariadb java connector + * the different options Failover and high availability were introduced in 1.2.0. -# Load balancing and failover distinction -Failover occurs when a connection to a primary database server fails and the connector will open up a connection to another database server.
+= Load balancing and failover distinction +Failover occurs when a connection to a primary database server fails and the connector will open up a connection to another database server.\\ For example, server A has the current connection. After a failure (server crash, network down …) the connection will switch to another server (B). Load balancing allows load (read and write) to be distributed over multiple servers. -
-# Replication cluster type +\\ += Replication cluster type In MariaDB (and MySQL) replication, there are 2 different replication roles: * Master role: Database server that permits read and write operations * Slave role: Database server that permits only read operations @@ -22,37 +23,37 @@ This document describes configuration and implementation for 3 types of clusters * Master/slaves cluster: one host has the master replication role with multiple hosts in slave replication role. * Hybrid cluster: multiple hosts in master replication role with multiple hosts in slave replication role. -# Load balancing implementation -## Random picking += Load balancing implementation +== Random picking When initializing a connection or after a failed connection, the connector will attempt to connect to a host with a certain role (slave/master). The connection is selected randomly among the valid hosts. Thereafter, all statements will run on that database server until the connection will be closed (or fails). The load-balancing will includes a pooling mechanism. Example: when creating a pool of 60 connections, each one will use a random host. With 3 master hosts, the pool will have about 20 connections to each host. -## Master/slave distributed load +== Master/slave distributed load -For a cluster composed of masters and slaves on connection initialization, there will be 2 underlying connections: one with a master host, another with a slave host. Only one connection is used at a time.
-For a cluster composed of master hosts only, each connection has only one underlying connection.
-The load will be distributed due to the random distribution of connections..
+For a cluster composed of masters and slaves on connection initialization, there will be 2 underlying connections: one with a master host, another with a slave host. Only one connection is used at a time. \\ +For a cluster composed of master hosts only, each connection has only one underlying connection. \\ +The load will be distributed due to the random distribution of connections..\\ -## Master/slave connection selection +== Master/slave connection selection -It’s the application that has to decide to use master or slave connection (the master connection is set by default).
-Switching the type of connection is done by using JDBC [connection.setReadOnly(boolean readOnly)](http://docs.oracle.com/javase/7/docs/api/java/sql/Connection.html#setReadOnly(boolean)) method. Setting read-only to true will use the slave connection, false, the master connection.
+It’s the application that has to decide to use master or slave connection (the master connection is set by default).\\ +Switching the type of connection is done by using JDBC [[http://docs.oracle.com/javase/7/docs/api/java/sql/Connection.html#setReadOnly(boolean)|connection.setReadOnly(boolean readOnly)]] method. Setting read-only to true will use the slave connection, false, the master connection.\\ Example in standard java: -``` java +{{{ connection = DriverManager.getConnection("jdbc:mysql:replication://master1,slave1/test"); stmt = connection.createStatement(); stmt.execute("SELECT 1"); // will execute query on the underlying master1 connection connection.setReadOnly(true); stmt.execute("SELECT 1"); // will execute query on the underlying slave1 connection -``` +}}} -Some frameworks render this kind of operation easier, as for example Spring [@transactionnal](http://docs.spring.io/spring/docs/current/javadoc-api/org/springframework/transaction/annotation/Transactional.html#readOnly--) readOnly parameter (since spring 3.0.1). +Some frameworks render this kind of operation easier, as for example Spring [[http://docs.spring.io/spring/docs/current/javadoc-api/org/springframework/transaction/annotation/Transactional.html#readOnly--|@transactionnal]] readOnly parameter (since spring 3.0.1). In this example, setting readOnly to false will call the connection.setReadOnly(false) and therefore use the master connection. -``` java +{{{ @Autowired private EntityManager em; @@ -63,30 +64,30 @@ public void createContacts() { contact1.setName("JIM"); em.persist(contact1); } -``` +}}} Generated Spring Data repository objects use the same logic: the find* method will use the slave connection, other use master connection without having to explicitly set that for each method. On a cluster with master hosts only, the use of connection.setReadOnly(true) does not change the connection, but if the database version is 10.0.0 or higher, the session is set to readOnly if option assureReadOnly is set to true, which means that any write query will throw an exception. -#Failover behaviour -##Basic failover +=Failover behaviour +==Basic failover When no failover/high availability parameter is set, the failover support is basic. Before executing a query, if the connection with the host is discarded, the connection will be reinitialized if parameter “autoReconnect” is set to true. -##Standard failover -When a failover /high availability parameter is set.Check the [configuration](#configuration) section for an overview on how to set the parameters. +==Standard failover +When a failover /high availability parameter is set.Check the [[configuration]] section for an overview on how to set the parameters. There can be multiple fail causes. When a failure occurs many things will be done: * The fail host address will be put on a blacklist (shared by JVM). This host will not be used for the amount of time defined by the “loadBalanceBlacklistTimeout” parameter (default to 50 seconds). The only time a blacklisted address can be used is if all host of the same type (master/slave) are blacklisted. -* The connector will check the connection (with the mysql [ping protocol](https://dev.mysql.com/doc/internals/en/com-ping.html)). If the connection is back, is not read-only, and is in a transaction, the transaction will be rollbacked (there is no way to know if the last query has been received by the server and executed). +* The connector will check the connection (with the mysql [[https://dev.mysql.com/doc/internals/en/com-ping.html|ping protocol]]). If the connection is back, is not read-only, and is in a transaction, the transaction will be rollbacked (there is no way to know if the last query has been received by the server and executed). * If the failure relates to a slave connection * If the master connection is still active, the master connection will be used immediately. The query that was read-only will be relaunched and the connector will not throw any exception. A "failover" thread will be launched to attempt to reconnect a slave host. (if the query was a prepared query, this query will be re-prepared before execution) - * If the master connection is not active, the driver will attempt to create a new master or slave connection with a [connection loop](#connection-loop). + * If the master connection is not active, the driver will attempt to create a new master or slave connection with a [[#connection-loop|connection loop]]. if any connection is found, the query will be relaunched, if not, an SQLException with sqlState like “08XXX” will be thrown. -* If the failure relates to a master connection, the driver will attempt to create a new master connection with a [connection loop](#connection-loop), so the connection object will be immediately reusable.
+* If the failure relates to a master connection, the driver will attempt to create a new master connection with a [[#connection-loop|connection loop]], so the connection object will be immediately reusable.\\ * on failure, an SQLException with be thrown with SQLState "08XXX". If using a pool, this connection will be discarded. * on success, * if possible query will be relaunched without throwing error (if was using a slave connection, or was a SELECT query not in a transaction for example). @@ -94,7 +95,7 @@ There can be multiple fail causes. When a failure occurs many things will be don * When throwing an SQLException with SQLState "08XXX", the connection will be marked as closed. * A “failover” thread will be launched to attempt to reconnect failing connection if connection is not closed. -It’s up to the application to take measures to handle SQLException. See details in [application concerns](#application-concerns). +It’s up to the application to take measures to handle SQLException. See details in [[#application-concerns|application concerns]]. #Connection loop When initializing a connection or after a failure, the driver will launch a connection loop the only case when this connection loop will not be executed is when the failure occurred on a slave with an active master. @@ -114,27 +115,26 @@ For a slave connection : The sequence stops as soon as all the underlying needed connections are found. Every time an attempt fails, the host will be blacklisted. If after an entire loop a master connection is missing, the connection will be marked as closed. -#Additional threads +=Additional threads -##Failover reconnection threads +==Failover reconnection threads A thread pool is created in case of a master/slave cluster, the size is defined according to the number of connection. After a failure on a slave connection, readonly operations are temporary executed on the master connection. Some “failover threads” will try to reconnect the failed underlying connections. -When a new slave connection is retrieved, this one will be immediately used if connection was still in read-only mode.
-More details in [Failover loop threads](./Failover_loop). +When a new slave connection is retrieved, this one will be immediately used if connection was still in read-only mode.\\ +More details in [[failover_loop.creole|Failover loop threads]]. -##Connection validation thread +==Connection validation thread An additional thread is created when setting the option "validConnectionTimeout". This thread will very that connections are all active. -This is normally done by pool that call [Connection.isValid()](https://docs.oracle.com/javase/7/docs/api/java/sql/Connection.html#isValid(int)). +This is normally done by pool that call [[https://docs.oracle.com/javase/7/docs/api/java/sql/Connection.html#isValid(int)|Connection.isValid()]]. -#Application concerns +=Application concerns When a failover happen a SQLException with sqlState like "08XXX" or "25S03" may be thrown. Here are the different connection error codes: -|Code |Condition | -|-----------|:----------| +|=Code |=Condition | |08000 | connection exception| |08001 |SQL client unable to establish SQL connection| |08002 |connection name in use| @@ -142,64 +142,61 @@ Here are the different connection error codes: |08004 |SQL server rejected SQL connection| |08006 |connection failure| |08007 |transaction resolution unknown| -|-----------|:----------| |25S03 |invalid transaction state-transaction is rolled back| -A connection pool will detect connection error in SQLException (SQLState begin with "08"), and this connection will be discarded from pool.
+A connection pool will detect connection error in SQLException (SQLState begin with "08"), and this connection will be discarded from pool.\\ When a failover occur the connector cannot know if the last request has been received by the database server and executed. Applications may have failover design to handle these particular cases: If the application was in autoCommit mode (not recommended), the last query may have been executed and committed. The application will have no possibility to know that but the application will be functional. -If not in autoCommit mode, the query has been launched in a transaction that will not be committed. Depending of what caused the exception, the host may have the connection open on his side during a certain amount of time. Take care of [transaction isolation](https://mariadb.com/kb/en/mariadb/set-transaction/) level that may lock too much rows. +If not in autoCommit mode, the query has been launched in a transaction that will not be committed. Depending of what caused the exception, the host may have the connection open on his side during a certain amount of time. Take care of [[https://mariadb.com/kb/en/mariadb/set-transaction/|transaction isolation]] level that may lock too much rows. -#Configuration +=Configuration -(See [About MariaDB java connector](./About-MariaDB-Connector-J.md) for all connection parameters) +(See [[about-mariadb-connector-j.creole|About MariaDB java connector]] for all connection parameters) JDBC connection string format is : -```script +{{{ jdbc:(mysql|mariadb):[replication:|failover:|loadbalance:|aurora:]//[,...]/[database][?=[&=]...] -``` +}}} -The standard option "connectTimeout" defined the socket connection timeout. by default, these option is set to 0 (no timeout).
-Since there are many servers, setting this option to a small amount of time make sense.
-During the [connection loop phase](#connection-loop), the driver will try to connect to server sequentially until the creation of an active connection. +The standard option "connectTimeout" defined the socket connection timeout. by default, these option is set to 0 (no timeout).\\ +Since there are many servers, setting this option to a small amount of time make sense.\\ +During the [[#connection-loop|connection loop phase]], the driver will try to connect to server sequentially until the creation of an active connection. Set this option to a small value (like 2000ms - to be set according to your environment) will permit to reject faulty server quickly. -##Failover / high availability parameters +==Failover / high availability parameters Each parameter corresponds to a specific use case: -|Failover option|Description| -|-----------|:----------| -| **failover** | High availability (random picking connection initialisation) with failover support for master replication cluster (exemple Galera).
* Since 1.2.0*| -| **sequential** |Failover support for master replication cluster (for example Galera) **without** High availability.
the host will be connected in the order in which they were declared.

Example when using the jdbc url string "jdbc:mysql:replication:host1,host2,host3/test" :
When connecting, the driver will always try first host1, and if not available host2 and following. After a host fail, the driver will reconnect according to this order.
*Since 1.3.0*| -| **replication** | High availability (random picking connection initialisation) with failover support for master/slaves replication cluster (one or multiple master).
* Since 1.2.0*| -| **aurora** | High availability (random picking connection initialisation) with failover support for Amazon Aurora replication cluster.
* Since 1.2.0*| +|=Failover option|=Description| +| **failover** | High availability (random picking connection initialisation) with failover support for master replication cluster (exemple Galera). \\* Since 1.2.0*| +| **sequential** |Failover support for master replication cluster (for example Galera) **without** High availability. \\the host will be connected in the order in which they were declared.\\\\Example when using the jdbc url string "jdbc:mysql:replication:host1,host2,host3/test" : \\When connecting, the driver will always try first host1, and if not available host2 and following. After a host fail, the driver will reconnect according to this order.\\*Since 1.3.0*| +| **replication** | High availability (random picking connection initialisation) with failover support for master/slaves replication cluster (one or multiple master).\\* Since 1.2.0*| +| **aurora** | High availability (random picking connection initialisation) with failover support for Amazon Aurora replication cluster.\\* Since 1.2.0*| -##Failover / high availability options +==Failover / high availability options -|Option|Description| -|-----------|:----------| -|autoReconnect|With basic failover only, if true, will attempt to recreate connection after a failover.
*Default is false. Since 1.1.7*| -|retriesAllDown|When searching a valid host, maximum number of connection attempts before throwing an exception.
*Default: 120. Since 1.2.0| -|failoverLoopRetries|When searching silently for a valid host, maximum number of connection attempts.
This differ from "retriesAllDown" parameter, because this silent search is for example used after a disconnection of a slave connection when using the master connection.
*Default: 120. Since 1.2.0*| -|validConnectionTimeout|With multiple hosts, after this time in seconds has elapsed it’s verified that the connections haven’t been lost.
When 0, no verification will be done.
*Default:120 seconds. Since 1.2.0*| -|loadBalanceBlacklistTimeout|When a connection fails, this host will be blacklisted during the "loadBalanceBlacklistTimeout" amount of time.
When connecting to a host, the driver will try to connect to a host in the list of not blacklisted hosts and after that only on blacklisted ones if none has been found before that.
This blacklist is shared inside the classloader.
*Default: 50 seconds. Since 1.2.0*| -|assureReadOnly|If true, in high availability, and switching to a read-only host, assure that this host is in read-only mode by setting session read-only.
alias "readOnlyPropagatesToServer" worked to for compatibility
*Default to false.
Since 1.3.0*| +|=Option|=Description| +|autoReconnect|With basic failover only, if true, will attempt to recreate connection after a failover.\\*Default is false. Since 1.1.7*| +|retriesAllDown|When searching a valid host, maximum number of connection attempts before throwing an exception.\\*Default: 120. Since 1.2.0| +|failoverLoopRetries|When searching silently for a valid host, maximum number of connection attempts.\\This differ from "retriesAllDown" parameter, because this silent search is for example used after a disconnection of a slave connection when using the master connection.\\*Default: 120. Since 1.2.0*| +|validConnectionTimeout|With multiple hosts, after this time in seconds has elapsed it’s verified that the connections haven’t been lost.\\When 0, no verification will be done.\\*Default:120 seconds. Since 1.2.0*| +|loadBalanceBlacklistTimeout|When a connection fails, this host will be blacklisted during the "loadBalanceBlacklistTimeout" amount of time.\\When connecting to a host, the driver will try to connect to a host in the list of not blacklisted hosts and after that only on blacklisted ones if none has been found before that.\\This blacklist is shared inside the classloader.\\*Default: 50 seconds. Since 1.2.0*| +|assureReadOnly|If true, in high availability, and switching to a read-only host, assure that this host is in read-only mode by setting session read-only.\\alias "readOnlyPropagatesToServer" worked to for compatibility\\*Default to false.\\ Since 1.3.0*| -#Specifics for Amazon Aurora +=Specifics for Amazon Aurora -Amazon Aurora is a Master/Slaves cluster composed of one master instance with a maximum of 15 slave instances. Amazon Aurora includes automatic promotion of a slave instance in case of the master instance failing. The MariaDB connector/J implementation for Aurora is specific to handle this automatic failover.
+Amazon Aurora is a Master/Slaves cluster composed of one master instance with a maximum of 15 slave instances. Amazon Aurora includes automatic promotion of a slave instance in case of the master instance failing. The MariaDB connector/J implementation for Aurora is specific to handle this automatic failover.\\ To permit development/integration on a single-node cluster, only one host can be defined. In this case, the driver behaves as for the configuration **failover**. -##Aurora failover implementation +==Aurora failover implementation Aurora failover management steps : * Instance A is in write replication mode, instance B and C are in read replication mode. * Instance A fails. @@ -207,9 +204,9 @@ Aurora failover management steps : * Cluster end-point will change to instance B end-point. * Instance A will recover and be in read replication mode. -##Aurora configuration +==Aurora configuration -###Aurora endpoints +===Aurora endpoints Every instance has a specific endpoint, ie an URL that identify the host. Those endpoints look like “xxxx.yyyy.us-east-1.rds.amazonaws.com”. @@ -217,25 +214,37 @@ There is another endpoint named “cluster endpoint” which is assigned to the This cluster endpoint must never be used in the URL connection string for 2 reasons: * When a failover occurs, a new master is promoted. The cluster endpoint change to this new master is not immediately effective. The connector doesn’t use this cluster endpoint. Instead it points to the new master immediately. -* More important, an instance will not be used, and load will be poorly distributed. \\Example : \\Normally JDBC url string must be like :\\##jdbc:mysql:aurora://A.XX.com,B.XX.com,C.XX.com/db##\\if the master endpoint is used : \\##jdbc:mysql:aurora://master.XX.com,B.XX.com,C.XX.com/db## \\If B become master, A will not be used at all and C will receive all the read queries. +* More important, an instance will not be used, and load will be poorly distributed. \\Example : +\\Normally JDBC url string must be like :\\ +{{{ + jdbc:mysql:aurora://A.XX.com,B.XX.com,C.XX.com/db +}}} + +if the master endpoint is used : \\ +{{{ + jdbc:mysql:aurora://master.XX.com,B.XX.com,C.XX.com/db +}}} \\ +If B become master, A will not be used at all and C will receive all the read queries. -###JDBC connection string +===JDBC connection string The implementation is activated by specifying the “aurora” failover parameter. -```script +{{{ jdbc:(mysql|mariadb):aurora://[instanceEndPoint[:port]][,instanceEndPoint[:port]...]/[database][?=[&=]...] -``` +}}} Host declaration use instance endpoint (never cluster endpoint). The replication role of each instance must not be defined for Aurora, because the role of each instance changes over time. The driver will check the instance role after connection initialisation. Example of connection string - `jdbc:mysql:aurora://host1.xxxx.us-east-1.rds.amazonaws.com,host2.xxxx.us-east-1.rds.amazonaws.com/db` +{{{ + jdbc:mysql:aurora://host1.xxxx.us-east-1.rds.amazonaws.com,host2.xxxx.us-east-1.rds.amazonaws.com/db +}}} Another difference is the option "socketTimeout" that defaults to 10 seconds, meaning that - if not changed - queries exceeding 10 seconds will throw exceptions. -##Aurora connection loop +==Aurora connection loop When searching for the master instance and connect to a slave instance, the connection order will be: * Every Aurora instance knows the hostname of the current master. If the host has been described using their instance endpoint, that will permit to know the master instance and connect directly to it. @@ -247,11 +256,11 @@ When searching for a slave instance, the loop will connection order will be: * random blacklisted instances The loop will retry until the connections are found or parameter “retriesAllDown” is exceeded. -##Aurora master verification +==Aurora master verification Without any query during the time defined by the parameter validConnectionTimeout (default to 120s) and if not set to 0, a verification will be done that the replication role of the underlying connections haven’t changed. -##Aurora connection validation thread -Aurora as a specific [connection validation thread](#connection-validation-thread) implementation. +==Aurora connection validation thread +Aurora as a specific [[#connection-validation-thread|connection validation thread]] implementation. Since role of each instance can change over time, this will validate that connection are active AND role have not changed. \ No newline at end of file diff --git a/documentation/Failover_loop.md b/documentation/failover_loop.creole similarity index 81% rename from documentation/Failover_loop.md rename to documentation/failover_loop.creole index 878c62727..26460793a 100644 --- a/documentation/Failover_loop.md +++ b/documentation/failover_loop.creole @@ -1,13 +1,14 @@ -> **This guide will teach you:** -> * The goal of the failover threads +//**This guide will teach you:**// -# Failover reconnection + * The goal of the failover threads -> ** This concern only master/slave cluster ** += Failover reconnection + +//** This concern only master/slave cluster **// On a master/slave cluster, driver will use underlying 2 connections: one to a master instance, one to a slave instance. -When one of the connection fail, if driver does need it at once, it will create a new connection immediately before re-executing query if possible.
+When one of the connection fail, if driver does need it at once, it will create a new connection immediately before re-executing query if possible.\\ If the failed connection is not needed immediately, this driver will subscribe to the "failover reconnection" that will be handle in other threads. Failover threads will attempt to create new connection to replace failing ones, so the interruption is minimal for the queries in progress. When client asked to use a failed connection, the new connection created by failover thread will replace the failed one. @@ -17,16 +18,16 @@ Failover thread will then create a new slave connection that will replace the fa A pool of threads is initialized when using a master/slave configuration. The pool size evolves according to the number of connection. -## Illustration +== Illustration -Here is an example of a failover on a aurora cluster of 3 instances (one master and 2 slaves).
+Here is an example of a failover on a aurora cluster of 3 instances (one master and 2 slaves).\\ (Source code https://github.com/rusher/connector-aurora-fail-test/tree/master) We can see 2 kinds of threads : * Threads named "test-thread-XXX" do 130 queries "SELECT 1". 1/3 use master connection, 2/3 slave connection. * Threads "mariaDb-reconnection-XXX" are created by the driver to handle failover. -#### Colour signification: +==== Colour signification: "test-thread-XXX" threads: * blue: querying * red: blocked waiting to connect @@ -38,6 +39,7 @@ We can see 2 kinds of threads : When the failover occur, most of the wasted time to reconnect is supported by the reconnection thread. Most of query will be executed normally, only a few query executions will have a small additional delay (red block on "test-thread-XXX" threads). - - + +[[misc/images/telemetry.png|complete results]]\\ +{{misc/images/aurora_fail_extract.png}} \ No newline at end of file diff --git a/documentation/plugin/GSSAPI.md b/documentation/plugin/GSSAPI.creole similarity index 77% rename from documentation/plugin/GSSAPI.md rename to documentation/plugin/GSSAPI.creole index 1ecdd7bbe..61dff43ad 100644 --- a/documentation/plugin/GSSAPI.md +++ b/documentation/plugin/GSSAPI.creole @@ -1,5 +1,5 @@ -# GSSAPI Authentication += GSSAPI Authentication MariaDB GSSAPI support GSSAPI since the 10.1 version (Server configuration can be found on https://github.com/MariaDB/server/blob/master/plugin/auth_gssapi/README.md). @@ -9,7 +9,7 @@ Support history: * version 1.4.0 : java connector support * version 1.5.0 : added native windows implementation. -## General configuration +== General configuration Database configuration must have been set. To use GSSAPI authentication, a user must be set to use GSSAPI : @@ -28,8 +28,8 @@ That mean on client, user must have obtained a TGT beforehand. As part of the security context establishment, the driver will initiate a context that will be authenticated by database. Database also be authenticated back to the driver ("mutual authentication"). -### GSSAPI configuration -#### Java system properties +=== GSSAPI configuration +==== Java system properties Realm information are generally defined by DNS, but this can be forced using system properties. "java.security.krb5.kdc" defined the Key Distribution Center (KDC), realm by "java.security.krb5.realm". @@ -44,9 +44,9 @@ Logging can be set using additional properties: System.setProperty("sun.security.jgss.debug", "true"); -#### Java JCE +==== Java JCE -Depending on the kerberos ticket encryption, you may have to install the [Java Cryptography Extension (JCE)](http://www.oracle.com/technetwork/java/javase/downloads/jce8-download-2133166.html) Unlimited Strength Jurisdiction Policy File. +Depending on the kerberos ticket encryption, you may have to install the [[http://www.oracle.com/technetwork/java/javase/downloads/jce8-download-2133166.html|Java Cryptography Extension (JCE)]] Unlimited Strength Jurisdiction Policy File. (CentOS/Red Hat Enterprise Linux 5.6 or later, Ubuntu are using AES-256 encryption by default for tickets). On unix, you can execute the "klist -e" command to view the encryption type in use: @@ -59,17 +59,17 @@ If AES is being used, output like the following is displayed after you type the Etype (skey, tkt): AES-256 CTS mode with 96-bit SHA-1 HMAC, AES-256 CTS mode with 96-bit SHA-1 HMAC -### Implementations +=== Implementations -On windows GSSAPI implementation is SSPI. The java 8 native implementation as many limitations ([see java ticket](http://bugs.java.com/bugdatabase/view_bug.do?bug_id=6722928)). +On windows GSSAPI implementation is SSPI. The java 8 native implementation as many limitations ([[http://bugs.java.com/bugdatabase/view_bug.do?bug_id=6722928|see java ticket]]). There is 2 Different implementations: * a java standard implementation will use JAAS to allow java to access TGT. -* a windows native implementation based on [Waffle](https://github.com/dblock/waffle) +* a windows native implementation based on [[https://github.com/dblock/waffle|Waffle]] -#### Standard java SSPI implementation +==== Standard java SSPI implementation -##### Jaas +===== Jaas The driver will use the native ticket cache to get the TGT available in it using JAAS. If the System property "java.security.auth.login.config" is empty, driver will use the following configuration : @@ -80,7 +80,7 @@ If the System property "java.security.auth.login.config" is empty, driver will u This permit to use current user TGT cache -##### limitation on windows +===== limitation on windows Main limitation are : * To permit java to retrieve TGT (Ticket-Granting-Ticket), windows host need to have a registry entry set. @@ -90,20 +90,20 @@ Main limitation are : Value: 1 * Kinit command must have been executed previously to connection. -### Windows native java implementation -Implementation is based on [Waffle](https://github.com/dblock/waffle) that support windows SSPI based on [JNA](https://github.com/java-native-access/jna). +=== Windows native java implementation +Implementation is based on [[https://github.com/dblock/waffle|Waffle]] that support windows SSPI based on [[https://github.com/java-native-access/jna|JNA]]. if on waffle-jna (and dependencies) is on classpath, native implementation will automatically be used. (This permit to avoid any specific problem with admin right, registry, kinit ...) Dependencies : -* [waffle-jna 1.8.1](https://maven-badges.herokuapp.com/maven-central/com.github.dblock.waffle/waffle-jna) -* [jna 4.2.1](https://maven-badges.herokuapp.com/maven-central/net.java.dev.jna/jna) -* [jna-platform 4.2.1](https://maven-badges.herokuapp.com/maven-central/net.java.dev.jna/jna-platform) -* [jcl-over-slf4j 1.7.14](https://maven-badges.herokuapp.com/maven-central/org.slf4j/jcl-over-slf4j) -* [slf4j-api 1.7.14](https://maven-badges.herokuapp.com/maven-central/org.slf4j/slf4j-api) -* [guava 19.0](https://maven-badges.herokuapp.com/com.google.guava/guava) - -##Possible errors +* [[https://maven-badges.herokuapp.com/maven-central/com.github.dblock.waffle/waffle-jna|waffle-jna 1.8.1]] +* [[https://maven-badges.herokuapp.com/maven-central/net.java.dev.jna/jna|jna 4.2.1]] +* [[https://maven-badges.herokuapp.com/maven-central/net.java.dev.jna/jna-platform|jna-platform 4.2.1]] +* [[https://maven-badges.herokuapp.com/maven-central/org.slf4j/jcl-over-slf4j|jcl-over-slf4j 1.7.14]] +* [[https://maven-badges.herokuapp.com/maven-central/org.slf4j/slf4j-api|slf4j-api 1.7.14]] +* [[https://maven-badges.herokuapp.com/com.google.guava/guava|guava 19.0]] + +==Possible errors * "GSSException: Failure unspecified at GSS-API level (Mechanism level: No Kerberos credentials available)" There is no active credential. Check with klist that there is an existing credential. If not create it with the "kinit" command diff --git a/documentation/use-mariadb-connector-j-driver.creole b/documentation/use-mariadb-connector-j-driver.creole new file mode 100644 index 000000000..708faf9b5 --- /dev/null +++ b/documentation/use-mariadb-connector-j-driver.creole @@ -0,0 +1,218 @@ += Using the driver + +The following subsections show the formatting of JDBC connection strings for +MariaDB and MySQL database servers. Additionally, sample code is provided that +demonstrates how to connect to one of these servers and create a table. + +== Getting a new connection + +There are two standard ways to get a connection: + +=== Using DriverManager +The prefered way to connect is to use [[https://docs.oracle.com/javase/7/docs/api/java/sql/DriverManager.html|DriverManager]]. +Applications designed to use the driver manager to locate the entry point need +no further configuration. MariaDB Connector/J will +automatically be loaded and used in the way any previous MySQL driver would +have been. + +Example: +{{{ +Connection connection = DriverManager.getConnection("jdbc:mariadb://localhost:3306/DB?user=root&password=myPwd"); +}}} + +//The legacy way of loading a JDBC driver (using Class.forName("org.mariadb.jdbc.Driver")) still works.// + +=== Using external pool + +When using an external connection pool, the mariadb Driver class {{{org.mariadb.jdbc.Driver}}} must be configured. + +Example using [[https://github.com/brettwooldridge/HikariCP|hikariCP JDBC connection pool]] : + +{{{ + final HikariDataSource ds = new HikariDataSource(); + ds.setMaximumPoolSize(20); + ds.setDriverClassName("org.mariadb.jdbc.Driver"); + ds.setJdbcUrl("jdbc:mariadb://localhost:3306/db"); + ds.addDataSourceProperty("user", "root"); + ds.addDataSourceProperty("password", "myPwd"); + ds.setAutoCommit(false); +}}} + +Please note that the driver class provided by MariaDB Connector/J **is not {{{com.mysql.jdbc.Driver}}} but {{{org.mariadb.jdbc.Driver}}}**! + +The {{{org.mariadb.jdbc.MariaDbDataSource}}} class can be used when the pool datasource configuration only permits the java.sql.Datasource implementation. + +== Connection strings + +The format of the JDBC connection string is +{{{ +jdbc:(mysql|mariadb):[replication:|failover:|sequential:|aurora:]//[,...]/[database][?=[&=]] +}}} + + HostDescription: +{{{ +[:] or address=(host=)[(port=)][(type=(master|slave))] +}}} + +Host must be a DNS name or IP address. In case of ipv6 and simple host +description, the IP address must be written inside brackets. The default port +is {{{3306}}}. The default type is {{{master}}}. If {{{replication}}} failover is +set, by default the first host is master, and the others are slaves. + +Examples : +* {{{localhost:3306}}} +* {{{[2001:0660:7401:0200:0000:0000:0edf:bdd7]:3306}}} +* {{{somehost.com:3306}}} +* {{{address=(host=localhost)(port=3306)(type=master)}}} + +== Failover parameters + +Failover was introduced in Connector/J 1.2.0. + +|=sequential |Failover support for master replication cluster (for example Galera) without High availability. The hosts will be connected in the order in which they were declared.\\\\Example when using the jdbc url string "jdbc:mysql:replication:host1,host2,host3/test" : \\When connecting, the driver will always first try host1, and if not available host2 and so on. After a host fail, the driver will reconnect according to this order. \\//since 1.3.0//| +|=failover | High availability (random picking connection initialisation) with failover support for master replication cluster (for example Galera). \\//since 1.2.0//| +|=replication | High availability (random picking connection initialisation) with failover support for master/slave replication cluster (one or multiple masters) \\//since 1.2.0//| +|=aurora | High availability (random picking connection initialisation) with failover support for Amazon Aurora replication cluster \\//since 1.2.0//| + +See [[failover-and-high-availability-with-mariadb-connector-j|failover description]] for more information. +\\\\ +== Optional URL parameters + +General remark: Unknown options are accepted and silently ignored. + +The following options are currently supported. +=== Essential options +|=user|Database user name. \\//since 1.0.0//| +|=password|Password of database user.\\//since 1.0.0//| +|=rewriteBatchedStatements| For insert queries, rewrite batchedStatement to execute in a single executeQuery.\\example:\\{{{insert into ab (i) values (1)}}}\\{{{insert into ab (i) values (2)}}}\\will be rewritten \\{{{insert into ab (i) values (1), (2)}}}.\\\\when active, the useServerPrepStmts option is set to false\\//Default: false. Since 1.1.8//| +|=connectTimeout| The connect timeout value, in milliseconds, or zero for no timeout.\\//Default: 0. Since 1.1.8//| +|=useServerPrepStmts|Queries are prepared on the server side before executing, permitting faster execution next time.\\if allowMultiQueries or rewriteBatchedStatements is set to true, this option will be set to false.\\//Default: true. Since 1.3.0//| + +\\ + +=== TLS (SSL) + +|=useSSL|Force [[https://mariadb.com/kb/en/mariadb/secure-connections-overview/secure-connections-overview|SSL/TLS on connection]].\\//Default: false. Since 1.1.0//| +|=trustServerCertificate|When using SSL/TLS, do not check server's certificate.\\//Default: false. Since 1.1.1//| +|=serverSslCert|Server's certificate in DER form, or server's CA certificate. \\Can be used in one of 3 forms : \\* sslServerCert=/path/to/cert.pem (full path to certificate)\\* sslServerCert=classpath:relative/cert.pem (relative to current classpath)\\* or as verbatim DER-encoded certificate string "------BEGING CERTIFICATE-----" .\\//since 1.1.3//| +|=clientCertificateKeyStoreUrl|Use the specified keystore for client certificates (can be the same as the trusted root certificate keystore).\\//Default: false. since 1.3.4//| +|=clientCertificateKeyStorePassword|Password for the client certificate keystore.\\//Default: false. since 1.3.4//| +|=trustCertificateKeyStoreUrl|Use the specified keystore for trusted root certificates. Overrides serverSslCert.\\//Default: false.\\ Since 1.3.4//| +|=trustCertificateKeyStorePassword|Password for the trusted root certificate keystore.\\//Default: false.\\ Since 1.3.4//| + +\\ + +=== Infrequently used + +|=useFractionalSeconds| Correctly handle subsecond precision in timestamps (feature available with MariaDB 5.3 and later).\\May confuse 3rd party components (Hibernated).\\//Default: true. Since 1.0.0//| +|=allowMultiQueries| Combine batchedStatement queries to execute in a single executeQuery.\\example:\\{{{insert into ab (i) values (1)}}}\\{{{insert into ab (i) values (2)}}}\\ will be combine in one query :\\{{{insert into ab (i) values (1); insert into ab (i) values (2)}}}.\\\\Option inactive when rewriteBatchedStatements is active. \\When active, the useServerPrepStmts option is set to false\\//Default: false. Since 1.0.0//| +|=dumpQueriesOnException|If set to 'true', an exception is thrown during query execution containing a query string.\\//Default: false. Since 1.1.0//| +|=useCompression|allow compression in the MySQL Protocol.\\//Default: false. Since 1.0.0//| +|=socketFactory| to use a custom socket factory, set it to the full name of the class that implements javax.net.SocketFactory.\\//since 1.0.0//| +|=tcpNoDelay|Sets corresponding option on the connection socket.\\//since 1.0.0//| +|=tcpKeepAlive|Sets corresponding option on the connection socket.\\//since 1.0.0//| +|=tcpAbortiveClose|Sets corresponding option on the connection socket.\\//since 1.1.1//| +|=tcpRcvBuf| set buffer size for TCP buffer (SO_RCVBUF).\\//since 1.0.0//| +|=tcpSndBuf| set buffer size for TCP buffer (SO_SNDBUF).\\//since 1.0.0//| +|=pipe| On Windows, specify named pipe name to connect to mysqld.exe.\\//since 1.1.3//| +|=tinyInt1isBit| Datatype mapping flag, handle MySQL Tiny as BIT(boolean).\\//Default: true. Since 1.0.0//| +|=yearIsDateType|Year is date type, rather than numerical.\\//Default: true. Since 1.0.0//| +|=sessionVariables|= pairs separated by comma, mysql session variables, set upon establishing successful connection.\\//since 1.1.0//| +|=localSocket|Permits connecting to the database via Unix domain socket, if the server allows it. \\The value is the path of Unix domain socket (i.e "socket" database parameter : select @@socket) .\\//since 1.1.4//| +|=sharedMemory|Permits connecting to the database via shared memory, if the server allows it. \\The value is the base name of the shared memory.\\//since 1.1.4//| +|=localSocketAddress|Hostname or IP address to bind the connection socket to a local (UNIX domain) socket.\\//since 1.1.7//| +|=socketTimeout|Defined the network socket timeout (SO_TIMEOUT) in milliseconds. \\Default: 0 milliseconds(0 disable this timeout).\\//since 1.1.7//| +|=interactiveClient|Session timeout is defined by the [[https://mariadb.com/kb/en/server-system-variables/#wait_timeout|wait_timeout]] server variable. Setting interactiveClient to true will tell the server to use the [[https://mariadb.com/kb/en/mariadb/server-system-variables/#interactive_timeout|interactive_timeout]] server variable.\\//Default: false. Since 1.1.7//| +|=useOldAliasMetadataBehavior|Metadata ResultSetMetaData.getTableName() returns the physical table name. "useOldAliasMetadataBehavior" permits activating the legacy code that sends the table alias if set. \\//Default: false. Since 1.1.9//| +|=createDatabaseIfNotExist|the specified database in the url will be created if nonexistent.\\//Default: false. Since 1.1.7//| +|=serverTimezone|Defines the server time zone.\\to use only if the jre server has a different time implementation of the server.\\(best to have the same server time zone when possible).\\//since 1.1.7//| +|=prepStmtCacheSize| if useServerPrepStmts = true, defines the prepared statement cache size. \\//Default: 250. Since 1.3.0//| +|=prepStmtCacheSqlLimit| if useServerPrepStmts = true, defined queries larger than this size will not be cached. \\//Default: 2048. Since 1.3.0//| +|=jdbcCompliantTruncation| Truncation error ("Data truncated for column '%' at row %", "Out of range value for column '%' at row %") will be thrown as an error, and not as a warning.\\//Default: true. Since 1.4.0//| +|=cacheCallableStmts| enable/disable callable Statement cache\\//Default: true. Since 1.4.0//| +|=callableStmtCacheSize| This sets the number of callable statements that the driver will cache per VM if "cacheCallableStmts" is enabled.\\//Default: true. Since 1.4.0//| + + +\\\\ +== Failover/High availability URL parameters + +|=autoReconnect|With basic failover: if true, will attempt to recreate connection after a failover. \\With standard failover: if true, will attempt to recreate connection even if there is a temporary solution (like using a master connection temporary until reconnect to a slave connection) \\//Default is false. Since 1.1.7//| +|=retriesAllDown|When searching a valid host, maximum number of connection attempts before throwing an exception.\\//Default: 120 seconds. Since 1.2.0//| +|=failoverLoopRetries|When searching silently for a valid host, maximum number of connection attempts.\\This differs from the "retriesAllDown" parameter because this silent search is for example used after a disconnection of a slave connection when using the master connection\\//Default: 120. Since 1.2.0//| +|=validConnectionTimeout|With multiple hosts, after this time in seconds has elapsed, verifies that the connections haven’t been lost.\\When 0, no verification will be done. \\//Default:120 seconds. Since 1.2.0//| +|=loadBalanceBlacklistTimeout|When a connection fails, this host will be blacklisted for the "loadBalanceBlacklistTimeout" amount of time.\\When connecting to a host, the driver will try to connect to a host in the list of non-blacklisted hosts and, only if none are found, attempt blacklisted ones.\\This blacklist is shared inside the classloader.\\//Default: 50 seconds. Since 1.2.0//| +|=assureReadOnly|If true, in high availability, and switching to a read-only host, assure that this host is in read-only mode by setting the session to read-only.\\//Default to false. Since 1.3.0//| +\\\\ += JDBC API implementation notes + +== "LOAD DATA INFILE" +The fastest way to load many datas is using query [[https://mariadb.com/kb/en/mariadb/load-data-infile/|LOAD DATA INFILE]]. \\Problem is using "LOAD DATA LOCAL INFILE" (ie : loading a file from client), may be a security problem : +* if server sources has been changed, server mays asked for a different file than the file in query. +* if someone has can execute query from client, he can have access to any file on client (according to the rights of the user running the client process). + +A specific option "allowLocalInfile" (default to true) can deactivate functionality on client side. +The global variable [[https://mariadb.com/kb/en/mariadb/server-system-variables/#local_infile|local_infile]] can disable LOAD DATA LOCAL INFILE on server side. + +A non-JDBC method can permit to use this kind of query without those security issue: The application has to create a InputStream with the file to load. If MariaDbStatement.setLocalInfileInputStream(InputStream inputStream) is set, the inputStream will be send to server, replacing the file content (working even with option "allowLocalInfile" disabled). + +Code example: +{{{ + Statement statement = ... + InputStream in = new FileInputStream("/file.sql"); + + if (statement.isWrapperFor(MariaDbStatement.class)) { + MariaDbStatement mariaDbStatement = statement.unwrap(MariaDbStatement.class); + mariaDbStatement.setLocalInfileInputStream(in); + String sql = "LOAD DATA LOCAL INFILE 'dummyFileName'" + + " INTO TABLE gigantic_load_data_infile " + + " FIELDS TERMINATED BY '\\t' ENCLOSED BY ''" + + " ESCAPED BY '\\\\' LINES TERMINATED BY '\\n'"; + statement.execute(sql); + } else { + in.close(); + throw new RuntimeException("Mariadb JDBC adaptor must be used"); + } + +}}} + +== Streaming result sets +By default, {{{Statement.executeQuery()}}} will read the full result set +from the server before returning. With large result sets, this will require large +amounts of memory. Better behavior in this case would be reading row-by-row, +with {{{ResultSet.next()}}}, so called "streaming". This is +activated using {{{Statement.setFetchSize(Integer.MIN_VALUE) }}} + +== Prepared statements +The driver uses server prepared statements as a standard to communicate with the database (since 1.3.0). If the "allowMultiQueries" or "rewriteBatchedStatements" options are set to true, the driver will only use text protocol. Prepared statements (parameter substitution) is handled by the driver, on the client side. + +== CallableStatement +Callable statement implementation won't need to access stored procedure +metadata ([[https://mariadb.com/kb/en/mariadb/mysqlproc-table/|mysql.proc]]) table if both of following are true + +* CallableStatement.getMetadata() is not used +* Parameters are accessed by index, not by name + +When possible, following the two rules above provides both better speed and +eliminates concerns about SELECT privileges on the +[[https://mariadb.com/kb/en/mariadb/mysqlproc-table/|mysql.proc]] table. + +== Optional JDBC classes +The following optional interfaces are implemented by the +org.mariadb.jdbc.MariaDbDataSource class : javax.sql.DataSource, +javax.sql.ConnectionPoolDataSource, javax.sql.XADataSource + +careful : org.mariadb.jdbc.MySQLDataSource doesn't exist anymore and should be replaced with org.mariadb.jdbc.MariaDbDataSource since v1.3.0 + +== Usage examples + +The following code provides a basic example of how to connect to a MariaDB or +MySQL server and create a table. + +=== Creating a table on a MariaDB or MySQL server +{{{ +try (Connection connection = DriverManager.getConnection("jdbc:mysql://localhost:3306/test", "username", "password")) { + try (Statement stmt = connection.createStatement()) { + stmt.executeUpdate("CREATE TABLE a (id int not null primary key, value varchar(20))"); + } +} +}}}