Skip to content

Commit

Permalink
Initial import of the new client
Browse files Browse the repository at this point in the history
  • Loading branch information
@zhicwu
zhicwu committed Oct 6, 2021
1 parent e6205b4 commit 007d167
Show file tree
Hide file tree
Showing 263 changed files with 35,180 additions and 1,761 deletions.
3 changes: 1 addition & 2 deletions LICENSE
View file
Original file line number Diff line number Diff line change
@@ -1,4 +1,3 @@
Copyright 2016-2021 Yandex LLC

Apache License
Version 2.0, January 2004
Expand Down Expand Up @@ -188,7 +187,7 @@ Copyright 2016-2021 Yandex LLC
same "printed page" as the copyright notice for easier
identification within third-party archives.

Copyright 2016-2021 Yandex LLC
Copyright [yyyy] [name of copyright owner]

Licensed under the Apache License, Version 2.0 (the "License");
you may not use this file except in compliance with the License.
Expand Down
122 changes: 101 additions & 21 deletions README.md
View file
Original file line number Diff line number Diff line change
@@ -1,27 +1,83 @@
ClickHouse JDBC driver
===============
# ClickHouse Java Client & JDBC Driver

[![clickhouse-jdbc](https://maven-badges.herokuapp.com/maven-central/ru.yandex.clickhouse/clickhouse-jdbc/badge.svg)](https://maven-badges.herokuapp.com/maven-central/ru.yandex.clickhouse/clickhouse-jdbc) ![Build Status(https://github.com/ClickHouse/clickhouse-jdbc/workflows/Build/badge.svg)](https://github.com/ClickHouse/clickhouse-jdbc/workflows/Build/badge.svg) [![Coverage](https://sonarcloud.io/api/project_badges/measure?project=ClickHouse_clickhouse-jdbc&metric=coverage)](https://sonarcloud.io/dashboard?id=ClickHouse_clickhouse-jdbc)

This is a basic and restricted implementation of jdbc driver for ClickHouse.
It has support of a minimal subset of features to be usable.
Java client and JDBC driver for ClickHouse.

## Usage

### Java Client

Use Java client when you prefer async and more "direct" way to communicate with ClickHouse. JDBC driver is actually a thin wrapper of the Java client.

```xml
<dependency>
<groupId>com.clickhouse</groupId>
<!-- you'll be able to use clickhouse-http-client and clickhouse-tcp-client as well in the near future -->
<artifactId>clickhouse-grpc-client</artifactId>
<version>0.3.2</version>
</dependency>
```

Example:

```Java
// declare a server to connect to
ClickHouseNode server = ClickHouseNode.of("server.domain", ClickHouseProtocol.GRPC, 9100, "my_db");

// run multiple queries in one go and wait until it's finished
ClickHouseClient.send(server,
"create database if not exists another_database",
"use another_database", // change current database from my_db to test
"create table if not exists my_table(s String) engine=Memory",
"insert into my_table values('1')('2')('3')",
"select * from my_table limit 1",
"truncate table my_table",
"drop table if exists my_table").get();

// query with named parameters
try (ClickHouseClient client = ClickHouseClient.newInstance(ClickHouseProtocol.GRPC);
ClickHouseResponse resp = client.connect(server)
.format(ClickHouseFormat.RowBinaryWithNamesAndTypes).set("send_logs_level", "trace")
.query("select id, name from some_table where id in :ids and name like :name").params(Arrays.asList(1,2,3), "%key%").execute().get()) {
// you can also use resp.recordStream() as well
for (ClickHouseRecord record : resp.records()) {
int id = record.getValue(0).asInteger();
String name = record.getValue(1).asString();
}

ClickHouseResponseSummary summary = resp.getSummary();
long totalRows = summary.getRows();
}

// load data with custom writer
ClickHouseClient.load(server, "target_table", ClickHouseFormat.TabSeparated,
ClickHouseCompression.NONE, new ClickHouseWriter() {
@Override
public void write(OutputStream output) throws IOException {
output.write("1\t\\N\n".getBytes());
output.write("2\t123".getBytes());
}
}).get();
```

### JDBC Driver

### Usage
```xml
<dependency>
<!-- groupId and package name will be changed to com.clickhouse starting from 0.4.0 -->
<groupId>ru.yandex.clickhouse</groupId>
<artifactId>clickhouse-jdbc</artifactId>
<version>0.3.2</version>
</dependency>
```

URL syntax:
`jdbc:clickhouse://<host>:<port>[/<database>]`, e.g. `jdbc:clickhouse://localhost:8123/test`
URL syntax: `jdbc:clickhouse://<host>:<port>[/<database>[?param1=value1&param2=value2]]`, e.g. `jdbc:clickhouse://localhost:8123/test?socket_timeout=120000`

JDBC Driver Class:
`ru.yandex.clickhouse.ClickHouseDriver`
JDBC Driver Class: `ru.yandex.clickhouse.ClickHouseDriver` (will be changed to `com.clickhouse.jdbc.ClickHouseDriver` starting from 0.4.0)

For example:

```java
String url = "jdbc:clickhouse://localhost:8123/test";
ClickHouseProperties properties = new ClickHouseProperties();
Expand All @@ -47,37 +103,40 @@ try (ClickHouseConnection conn = dataSource.getConnection();

Additionally, if you have a few instances, you can use `BalancedClickhouseDataSource`.


### Extended API

In order to provide non-JDBC complaint data manipulation functionality, proprietary API exists.
Entry point for API is `ClickHouseStatement#write()` method.

#### Importing file into table

```java
import ru.yandex.clickhouse.ClickHouseStatement;
ClickHouseStatement sth = connection.createStatement();
sth
.write() // Write API entrypoint
.table("default.my_table") // where to write data
.option("format_csv_delimiter", ";") // specific param
.data(new File("/path/to/file.csv.gz"), ClickHouseFormat.CSV, ClickHouseCompression.gzip) // specify input
.data(new File("/path/to/file.csv.gz"), ClickHouseFormat.CSV, ClickHouseCompression.gzip) // specify input
.send();
```

#### Configurable send

```java
import ru.yandex.clickhouse.ClickHouseStatement;
ClickHouseStatement sth = connection.createStatement();
sth
.write()
.sql("INSERT INTO default.my_table (a,b,c)")
.data(new MyCustomInputStream(), ClickHouseFormat.JSONEachRow)
.dataCompression(ClickHouseCompression.brotli)
.dataCompression(ClickHouseCompression.brotli)
.addDbParam(ClickHouseQueryParam.MAX_PARALLEL_REPLICAS, 2)
.send();
```

#### Send data in binary formatted with custom user callback

```java
import ru.yandex.clickhouse.ClickHouseStatement;
ClickHouseStatement sth = connection.createStatement();
Expand All @@ -93,19 +152,40 @@ sth.write().send("INSERT INTO test.writer", new ClickHouseStreamCallback() {
ClickHouseFormat.RowBinary); // RowBinary or Native are supported
```

## Compatibility

Java 8 or higher is required in order to use Java client and/or JDBC driver.

### Data Format

### Supported Server Versions
All [active releases](../ClickHouse/pulls?q=is%3Aopen+is%3Apr+label%3Arelease) are supported. You can still use the driver for older versions like 18.14 or 19.16 but please keep in mind that they're no longer supported.
TODO

### Data Type

### Compiling with maven
The driver is built with maven.
`mvn package -DskipTests=true`
TODO: a table

To build a jar with dependencies use
### Server Version

`mvn package assembly:single -DskipTests=true`
All [active releases](../ClickHouse/pulls?q=is%3Aopen+is%3Apr+label%3Arelease) are supported. You can still use the JDBC driver for older versions like 18.14 or 19.16 but please keep in mind that they're no longer supported.

## Build with Maven

### Build requirements
In order to build the jdbc client one need to have jdk 1.8 or higher.
### JDK 8

Use below command line to compile, test and generate packages.
`mvn clean verify`

### JDK 11+

You need a [toolchains.xml](https://maven.apache.org/guides/mini/guide-using-toolchains.html) under `<home directory>/.m2/`, as multi-release jar will be generated.

`mvn -Drelease clean verify`

## Testing

By default, docker container will be created automatically for integration test. In case you need to test against an existing server, you may put `test.properties` under either test/resources or <home directory>/.m2/clickhouse with content like below:

```properties
clickhouseServer=127.0.0.1
clickhouseVersion=21.9
```
92 changes: 35 additions & 57 deletions clickhouse-benchmark/pom.xml
View file
Original file line number Diff line number Diff line change
@@ -1,9 +1,8 @@
<project xmlns="http://maven.apache.org/POM/4.0.0"
xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance" xsi:schemaLocation="http://maven.apache.org/POM/4.0.0 http://maven.apache.org/xsd/maven-4.0.0.xsd">
<project xmlns="http://maven.apache.org/POM/4.0.0" xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance" xsi:schemaLocation="http://maven.apache.org/POM/4.0.0 http://maven.apache.org/xsd/maven-4.0.0.xsd">
<modelVersion>4.0.0</modelVersion>

<parent>
<groupId>tech.clickhouse</groupId>
<groupId>com.clickhouse</groupId>
<artifactId>clickhouse-java</artifactId>
<version>${revision}</version>
</parent>
Expand All @@ -19,11 +18,11 @@
<properties>
<clickhouse4j-driver.version>1.4.4</clickhouse4j-driver.version>
<mariadb-driver.version>2.7.3</mariadb-driver.version>
<mysql-driver.version>8.0.25</mysql-driver.version>
<mysql-driver.version>8.0.26</mysql-driver.version>
<native-driver.version>2.5.6</native-driver.version>
<postgresql-driver.version>42.2.22</postgresql-driver.version>
<postgresql-driver.version>42.2.23</postgresql-driver.version>
<project.build.sourceEncoding>UTF-8</project.build.sourceEncoding>
<jmh.version>1.32</jmh.version>
<jmh.version>1.33</jmh.version>
<shade.name>benchmarks</shade.name>
</properties>

Expand All @@ -40,6 +39,32 @@
<scope>provided</scope>
</dependency>

<dependency>
<groupId>${project.parent.groupId}</groupId>
<artifactId>clickhouse-client</artifactId>
<version>${revision}</version>
</dependency>
<dependency>
<groupId>${project.parent.groupId}</groupId>
<artifactId>clickhouse-grpc-client</artifactId>
<version>${revision}</version>
<!-- exclusions>
<exclusion>
<groupId>${project.parent.groupId}</groupId>
<artifactId>io.grpc</artifactId>
</exclusion>
</exclusions -->
</dependency>
<dependency>
<groupId>io.grpc</groupId>
<artifactId>grpc-netty-shaded</artifactId>
</dependency>
<!-- separate classifier did not work well with flatten plugin -->
<dependency>
<groupId>io.grpc</groupId>
<artifactId>grpc-okhttp</artifactId>
</dependency>

<!-- JDBC drivers -->
<dependency>
<groupId>ru.yandex.clickhouse</groupId>
Expand Down Expand Up @@ -116,15 +141,6 @@

<build>
<plugins>
<plugin>
<groupId>org.apache.maven.plugins</groupId>
<artifactId>maven-compiler-plugin</artifactId>
<configuration>
<compilerVersion>${jdk.version}</compilerVersion>
<source>${jdk.version}</source>
<target>${jdk.version}</target>
</configuration>
</plugin>
<plugin>
<groupId>org.apache.maven.plugins</groupId>
<artifactId>maven-shade-plugin</artifactId>
Expand All @@ -140,12 +156,14 @@
<transformer implementation="org.apache.maven.plugins.shade.resource.ManifestResourceTransformer">
<mainClass>org.openjdk.jmh.Main</mainClass>
</transformer>
<transformer implementation="org.apache.maven.plugins.shade.resource.ServicesResourceTransformer"/>
<transformer implementation="org.apache.maven.plugins.shade.resource.ServicesResourceTransformer" />
</transformers>
<filters>
<filter>
<artifact>*:*</artifact>
<excludes>
<exclude>module-info.class</exclude>
<exclude>META-INF/MANIFEST.MF</exclude>
<exclude>META-INF/*.SF</exclude>
<exclude>META-INF/*.DSA</exclude>
<exclude>META-INF/*.RSA</exclude>
Expand All @@ -157,45 +175,5 @@
</executions>
</plugin>
</plugins>
<pluginManagement>
<plugins>
<plugin>
<artifactId>maven-clean-plugin</artifactId>
<version>2.5</version>
</plugin>
<plugin>
<artifactId>maven-deploy-plugin</artifactId>
<version>2.8.1</version>
</plugin>
<plugin>
<artifactId>maven-install-plugin</artifactId>
<version>2.5.1</version>
</plugin>
<plugin>
<artifactId>maven-jar-plugin</artifactId>
<version>2.4</version>
</plugin>
<plugin>
<artifactId>maven-javadoc-plugin</artifactId>
<version>2.9.1</version>
</plugin>
<plugin>
<artifactId>maven-resources-plugin</artifactId>
<version>2.6</version>
</plugin>
<plugin>
<artifactId>maven-site-plugin</artifactId>
<version>3.3</version>
</plugin>
<plugin>
<artifactId>maven-source-plugin</artifactId>
<version>2.2.1</version>
</plugin>
<plugin>
<artifactId>maven-surefire-plugin</artifactId>
<version>2.17</version>
</plugin>
</plugins>
</pluginManagement>
</build>
</project>
</project>
Loading

0 comments on commit 007d167

Please sign in to comment.