Skip to content

Commit

Permalink
Add ClickHouse Connector
Browse files Browse the repository at this point in the history 
Co-authored-by: Piotr Findeisen <piotr.findeisen@gmail.com>
  • Loading branch information
@wgzhao @findepi
wgzhao and findepi committed Mar 2, 2021
1 parent 4d34ca5 commit 174d429
Show file tree
Hide file tree
Showing 18 changed files with 1,645 additions and 1 deletion.
6 changes: 6 additions & 0 deletions core/trino-server/src/main/provisio/presto.xml
View file
Original file line number Diff line number Diff line change
Expand Up @@ -56,6 +56,12 @@
</artifact>
</artifactSet>

<artifactSet to="plugin/clickhouse">
<artifact id="${project.groupId}:trino-clickhouse:zip:${project.version}">
<unpack />
</artifact>
</artifactSet>

<artifactSet to="plugin/example-http">
<artifact id="${project.groupId}:trino-example-http:zip:${project.version}">
<unpack />
Expand Down
1 change: 1 addition & 0 deletions docs/src/main/sphinx/connector.rst
View file
Original file line number Diff line number Diff line change
Expand Up @@ -12,6 +12,7 @@ from different data sources.
BigQuery <connector/bigquery>
Black Hole <connector/blackhole>
Cassandra <connector/cassandra>
ClickHouse <connector/clickhouse>
Druid <connector/druid>
Elasticsearch <connector/elasticsearch>
Google Sheets <connector/googlesheets>
Expand Down
73 changes: 73 additions & 0 deletions docs/src/main/sphinx/connector/clickhouse.rst
View file
Original file line number Diff line number Diff line change
@@ -0,0 +1,73 @@
====================
ClickHouse Connector
====================

The ClickHouse connector allows querying tables in an external
`Yandex ClickHouse <https://clickhouse.tech/>`_ instance. This can be used to join data between different
systems like ClickHouse and Hive, or between two different ClickHouse instances.

Configuration
-------------

To configure the ClickHouse connector, create a catalog properties file``etc/catalog/clickhouse.properties``,
replace the connection properties as needed for your setup:

.. code-block:: none
connector.name=clickhouse
connection-url=jdbc:clickhouse://host1:8123/
connection-user=default
connection-password=
Multiple ClickHouse servers
^^^^^^^^^^^^^^^^^^^^^^^^^^^

If you have multiple ClickHouse servers you need to configure one catalog for each instance.
To add another catalog:

* Add another properties file to ``etc/catalog``
* Save it with a different name that ends in ``.properties``

For example, if you name the property file ``sales.properties``, Trino uses the configured
connector to create a catalog named ``sales``.

Querying ClickHouse
-------------------

The ClickHouse connector provides a schema for every ClickHouse *database*.
run ``SHOW SCHEMAS`` to see the available ClickHouse databases::

SHOW SCHEMAS FROM clickhouse;

If you have a ClickHouse database named ``web``, run ``SHOW TABLES`` to view the tables
in this database::

SHOW TABLES FROM clickhouse.web;

Run ``DESCRIBE`` or ``SHOW COLUMNS`` to list the columns in the ``clicks`` table in the
``web`` databases::

DESCRIBE clickhouse.web.clicks;
SHOW COLUMNS FROM clickhouse.web.clicks;

Run ``SELECT`` to access the ``clicks`` table in the ``web`` database::

SELECT * FROM clickhouse.web.clicks;

.. note::

If you used a different name for your catalog properties file, use
that catalog name instead of ``ClickHouse`` in the above examples.


ClickHouse Connector Limitations
--------------------------------

The following SQL statements aren't supported:

* :doc:`/sql/grant`
* :doc:`/sql/revoke`
* :doc:`/sql/show-grants`
* :doc:`/sql/show-roles`
* :doc:`/sql/show-role-grants`
176 changes: 176 additions & 0 deletions plugin/trino-clickhouse/pom.xml
View file
Original file line number Diff line number Diff line change
@@ -0,0 +1,176 @@
<?xml version="1.0" encoding="UTF-8"?>
<project xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance" xmlns="http://maven.apache.org/POM/4.0.0" xsi:schemaLocation="http://maven.apache.org/POM/4.0.0 http://maven.apache.org/xsd/maven-4.0.0.xsd">
<modelVersion>4.0.0</modelVersion>

<parent>
<groupId>io.trino</groupId>
<artifactId>trino-root</artifactId>
<version>353-SNAPSHOT</version>
<relativePath>../../pom.xml</relativePath>
</parent>

<artifactId>trino-clickhouse</artifactId>
<description>Trino - ClickHouse Connector</description>
<packaging>trino-plugin</packaging>

<properties>
<air.main.basedir>${project.parent.basedir}</air.main.basedir>
</properties>

<dependencies>
<dependency>
<groupId>io.trino</groupId>
<artifactId>trino-base-jdbc</artifactId>
</dependency>

<dependency>
<groupId>io.airlift</groupId>
<artifactId>configuration</artifactId>
</dependency>

<dependency>
<groupId>io.airlift</groupId>
<artifactId>log</artifactId>
</dependency>

<dependency>
<groupId>io.airlift</groupId>
<artifactId>log-manager</artifactId>
</dependency>

<dependency>
<groupId>com.google.code.findbugs</groupId>
<artifactId>jsr305</artifactId>
</dependency>

<dependency>
<groupId>com.google.guava</groupId>
<artifactId>guava</artifactId>
</dependency>

<dependency>
<groupId>com.google.inject</groupId>
<artifactId>guice</artifactId>
</dependency>

<dependency>
<groupId>javax.inject</groupId>
<artifactId>javax.inject</artifactId>
</dependency>

<dependency>
<groupId>ru.yandex.clickhouse</groupId>
<artifactId>clickhouse-jdbc</artifactId>
<version>0.2.4</version>
<exclusions>
<exclusion>
<groupId>org.slf4j</groupId>
<artifactId>jcl-over-slf4j</artifactId>
</exclusion>
<exclusion>
<groupId>commons-logging</groupId>
<artifactId>commons-logging</artifactId>
</exclusion>
</exclusions>
</dependency>

<!-- Trino SPI -->
<dependency>
<groupId>io.trino</groupId>
<artifactId>trino-spi</artifactId>
<scope>provided</scope>
</dependency>

<dependency>
<groupId>io.airlift</groupId>
<artifactId>slice</artifactId>
<scope>provided</scope>
</dependency>

<dependency>
<groupId>com.fasterxml.jackson.core</groupId>
<artifactId>jackson-annotations</artifactId>
<scope>provided</scope>
</dependency>

<dependency>
<groupId>org.openjdk.jol</groupId>
<artifactId>jol-core</artifactId>
<scope>provided</scope>
</dependency>

<!-- for testing -->
<dependency>
<groupId>io.trino</groupId>
<artifactId>trino-main</artifactId>
<scope>test</scope>
</dependency>

<dependency>
<groupId>io.trino</groupId>
<artifactId>trino-main</artifactId>
<type>test-jar</type>
<scope>test</scope>
</dependency>

<dependency>
<groupId>io.trino</groupId>
<artifactId>trino-testing</artifactId>
<scope>test</scope>
</dependency>

<dependency>
<groupId>io.trino</groupId>
<artifactId>trino-tpch</artifactId>
<scope>test</scope>
</dependency>

<dependency>
<groupId>io.trino.tpch</groupId>
<artifactId>tpch</artifactId>
<scope>test</scope>
</dependency>

<dependency>
<groupId>io.airlift</groupId>
<artifactId>testing</artifactId>
<scope>test</scope>
</dependency>

<dependency>
<groupId>org.assertj</groupId>
<artifactId>assertj-core</artifactId>
<scope>test</scope>
</dependency>

<dependency>
<groupId>org.jetbrains</groupId>
<artifactId>annotations</artifactId>
<scope>test</scope>
</dependency>

<dependency>
<groupId>org.testcontainers</groupId>
<artifactId>clickhouse</artifactId>
<scope>test</scope>
</dependency>

<dependency>
<groupId>org.testcontainers</groupId>
<artifactId>jdbc</artifactId>
<scope>test</scope>
</dependency>

<dependency>
<groupId>org.testcontainers</groupId>
<artifactId>testcontainers</artifactId>
<scope>test</scope>
</dependency>

<dependency>
<groupId>org.testng</groupId>
<artifactId>testng</artifactId>
<scope>test</scope>
</dependency>
</dependencies>
</project>
Loading

0 comments on commit 174d429

Please sign in to comment.