Skip to content
Snowflake Connector for Dremio using the ARP SDK.
Java TSQL
Branch: master
Clone or download
narendrans Merge pull request #16 from ParensAllTheWayDown/groupId-change
The groupId com.dremio.plugin (singular) installs to
Latest commit 3a5d616 Aug 14, 2019
Permalink
Type Name Latest commit message Commit time
Failed to load latest commit information.
.github/ISSUE_TEMPLATE Create enhancement.md Jul 12, 2019
src/main rewrite extract functions Aug 14, 2019
tpch All tpch queries pass with full pushdown except 13,15,22 Jul 11, 2019
.gitignore initial commit Jul 9, 2019
.travis.yml
LICENSE initial commit Jul 9, 2019
README.md Add Codacy badge Aug 13, 2019
pom.xml The groupId com.dremio.plugin (singular) installs to Aug 14, 2019
snowflake.gif update readme Jul 9, 2019

README.md

Dremio Snowflake Connector

Codacy Badge Build Status Last Commit

License Platform

Overview

This is a community based Snowflake Dremio connector made using the ARP framework. Check Dremio Hub for more examples and ARP Docs for documentation.

This is not an official Dremio connector. THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND.

Demo

Snowflake demo

Downloading a Release

Usage

Creating a new Snowflake Source

Required Parameters

  • JDBC URL
    • Ex: jdbc❄️//<account_name>.snowflakecomputing.com/?param1=value&param2=value. More details.
  • Username, Password
    • The username and password with which you want to connect to Snowflake

Development

Building and Installation

  1. In root directory with the pom.xml file run mvn clean install
  2. Take the resulting .jar file in the target folder and put it in the <DREMIO_HOME>\jars folder in Dremio
  3. Download the Snowflake JDBC driver from (https://mvnrepository.com/artifact/net.snowflake/snowflake-jdbc/3.8.6 and click on the JAR link) and put in in the <DREMIO_HOME>\jars\3rdparty folder
  4. Restart Dremio

Building a Docker image

Dockerfile:

FROM dremio/dremio-oss
USER root
RUN cd /opt/dremio/jars && wget https://repo1.maven.org/maven2/net/snowflake/snowflake-jdbc/3.8.8/snowflake-jdbc-3.8.8.jar && wget https://github.com/narendrans/dremio-snowflake/releases/download/1.2/dremio-snowflake-plugin-3.3.1-201907291852280797-df23756.jar && chown dremio snowflake-jdbc-3.8.8.jar dremio-snowflake-plugin-3.3.1-201907291852280797-df23756.jar

Build:

docker build . -t dremio-snowflake

Run:

docker run -p 9047:9047 -p 31010:31010 dremio-snowflake

Debugging

To debug pushdowns for queries set the following line in logback.xml

  <logger name="com.dremio.exec.store.jdbc">
    <level value="${dremio.log.level:-trace}"/>
  </logger>

You can then notice lines like below in server.log file after which you can revist the YAML file to add pushdowns based on Snowflake SQL Reference:

- 2019-07-11 18:56:24,001 [22d879a7-ce3d-f2ca-f380-005a88865700/0:foreman-planning] DEBUG c.d.e.store.jdbc.dialect.arp.ArpYaml - Operator / not supported. Aborting pushdown.

You can also take a look at the planning tab/visualized plan of the profile to determine if everything is pushed down or not.

Contribution

Submitting an issue

Pull Requests

PRs are welcome. When submitting a PR make sure of the following:

  • Try to follow Google's Java style coding when modifying/creating Java related content.
  • Use a YAML linter to check the syntactic correctness of YAML file
  • Make sure the build passes
  • Run basic queries at least to ensure things are working properly
You can’t perform that action at this time.