Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Apache Drill support #2157

Open
dcoueron opened this issue Mar 16, 2016 · 129 comments

Comments

@dcoueron
Copy link

@dcoueron dcoueron commented Mar 16, 2016

Hi,

Hive / Apache Drill support would be great !

Best regards,
Damien

⬇️ Please click the 👍 reaction instead of leaving a +1 or 👍 comment

@agilliland

This comment has been minimized.

Copy link
Contributor

@agilliland agilliland commented Mar 16, 2016

We implement support for new drivers based on community support, so if there are clear signs the community is interested in this database we'll consider building it.

@MrMauricioLeite

This comment was marked as off-topic.

Copy link

@MrMauricioLeite MrMauricioLeite commented Mar 16, 2016

+1

@camsaul camsaul changed the title Hive or Drill support (Generic JDBC connectivity ?) Hive / Apache Drill support Apr 11, 2016
@MrMauricioLeite

This comment has been minimized.

Copy link

@MrMauricioLeite MrMauricioLeite commented Apr 18, 2016

This feature would really help on accessing our datalake directly.
I just found it again while searching if anyone opened a Hive connection proposal.

@NahumLitvin

This comment was marked as off-topic.

Copy link

@NahumLitvin NahumLitvin commented Apr 18, 2016

+1 for this

@leoatavalancha

This comment was marked as spam.

Copy link

@leoatavalancha leoatavalancha commented May 9, 2016

+1

6 similar comments
@silverma

This comment was marked as spam.

Copy link

@silverma silverma commented May 10, 2016

+1

@nabilblk

This comment was marked as spam.

Copy link

@nabilblk nabilblk commented May 24, 2016

+1

@inpefess

This comment was marked as spam.

Copy link

@inpefess inpefess commented Jun 3, 2016

+1

@beljun

This comment was marked as spam.

Copy link

@beljun beljun commented Jun 17, 2016

+1

@adityareddy

This comment was marked as spam.

Copy link

@adityareddy adityareddy commented Jul 10, 2016

+1

@giaosudau

This comment was marked as spam.

Copy link

@giaosudau giaosudau commented Jul 18, 2016

+1

@lm2s

This comment was marked as spam.

Copy link

@lm2s lm2s commented Jul 19, 2016

+2 😁
Currently we're investigating the best way of how to fuse all our datastores to then analyse the data with Metabase. Apache Drill support directly in Metabase would be awesome.

@joshwalters

This comment was marked as spam.

Copy link

@joshwalters joshwalters commented Aug 8, 2016

+1 for Hive

@hainesc

This comment was marked as spam.

Copy link

@hainesc hainesc commented Aug 30, 2016

+1 for Hive, thanks.

@yank1

This comment was marked as spam.

Copy link

@yank1 yank1 commented Aug 30, 2016

+1

@QLeelulu

This comment was marked as spam.

Copy link

@QLeelulu QLeelulu commented Aug 31, 2016

+1 for Hive & Apache Drill

@svairavelu

This comment was marked as spam.

Copy link

@svairavelu svairavelu commented Sep 19, 2016

+1

@julien-gm

This comment was marked as spam.

Copy link

@julien-gm julien-gm commented Oct 3, 2016

+1

2 similar comments
@xiqiangchen

This comment was marked as spam.

Copy link

@xiqiangchen xiqiangchen commented Oct 12, 2016

+1

@sundy-li

This comment was marked as spam.

Copy link

@sundy-li sundy-li commented Oct 12, 2016

+1

@yqylovy

This comment was marked as spam.

Copy link

@yqylovy yqylovy commented Oct 12, 2016

+++++++++++1

@chenzhixi

This comment was marked as spam.

Copy link

@chenzhixi chenzhixi commented Oct 12, 2016

+1

@vsnuvt

This comment was marked as spam.

Copy link

@vsnuvt vsnuvt commented Oct 27, 2016

+1 for drill.

@manujainz

This comment was marked as spam.

Copy link

@manujainz manujainz commented Nov 4, 2016

+1 for Drill

@karthikg001

This comment was marked as spam.

Copy link

@karthikg001 karthikg001 commented Nov 4, 2016

Metabase solves the pain of BI / Dashboarding, the way Apache Drill solves querying.
Metabase + Drill will be a powerful combination.

+1 Much Much needed.

@edklimov

This comment was marked as spam.

Copy link

@edklimov edklimov commented Nov 8, 2016

+1 for Hive

@wjoel

This comment has been minimized.

Copy link
Contributor

@wjoel wjoel commented Nov 2, 2017

@witwall

This comment has been minimized.

Copy link

@witwall witwall commented Nov 29, 2017

@wjoel can you provide this plugin as a separate one, without metabase?

@wjoel

This comment has been minimized.

Copy link
Contributor

@wjoel wjoel commented Nov 29, 2017

@witwall as far as I know, Metabase does not support plugins (and my patch set is not a plugin). I expect this to be merged into Metabase proper, or not at all.

@suganda7

This comment was marked as spam.

Copy link

@suganda7 suganda7 commented Mar 8, 2018

+1

@markac

This comment has been minimized.

Copy link

@markac markac commented Mar 14, 2018

@wjoel Have you had any success with merging this with metabase?

@wjoel

This comment has been minimized.

Copy link
Contributor

@wjoel wjoel commented Mar 14, 2018

@markac Won't have time to work on this until next week, unfortunately.

@markac

This comment has been minimized.

Copy link

@markac markac commented Mar 14, 2018

@wjoel Thanks for the update. I'm just exploring Metabase and was more wondering if you'd made any progress with merging into Metabase proper - Spark connectivity is pretty important to me

@lucasloami

This comment was marked as spam.

Copy link

@lucasloami lucasloami commented Mar 19, 2018

+1

@zhxuan

This comment was marked as spam.

Copy link

@zhxuan zhxuan commented Mar 26, 2018

+1 hive

@lucasloami

This comment has been minimized.

Copy link

@lucasloami lucasloami commented Mar 26, 2018

It seems that this issue was addressed in #4645 and it's already finished, tested and have no failures. So I believe it's only a matter of merge the PR =)

@camsaul

This comment has been minimized.

Copy link
Member

@camsaul camsaul commented Apr 12, 2018

Just merged PR #7323 (based on #4645); Spark support is shipping with our upcoming 0.29.0 release. 🎉

@ghost

This comment has been minimized.

Copy link

@ghost ghost commented Apr 13, 2018

@camsaul .. great to hear Spark Sql support is available from 0.29.0. Spark Sql is kind of blocker for us to use Metabase. Any tentative date on 0.29.0 release ?

@camsaul

This comment has been minimized.

Copy link
Member

@camsaul camsaul commented Apr 13, 2018

Not 100% sure but it should be within the next few weeks.

@lucasloami

This comment has been minimized.

Copy link

@lucasloami lucasloami commented Apr 23, 2018

Shouldn't this issue be closed since the feature was already implemented?

@jornh

This comment has been minimized.

Copy link
Contributor

@jornh jornh commented Apr 23, 2018

@lucasloami it probably should (For SparkSQL - and as I understood Hive as well?). But Drill support, mentioned in the title of this issue, ended up being a blocker for merge, see #4645 (comment). It was therefore reverted out (in b2ced45), as the Drill lib uses an unsupported Java 8 API which was removed in Java 9 - so support for Drill is still pending. It’s makes a lot of sense that the Metabase core devs don’t want to include such dependencies.

The proper thing would IMO be to adjust the title here - and then either open a new separate issue for Drill support (which is then blocked on the upstream Java 9 issue) - or lump it together with #5562 Dremio which is fork of Drill. The Dremio codebase BTW seems to have the same Java 9 problem (https://github.com/dremio/dremio-oss/blob/8754537fe040374c67cfbbf76dac4a7d8d2c20c7/common/src/main/java/com/dremio/common/config/SabotConfig.java#L49) ... but I’d better stop ranting here now ...

@allansene

This comment has been minimized.

Copy link

@allansene allansene commented May 2, 2018

Anyone here was able to test? I'm trying but I'm getting this error 😞

@wjoel

This comment has been minimized.

Copy link
Contributor

@wjoel wjoel commented May 2, 2018

@lucasloami

This comment has been minimized.

Copy link

@lucasloami lucasloami commented May 2, 2018

Same error here, @allansene

@camsaul

This comment has been minimized.

Copy link
Member

@camsaul camsaul commented May 15, 2018

@allansene @lucasloami we ran into some issues including SparkSQL dependencies in metabase.jar because of how large they were. We separated them out into a separate JAR. To get SparkSQL working, please:

  • Upgrade to Metabase 0.29.3 or higher
  • Follow the steps here to download the dependencies JAR and put it where Metabase can find it

Let me know if you have any more questions!

@Micka33

This comment has been minimized.

Copy link

@Micka33 Micka33 commented Nov 20, 2018

Hi! Is there a way for me to query Apache Drill?

@jackchongs

This comment has been minimized.

Copy link

@jackchongs jackchongs commented Nov 29, 2018

apache/drill#1446 apache drill jdbc could run with jdk 9 now。 Anything Update ?

@AnonyV

This comment has been minimized.

Copy link

@AnonyV AnonyV commented Dec 19, 2018

Hi there,
im new in Clojure,
Unfortunately, this SparkSQL driver doesn't work with my hiveserver.
I found that only jdbc:hive but not jdbc:hive2 would be worked.
so maybe the root cause is "org.apache.hive.jdbc", how can i use "org.apache.hadoop.hive.jdbc" instead?
any idea of this?

@AnonyV

This comment has been minimized.

Copy link

@AnonyV AnonyV commented Dec 19, 2018

@allansene @lucasloami we ran into some issues including SparkSQL dependencies in metabase.jar because of how large they were. We separated them out into a separate JAR. To get SparkSQL working, please:

  • Upgrade to Metabase 0.29.3 or higher
  • Follow the steps here to download the dependencies JAR and put it where Metabase can find it

Let me know if you have any more questions!

Unfortunately, it doesn't work on my old Hiveserver

12-19 09:11:57 DEBUG metabase.middleware :: POST /api/database 400 (5 s) (0 DB calls).
{:valid false, :dbname "Timed out after 5000 milliseconds.", :message "Timed out after 5000 milliseconds."}
12-19 09:13:08 ERROR metabase.driver :: Failed to connect to database: java.sql.SQLException: Could not open client transport with JDBC Uri: jdbc:hive2://172.0.0.11:8080/test: Invalid status 72

any idea of it

@camsaul camsaul changed the title Hive / Apache Drill support Apache Drill support Mar 29, 2019
@camsaul

This comment has been minimized.

Copy link
Member

@camsaul camsaul commented Mar 29, 2019

For anyone who stumbles across this thread, in Metabase 0.32.0 and newer it is not longer neccessary to download a separate dependencies JAR to use Metabase with Spark SQL

@binodsarkar

This comment has been minimized.

Copy link

@binodsarkar binodsarkar commented Apr 8, 2019

I am trying to write apache drill driver and now able to use native query. But schema sync not working and I am not finding how to write a own sync process for driver.

@mazameli mazameli added Type:New Feature and removed .Proposal labels Oct 9, 2019
@dobesv

This comment has been minimized.

Copy link

@dobesv dobesv commented Nov 10, 2019

Is it possible to merge in the previously created drill support now?

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
You can’t perform that action at this time.