Hive JDBC support #54
Comments
|
Not yet. It can be added using pyhive: https://github.com/dropbox/PyHive
Are you open to contributing the integration? I can guide you through it.
…On Mon, 27 Jan 2020 at 19:01, jayeshagwan1 ***@***.***> wrote:
Does piicatcher support Hive ?
—
You are receiving this because you are subscribed to this thread.
Reply to this email directly, view it on GitHub
<#54?email_source=notifications&email_token=AAMP7GSPTCETPUW52Q5OJ2TQ73OZXA5CNFSM4KMBIZ2KYY3PNVWWK3TUL52HS4DFUVEXG43VMWVGG33NNVSW45C7NFSM4II5M7TQ>,
or unsubscribe
<https://github.com/notifications/unsubscribe-auth/AAMP7GXDGU4BS3AXCWETMODQ73OZXANCNFSM4KMBIZ2A>
.
|
|
I am interested in contributing. |
|
Thanks! Install a developer version of piicatcher
Hive installationI am not sure about your tech setup. A web search should provide a lot of websites with instrutions to setup Hive. Load data into HiveI use a couple of simple datasets:
Add pyhiveAdd pyhive as a requirement in requirements.txt Rerun pipenv update to install pyhive. Write a explorerAn explorer is the base class for supporting different types of technologies. You'll have to:
I can answer any questions while you develop. |
|
Thanks @vrajat. Will follow the above steps. If any issue, will let you know. |
|
Above steps were followed. After running the command piicatcher --config hiveconfig.ini hive It seems its issue on windows system while installing pyhive. |
|
Try adding sasl to requirements.txt
…On Tue, 28 Jan 2020 at 20:21, jayeshagwan1 ***@***.***> wrote:
Above steps were followed. After running the command *piicatcher --config
hiveconfig.ini hive*
getting below error :
[image: image]
<https://user-images.githubusercontent.com/616970/73274446-93cd4c00-420b-11ea-9c96-fd076896ecfe.png>
It seems its issue on windows system while installing pyhive.
—
You are receiving this because you were mentioned.
Reply to this email directly, view it on GitHub
<#54?email_source=notifications&email_token=AAMP7GWFOYTN3XZIS33VAC3RABA7DA5CNFSM4KMBIZ2KYY3PNVWWK3TUL52HS4DFVREXG43VMVBW63LNMVXHJKTDN5WW2ZLOORPWSZGOEKDSKGQ#issuecomment-579282202>,
or unsubscribe
<https://github.com/notifications/unsubscribe-auth/AAMP7GXMGMJTCRL4TKU3M5TRABA7DANCNFSM4KMBIZ2A>
.
|
|
|
|
Does this help ? https://stackoverflow.com/a/58167572
…On Wed, Jan 29, 2020 at 9:58 AM jayeshagwan1 ***@***.***> wrote:
[image: image]
<https://user-images.githubusercontent.com/616970/73327919-c6179180-427d-11ea-8ead-966d2ee92113.png>
—
You are receiving this because you were mentioned.
Reply to this email directly, view it on GitHub
<#54?email_source=notifications&email_token=AAMP7GTOEO6FCTGN2RFIVW3RAEAVHA5CNFSM4KMBIZ2KYY3PNVWWK3TUL52HS4DFVREXG43VMVBW63LNMVXHJKTDN5WW2ZLOORPWSZGOEKF4NNQ#issuecomment-579585718>,
or unsubscribe
<https://github.com/notifications/unsubscribe-auth/AAMP7GTGMBGKELAIWFADKXDRAEAVHANCNFSM4KMBIZ2A>
.
|
|
Thanks. Now facing :
|
|
Which version of Hive have you installed ? Are you using HiveServer2 ?
…On Wed, Jan 29, 2020 at 11:48 AM jayeshagwan1 ***@***.***> wrote:
https://community.cloudera.com/t5/Support-Questions/pyhive-connection-error-thrift-transport-TTransport/td-p/206372
—
You are receiving this because you were mentioned.
Reply to this email directly, view it on GitHub
<#54?email_source=notifications&email_token=AAMP7GUOWNGUFAA23OZE5FDRAENR5A5CNFSM4KMBIZ2KYY3PNVWWK3TUL52HS4DFVREXG43VMVBW63LNMVXHJKTDN5WW2ZLOORPWSZGOEKGC2JY#issuecomment-579611943>,
or unsubscribe
<https://github.com/notifications/unsubscribe-auth/AAMP7GUYA6RAXP7UMHGSV7TRAENR5ANCNFSM4KMBIZ2A>
.
|
|
Hive2 |
|
Do you use HiveServer2 ?
…On Wed, Jan 29, 2020 at 12:05 PM jayeshagwan1 ***@***.***> wrote:
Hive2
—
You are receiving this because you were mentioned.
Reply to this email directly, view it on GitHub
<#54?email_source=notifications&email_token=AAMP7GULSZU2JMESGLRZ55DRAEPRRA5CNFSM4KMBIZ2KYY3PNVWWK3TUL52HS4DFVREXG43VMVBW63LNMVXHJKTDN5WW2ZLOORPWSZGOEKGDXGA#issuecomment-579615640>,
or unsubscribe
<https://github.com/notifications/unsubscribe-auth/AAMP7GUA43OEXF54EXOZF6TRAEPRRANCNFSM4KMBIZ2A>
.
|
|
yes |
|
Can you try connecting using pyhive in a python console ? Let's make sure
pyhive is working. For example,
conn = hive.Connection(host=host_name, port=port, username=user,
password=password,
database=database)
cur = conn.cursor()
cur.execute('select item_sk,reason_sk, account_credit from returns
limit 5')
result = cur.fetchall()
return result
…On Wed, Jan 29, 2020 at 12:34 PM jayeshagwan1 ***@***.***> wrote:
yes
—
You are receiving this because you were mentioned.
Reply to this email directly, view it on GitHub
<#54?email_source=notifications&email_token=AAMP7GTIQS5HOXOFXM76X7LRAETARA5CNFSM4KMBIZ2KYY3PNVWWK3TUL52HS4DFVREXG43VMVBW63LNMVXHJKTDN5WW2ZLOORPWSZGOEKGFN2Q#issuecomment-579622634>,
or unsubscribe
<https://github.com/notifications/unsubscribe-auth/AAMP7GUYQMRGX223ADCO5ITRAETARANCNFSM4KMBIZ2A>
.
|
|
Now able to connect to hiveserver2. But getting below error: raise ValueError("Password should be set if and only if in LDAP or CUSTOM mode; " ValueError: Password should be set if and only if in LDAP or CUSTOM mode; Remove password or use one of th ose modesCurrently I am passing auth='NOSASL' in connection. If I pass auth='Custom or none' then getting this error:
|
|
Can you confirm if these are errors when you try to connect to hive through python console ? No PIICatcher involved ? Can you confirm if you can connect to Hive and run queries from python console ? |
|
Sure. Will confirm. I think there similar open issues with pyhive also. Do we have other option for pyhive ? |
1 is probably the better option |
|
There is some issue with pyhive. I have tried with python, but still getting same error.
|
|
Is it specific to OS ? Haven't tried with linux or ubuntu yet. |
|
I am not sure. I've used in Centos and it worked. That was for a specific configuration of hive. OS or the configuration of python/hive can be the problem. Dont know how to help remotely with no knowledge about the setup. |
|
Can you try impyla ? |
|
Is this uses impala ? |
|
I am trying on centOS, but getting this error: [Errno 14] problem making ssl connection So could not install anything. Tried couple of things for ssl but its not working |
|
Impyla is supposed to work with hiveserver2 as well.
That seems like a rpm issue and not a pyhive issue ?
…On Wed, 5 Feb 2020 at 12:11, jayeshagwan1 ***@***.***> wrote:
I am trying on centOS, but getting this error:
[Errno 14] problem making ssl connection
Trying other mirror.
Error: Cannot retrieve repository metadata (repomd.xml) for repository:
bintray--sbt-rpm. Please verify its path and try again
So could not install anything. Tried couple of things for ssl but its not
working
—
You are receiving this because you were mentioned.
Reply to this email directly, view it on GitHub
<#54?email_source=notifications&email_token=AAMP7GQ2V6ATFMMA2O63J33RBJNQ7A5CNFSM4KMBIZ2KYY3PNVWWK3TUL52HS4DFVREXG43VMVBW63LNMVXHJKTDN5WW2ZLOORPWSZGOEK2KNSA#issuecomment-582264520>,
or unsubscribe
<https://github.com/notifications/unsubscribe-auth/AAMP7GQM3K3VPVKOUJ324XTRBJNQ7ANCNFSM4KMBIZ2A>
.
|
|
ftw superset uses pyhive. https://github.com/apache/incubator-superset/blob/master/superset/db_engine_specs/hive.py#L71 There are also hive related issues but in general it works. I still think there is something about your installation that pyhive does not work with. |
|
I will start working on Hive from next week and keep you posted. |
|
@jayeshagwan1 hello. I am wondering how this implementation go. it would be great if this feature can be added soon. |
|
There hasnt been any progress on this feature. IIRC @jayeshagwan1 got stuck in installing a test Hive cluster. @zer0pool will you be able to help out? |
|
closing this as there is not much demand for Hive. There is more interest in redshift, snowflake and Trino. |
Does piicatcher support Hive ?
The text was updated successfully, but these errors were encountered: