conn.cursor(): impala.error.HiveServer2Error

[paulocheque]

• impyla==0.14.0
• Python 3.6

Sorry I am newbie on this. Any ideas?

from impala.dbapi import connect
conn = connect(host='my ip', port=21050) # it works
# or conn = connect(host='my ip', port=21000) # it works
cursor = conn.cursor() # raise an error
cursor = conn.cursor(user='some user') # raise an error

env/lib/python3.6/site-packages/impala/hiveserver2.py", line 125, in cursor
    session = self.service.open_session(user, configuration)
env/lib/python3.6/site-packages/impala/hiveserver2.py", line 998, in open_session
    resp = self._rpc('OpenSession', req)
env/lib/python3.6/site-packages/impala/hiveserver2.py", line 924, in _rpc
    response = self._execute(func_name, request)
env/lib/python3.6/site-packages/impala/hiveserver2.py", line 957, in _execute
    .format(self.retries))
impala.error.HiveServer2Error: Failed after retrying 3 times

Internally, it raise this error on this line session = self.service.open_session(user, configuration)

[OneCricketeer]

The defaults for the parameters you listed are

host='localhost', port=21050, and user=None.

I see that both statements that work you give 'my ip', so if we assume that you are passing in 'localhost' there, then the only difference would be the user does not have permissions.

Otherwise, you are giving a non-local host parameter, and therefore communicating with a remote machine which happily accepts the connection.

Does that all make sense?

[paulocheque]

First, thanks for the fast response!

By 'my ip' I mean: a IP for a remote machine.

It could be related with the user. What is the difference between user in the connection method and in the cursor method?

The hard part is that the error message is not clear enough to identify what is happening. So I am not sure this is a network or authentication issue.

Thanks again

[OneCricketeer]

Seeing as conn = connect(host='my ip', port=21050) # it works, then it is unlikely a connection issue.

Are you able to SSH into this machine to run any query through impala-shell without explicitly being the impala user ; and/or belonging to an impala group?

Have you also tried passing user='name' into the connection? Did that work, or not?

If you don't specify the value, it will use the name of the user executing the code in the current session, retreived from the OS

[paulocheque]

Thanks for the help @Cricket007

I configured log for DEBUG mode then I got more messages.

I guess it is a connection issue instead of an authorization issue, because I am not receiving any error related to the user/pass. Does that makes sense? Maybe a firewall rule?

DEBUG:impala.hiveserver2:Connecting to HiveServer2 SOME_REMOTE_IP:21050 with NOSASL authentication mechanism
DEBUG:impala._thrift_api:get_socket: host=SOME_REMOTE_IP port=21050 use_ssl=False ca_cert=None
DEBUG:impala._thrift_api:get_transport: socket=<thriftpy.transport.socket.TSocket object at 0x101314400> host=SOME_REMOTE_IP kerberos_service_name=impala auth_mechanism=NOSASL user=None password=fuggetaboutit
DEBUG:impala.hiveserver2:sock=<thriftpy.transport.socket.TSocket object at 0x101314400> transport=<thriftpy.transport.buffered.TBufferedTransport object at 0x1013145c0> protocol=<thriftpy.protocol.binary.TBinaryProtocol object at 0x103820f28> service=<thriftpy.thrift.TClient object at 0x10363f978>
DEBUG:impala.hiveserver2:HiveServer2Connection(service=<impala.hiveserver2.HS2Service object at 0x103823278>, default_db=None)

DEBUG:impala.hiveserver2:Getting a cursor (Impala session)
DEBUG:impala.hiveserver2:.cursor(): getting new session_handle
DEBUG:impala.hiveserver2:OpenSession: req=TOpenSessionReq(client_protocol=5, username='paulo-cheque', password=None, configuration=None)
DEBUG:impala.hiveserver2:Attempting to open transport (tries_left=3)
DEBUG:impala.hiveserver2:Transport opened
ERROR:impala.hiveserver2:Failed to open transport (tries_left=3)
Traceback (most recent call last):
File "env/lib/python3.6/site-packages/impala/hiveserver2.py", line 940, in _execute
return func(request)
File "env/lib/python3.6/site-packages/thriftpy/thrift.py", line 198, in _req
return self._recv(_api)
File "env/lib/python3.6/site-packages/thriftpy/thrift.py", line 210, in _recv
fname, mtype, rseqid = self._iprot.read_message_begin()
File "env/lib/python3.6/site-packages/thriftpy/protocol/binary.py", line 372, in read_message_begin
self.trans, strict=self.strict_read)
File "env/lib/python3.6/site-packages/thriftpy/protocol/binary.py", line 164, in read_message_begin
sz = unpack_i32(inbuf.read(4))
File "env/lib/python3.6/site-packages/thriftpy/transport/init.py", line 32, in read
return readall(self._read, sz)
File "env/lib/python3.6/site-packages/thriftpy/transport/init.py", line 14, in readall
chunk = read_fn(sz - have)
File "env/lib/python3.6/site-packages/thriftpy/transport/buffered/init.py", line 39, in _read
self._rbuf = BytesIO(self._trans.read(max(sz, self._buf_size)))
File "env/lib/python3.6/site-packages/thriftpy/transport/socket.py", line 125, in read
message='TSocket read 0 bytes')
thriftpy.transport.TTransportException: TTransportException(type=4, message='TSocket read 0 bytes')
DEBUG:impala.hiveserver2:Closing transport (tries_left=3)
DEBUG:impala.hiveserver2:Attempting to open transport (tries_left=2)
DEBUG:impala.hiveserver2:Transport opened
ERROR:impala.hiveserver2:Failed to open transport (tries_left=2)

[paulocheque]

To help others: It was a authentication issue. I added auth_mechanism='PLAIN'.

Thanks for the help @Cricket007 !

[y2k-shubham]

For Hive (hive-server2), is there a workaround if I have to use auth_mechanism='NOSASL'?

Because specifying NOSASL as auth_mechanism still gives the same error (meaning that whether or not I specify the auth_mechanism, it automatically defaults to the mechanism specified in my hive-site.xml)

[yerra]

when to use 'user' vs 'ldap_user' .we are given user and password but not sure if its ldap user or user ?

[huanqingdong]

DEBUG
@paulocheque I want to know how to config log for DEBUG mode,thank you !

[lss199411]

how to due it?

[mrajcok]

@huanqingdong

import logging
logging.basicConfig(level=logging.DEBUG)
from impala.dbapi import connect
conn = connect(...