Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Unable to set http header with thrift version 0.15.0 #417

Open
RamakrishnaChilaka opened this issue Oct 5, 2021 · 2 comments
Open

Unable to set http header with thrift version 0.15.0 #417

RamakrishnaChilaka opened this issue Oct 5, 2021 · 2 comments

Comments

@RamakrishnaChilaka
Copy link

No description provided.

@gthomas-slack
Copy link

We also found this using using PyHive 0.6.4 and thrift 0.15.0.

To replicate the issue you can create a hive session using a HTTP transport with the custom header that HiveServer2 requires for authorization:

from pyhive import hive
import base64
import thrift.transport.THttpClient

def thrift_http_transport():
    transport = thrift.transport.THttpClient.THttpClient(uri_or_host='http://localhost:12001/cliservice')

    auth_credentials = '{}:{}'.format('-', '-').encode('UTF-8')
    auth_credentials_base64 = base64.standard_b64encode(auth_credentials).decode('UTF-8')
    transport.setCustomHeaders(
        {
            'Authorization': 'Basic {}'.format(auth_credentials_base64),  # HiveServer2 BASIC auth
        }
    )
    return transport

conn = hive.connect(thrift_transport=thrift_http_transport())
cursor = conn.cursor()
cursor.execute("""SELECT SUM(1) from model.dim_date""")
data = cursor.fetchall()
cursor.close()
conn.close()
print(data)

With thrift 0.13.0 this code executes fine, but with 0.15.0 the following error/traceback is seen:

Traceback (most recent call last):
  File "test.py", line 17, in <module>
    conn = hive.connect(thrift_transport=thrift_http_transport())
  File "/versions/3.8.7/envs/lib/python3.8/site-packages/pyhive/hive.py", line 104, in connect
    return Connection(*args, **kwargs)
  File "/versions/3.8.7/envs/lib/python3.8/site-packages/pyhive/hive.py", line 249, in __init__
    response = self._client.OpenSession(open_session_req)
  File "/versions/3.8.7/envs/lib/python3.8/site-packages/TCLIService/TCLIService.py", line 186, in OpenSession
    self.send_OpenSession(req)
  File "/versions/3.8.7/envs/lib/python3.8/site-packages/TCLIService/TCLIService.py", line 195, in send_OpenSession
    self._oprot.trans.flush()
  File "/versions/3.8.7/envs/lib/python3.8/site-packages/thrift/transport/THttpClient.py", line 191, in flush
    self.__http.putheader('Cookie', self.headers['Set-Cookie'])
  File "/versions/3.8.7/lib/python3.8/http/client.py", line 1217, in putheader
    raise CannotSendHeader()
http.client.CannotSendHeader

This appears to be related to changes made for https://issues.apache.org/jira/browse/THRIFT-5165

Related PR: apache/thrift#2086
Related Commit: apache/thrift@69642f3

@csordasmarton
Copy link

There is an open issue on Jira which is related to this problem.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

No branches or pull requests

3 participants