databricks.sql.exc.RequestError: Error during request to server

[PaulCornellDB]

Repro steps:

1. pipenv --python 3.8
2. pipenv shell
3. pip install databricks-sql-connector
4. export DATABRICKS_SERVER_HOSTNAME="https://<redacted>.cloud.databricks.com" && export DATABRICKS_HTTP_PATH="sql/protocolv1/o/<redacted>/<redacted>" && export DATABRICKS_TOKEN="dapi<redacted>"
5. Code (main.py):

from databricks import sql
import os

with sql.connect(server_hostname = os.getenv("DATABRICKS_SERVER_HOSTNAME"),
                 http_path       = os.getenv("DATABRICKS_HTTP_PATH"),
                 access_token    = os.getenv("DATABRICKS_TOKEN")) as connection:

  with connection.cursor() as cursor:
    cursor.execute("SELECT * FROM default.diamonds LIMIT 2")
    result = cursor.fetchall()

    for row in result:
      print(row)

6. python main.py
7. Traceback:

Traceback (most recent call last):
  File "main.py", line 10, in <module>
    with sql.connect(server_hostname = os.getenv("DATABRICKS_SERVER_HOSTNAME"),
  File "/Users/paul.cornell/.local/share/virtualenvs/paul.cornell-<redacted>/lib/python3.8/site-packages/databricks/sql/__init__.py", line 48, in connect
    return Connection(server_hostname, http_path, access_token, **kwargs)
  File "/Users/paul.cornell/.local/share/virtualenvs/paul.cornell-<redacted>/lib/python3.8/site-packages/databricks/sql/client.py", line 112, in __init__
    self._session_handle = self.thrift_backend.open_session(session_configuration, catalog,
  File "/Users/paul.cornell/.local/share/virtualenvs/paul.cornell-<redacted>/lib/python3.8/site-packages/databricks/sql/thrift_backend.py", line 341, in open_session
    response = self.make_request(self._client.OpenSession, open_session_req)
  File "/Users/paul.cornell/.local/share/virtualenvs/paul.cornell-<redacted>/lib/python3.8/site-packages/databricks/sql/thrift_backend.py", line 287, in make_request
    self._handle_request_error(error_info, attempt, elapsed)
  File "/Users/paul.cornell/.local/share/virtualenvs/paul.cornell-<redacted>/lib/python3.8/site-packages/databricks/sql/thrift_backend.py", line 199, in _handle_request_error
    raise network_request_error
databricks.sql.exc.RequestError: Error during request to server

8. Tried export CERT_PATH=$(python -m certifi) && export SSL_CERT_FILE=${CERT_PATH} && export REQUESTS_CA_BUNDLE=${CERT_PATH} and python main.py again, but same traceback.

Environment:

• Python 3.8.10
• pip list:

Package Version
certifi 2022.6.15
databricks-sql-connector 2.0.2
numpy 1.23.0
pandas 1.4.3
pip 21.0.1
pyarrow 8.0.0
python-dateutil 2.8.2
pytz 2022.1
setuptools 54.1.2
six 1.16.0
thrift 0.16.0
wheel 0.36.2

[zhangzuolin16]

dears,

is there any solution to fix this issue?

[susodapop]

As yet I'm not able to reproduce this behaviour in the connector. Even with this specified version of Python. My best recommendation is to try with a fresh installation of Python within a virtual environment.

[timon-schmelzer-gcx]

I see the same behavior since a few weeks. Before that, everything was fine and a connection could be established. Nothing changed in the code / the SQL endpoint (at least not from my side). Just updated databricks-sql-connector to 2.1.0 but the result is the same.

[timon-schmelzer-gcx]

...actually updating to 2.1.0 helped solving my issue! The error message changed from:

databricks.sql.thrift_backend: Error during request to server: {...}

to:

databricks.sql.thrift_backend: Error during request to server: : Invalid access token. {...}

So my databricks access token has been expired. After adding a new token to my script everything is running again 👍

[susodapop]

I'm closing this issue out due to its age. Feel free to reopen or make a new issue if a similar issue arrises.

[chozillla]

The issue still exists. I had success with an initial access token, but I am getting RequestError: Error during request to server: : Invalid access token. when I used a new token and revoked the old one.

[susodapop]

@chozillla Can you provide reproduction steps?

[chozillla]

@susodapop

Interestingly, it works fine when I explicitly put in the values. This is on a jupyter notebook on vscode via Anaconda.

[chozillla]

Uh, so I just exited vscode and restarted and it worked. Strange issue.

[NephilimJaeger]

I'm experiencing the same problem, but in my case, it only occurs when I increase the number of rows that will be written to the Databricks table (50 rows).

[vcakdwivedi]

Hi Everyone,
Is there any fix for above issue? I tried everything mentioned above.
main(server_hostname = args.server_hostname, http_path = args.http_path, schema = args.schema, access_token = args.access_token)
File "extract.py", line 24, in main
dbx_extractor.execute_sql(sql_query, server_hostname, http_path, access_token)
File "/mnt/azureml/cr/j/2c91464c9e0743f5860e8f48b407683d/exe/wd/dbextractor.py", line 23, in execute_sql
with db_sql.connect(
File "/azureml-envs/azureml_9806bfc7c716eaf0df12bea8512e961b/lib/python3.8/site-packages/databricks/sql/init.py", line 50, in connect
return Connection(server_hostname, http_path, access_token, **kwargs)
File "/azureml-envs/azureml_9806bfc7c716eaf0df12bea8512e961b/lib/python3.8/site-packages/databricks/sql/client.py", line 189, in init
self._session_handle = self.thrift_backend.open_session(
File "/azureml-envs/azureml_9806bfc7c716eaf0df12bea8512e961b/lib/python3.8/site-packages/databricks/sql/thrift_backend.py", line 464, in open_session
response = self.make_request(self._client.OpenSession, open_session_req)
File "/azureml-envs/azureml_9806bfc7c716eaf0df12bea8512e961b/lib/python3.8/site-packages/databricks/sql/thrift_backend.py", line 393, in make_request
self._handle_request_error(error_info, attempt, elapsed)
File "/azureml-envs/azureml_9806bfc7c716eaf0df12bea8512e961b/lib/python3.8/site-packages/databricks/sql/thrift_backend.py", line 261, in _handle_request_error
raise network_request_error
databricks.sql.exc.RequestError: Error during request to server

[susodapop]

Hi there, to investigate this we need reproduction steps. So far, every instance of this issue that we've traced has been an environment issue rather than a bug in the code. If you can provide reproduction steps we can try to guide you toward the fix. At the very least we need to know the exception text, since RequestError can point in many different directions.

…

On Fri, Jan 26, 2024 at 1:30 AM vcakdwivedi ***@***.***> wrote: Hi Everyone, Is there any fix for above issue? I tried everything mentioned above. main(server_hostname = args.server_hostname, http_path = args.http_path, schema = args.schema, access_token = args.access_token) File "extract.py", line 24, in main dbx_extractor.execute_sql(sql_query, server_hostname, http_path, access_token) File "/mnt/azureml/cr/j/2c91464c9e0743f5860e8f48b407683d/exe/wd/dbextractor.py", line 23, in execute_sql with db_sql.connect( File "/azureml-envs/azureml_9806bfc7c716eaf0df12bea8512e961b/lib/python3.8/site-packages/databricks/sql/ *init*.py", line 50, in connect return Connection(server_hostname, http_path, access_token, **kwargs) File "/azureml-envs/azureml_9806bfc7c716eaf0df12bea8512e961b/lib/python3.8/site-packages/databricks/sql/client.py", line 189, in *init* self._session_handle = self.thrift_backend.open_session( File "/azureml-envs/azureml_9806bfc7c716eaf0df12bea8512e961b/lib/python3.8/site-packages/databricks/sql/thrift_backend.py", line 464, in open_session response = self.make_request(self._client.OpenSession, open_session_req) File "/azureml-envs/azureml_9806bfc7c716eaf0df12bea8512e961b/lib/python3.8/site-packages/databricks/sql/thrift_backend.py", line 393, in make_request self._handle_request_error(error_info, attempt, elapsed) File "/azureml-envs/azureml_9806bfc7c716eaf0df12bea8512e961b/lib/python3.8/site-packages/databricks/sql/thrift_backend.py", line 261, in _handle_request_error raise network_request_error databricks.sql.exc.RequestError: Error during request to server — Reply to this email directly, view it on GitHub <#23 (comment)>, or unsubscribe <https://github.com/notifications/unsubscribe-auth/AECG7B5ZOOUNLZQFPX36H4DYQNEORAVCNFSM54TYKLLKU5DIOJSWCZC7NNSXTN2JONZXKZKDN5WW2ZLOOQ5TCOJRGE2TKMJXGQ4Q> . You are receiving this because you were mentioned.Message ID: ***@***.***>

[Ownmarc]

Hello,

We are seeing the same error, here is the relevant exception text. We use polars (0.20.6) to make requests to our Databricks endpoint using sql.connect. This works fine for some time until it stops working. Then if we simply restart our pod (lives as a docker container in kubernetes), it starts working again.

def get_databricks_connection():
    return sql.connect(
        server_hostname=os.getenv("DATABRICKS_SERVER_HOSTNAME"),
        http_path=os.getenv("DATABRICKS_HTTP_PATH"),
        access_token=os.getenv("DATABRICKS_TOKEN"),
        use_inline_params=True,
    )

databricks_connection = get_databricks_connection()

pl.read_database(
            query=QUERY,
            connection=databricks_connection,
            execute_options={
                "parameters": {
                    "some_param": some_param,
                }
            },
        )

certifi==2024.2.2
polars==0.20.6
databricks-sql-connector==3.0.3
thrift==0.16.0
pyarrow==14.0.2
flask==2.2.5
requests==2.28.2
urllib3==1.26.18

line 255, in _query_event\n return pl.read_database(\n File \"/home/python/.local/lib/python3.10/site-packages/polars/io/database.py\", line 582, in read_database\n return cx.execute(\n File \"/home/python/.local/lib/python3.10/site-packages/polars/io/database.py\", line 345, in execute\n result = cursor_execute(query, **options)\n File \"/home/python/.local/lib/python3.10/site-packages/databricks/sql/client.py\", line 761, in execute\n execute_response = self.thrift_backend.execute_command(\n File \"/home/python/.local/lib/python3.10/site-packages/databricks/sql/thrift_backend.py\", line 868, in execute_command\n resp = self.make_request(self._client.ExecuteStatement, req)\n File \"/home/python/.local/lib/python3.10/site-packages/databricks/sql/thrift_backend.py\", line 507, in make_request\n self._handle_request_error(error_info, attempt, elapsed)\n File \"/home/python/.local/lib/python3.10/site-packages/databricks/sql/thrift_backend.py\", line 337, in _handle_request_error\n raise network_request_error\ndatabricks.sql.exc.RequestError: Error during request to server"