'str' object does not support item assignment, while Databrciks database connection

[nagarajmmu]

Hi Team

I have installed superset with docker container, logged in to superset.
I wanted to connect to Azure Databricks. I have followed the below steps.

1. I have created "requirements-local.txt" file in /superset/docker/ folder.
2. Added "databricks-dbapi[hive,odbc,sqlalchemy]" in "requirements-local.txt" file.
3. Ran "docker-compose build --force-rm" command to build.
4. Ram "docker-compose -f docker-compose-non-dev.yml up" command to run the docker.
5. Superset is up and running and able to login.

How to reproduce the bug

1. Go to Data -> "Connect a database"
2. Select "Databricks SQL Endpoint" in "SUPPORTED DATABASES" dropdown .
3. Provide "databricks+pyhive://token:{access_token}@{server_hostname}:{port}/{database_name}" URL details from Databricks, for "SQLALCHEMY URI"
4. Provide "{"connect_args": {"http_path": "sql/protocolv1/o/****"}}" with Http Path, in ADVANCED > Other > ENGINE PARAMETERS.
5. Click on "TEST CONNECTION"
6. Below ERROR occurred
   ERROR:
   superset_app | SupersetErrorsException
   superset_app | Traceback (most recent call last):
   superset_app | File "/app/superset/databases/commands/test_connection.py", line 102, in run
   superset_app | with database.get_sqla_engine_with_context() as engine:
   superset_app | File "/usr/local/lib/python3.8/contextlib.py", line 113, in enter
   superset_app | return next(self.gen)
   superset_app | File "/app/superset/models/core.py", line 372, in get_sqla_engine_with_context
   superset_app | yield self._get_sqla_engine(schema=schema, nullpool=nullpool, source=source)
   superset_app | File "/app/superset/models/core.py", line 396, in _get_sqla_engine
   superset_app | params["poolclass"] = NullPool
   superset_app | TypeError: 'str' object does not support item assignment
   superset_app |
   superset_app | The above exception was the direct cause of the following exception:
   superset_app |
   superset_app | Traceback (most recent call last):
   superset_app | File "/usr/local/lib/python3.8/site-packages/flask/app.py", line 1516, in full_dispatch_request
   superset_app | rv = self.dispatch_request()
   superset_app | File "/usr/local/lib/python3.8/site-packages/flask/app.py", line 1502, in dispatch_request
   superset_app | return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args)
   superset_app | File "/usr/local/lib/python3.8/site-packages/flask_appbuilder/security/decorators.py", line 89, in wraps
   superset_app | return f(self, *args, **kwargs)
   superset_app | File "/app/superset/views/base_api.py", line 114, in wraps
   superset_app | raise ex
   superset_app | File "/app/superset/views/base_api.py", line 111, in wraps
   superset_app | duration, response = time_function(f, self, *args, **kwargs)
   superset_app | File "/app/superset/utils/core.py", line 1604, in time_function
   superset_app | response = func(*args, **kwargs)
   superset_app | File "/app/superset/utils/log.py", line 265, in wrapper
   superset_app | value = f(*args, **kwargs)
   superset_app | File "/app/superset/views/base_api.py", line 84, in wraps
   superset_app | return f(self, *args, **kwargs)
   superset_app | File "/app/superset/databases/api.py", line 726, in test_connection
   superset_app | TestConnectionDatabaseCommand(item).run()
   superset_app | File "/app/superset/databases/commands/test_connection.py", line 176, in run
   superset_app | raise DatabaseTestConnectionUnexpectedError(errors) from ex
   superset_app | superset.databases.commands.exceptions.DatabaseTestConnectionUnexpectedError: [SupersetError(message="'str' object does not support item assignment", error_type=<SupersetErrorType.GENERIC_DB_ENGINE_ERROR: 'GENERIC_DB_ENGINE_ERROR'>, level=<ErrorLevel.ERROR: 'error'>, extra={'engine_name': 'Databricks Interactive Cluster', 'issue_codes': [{'code': 1002, 'message': 'Issue 1002 - The database returned an unexpected error.'}]})]
   superset_app | 2022-11-29 23:29:08,364:WARNING:superset.views.base:SupersetErrorsException
   superset_app | Traceback (most recent call last):
   superset_app | File "/app/superset/databases/commands/test_connection.py", line 102, in run
   superset_app | with database.get_sqla_engine_with_context() as engine:
   superset_app | File "/usr/local/lib/python3.8/contextlib.py", line 113, in enter
   superset_app | return next(self.gen)
   superset_app | File "/app/superset/models/core.py", line 372, in get_sqla_engine_with_context
   superset_app | yield self._get_sqla_engine(schema=schema, nullpool=nullpool, source=source)
   superset_app | File "/app/superset/models/core.py", line 396, in _get_sqla_engine
   superset_app | params["poolclass"] = NullPool
   superset_app | TypeError: 'str' object does not support item assignment
   superset_app |
   superset_app | The above exception was the direct cause of the following exception:
   superset_app |
   superset_app | Traceback (most recent call last):
   superset_app | File "/usr/local/lib/python3.8/site-packages/flask/app.py", line 1516, in full_dispatch_request
   superset_app | rv = self.dispatch_request()
   superset_app | File "/usr/local/lib/python3.8/site-packages/flask/app.py", line 1502, in dispatch_request
   superset_app | return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args)
   superset_app | File "/usr/local/lib/python3.8/site-packages/flask_appbuilder/security/decorators.py", line 89, in wraps
   superset_app | return f(self, *args, **kwargs)
   superset_app | File "/app/superset/views/base_api.py", line 114, in wraps
   superset_app | raise ex
   superset_app | File "/app/superset/views/base_api.py", line 111, in wraps
   superset_app | duration, response = time_function(f, self, *args, **kwargs)
   superset_app | File "/app/superset/utils/core.py", line 1604, in time_function
   superset_app | response = func(*args, **kwargs)
   superset_app | File "/app/superset/utils/log.py", line 265, in wrapper
   superset_app | value = f(*args, **kwargs)
   superset_app | File "/app/superset/views/base_api.py", line 84, in wraps
   superset_app | return f(self, *args, **kwargs)
   superset_app | File "/app/superset/databases/api.py", line 726, in test_connection
   superset_app | TestConnectionDatabaseCommand(item).run()
   superset_app | File "/app/superset/databases/commands/test_connection.py", line 176, in run
   superset_app | raise DatabaseTestConnectionUnexpectedError(errors) from ex
   superset_app | superset.databases.commands.exceptions.DatabaseTestConnectionUnexpectedError: [SupersetError(message="'str' object does not support item assignment", error_type=<SupersetErrorType.GENERIC_DB_ENGINE_ERROR: 'GENERIC_DB_ENGINE_ERROR'>, level=<ErrorLevel.ERROR: 'error'>, extra={'engine_name': 'Databricks Interactive Cluster', 'issue_codes': [{'code': 1002, 'message': 'Issue 1002 - The database returned an unexpected error.'}]})]
   superset_app | 10.48.4.5 - - [29/Nov/2022:23:29:08 +0000] "POST /api/v1/database/test_connection/ HTTP/1.1" 422 296 "http://[Host_IP]:8088/superset/welcome/" "Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/107.0.0.0 Safari/537.36"
   superset_app | 10.48.4.5 - - [29/Nov/2022:23:29:39 +0000] "GET /api/v1/me/ HTTP/1.1" 200 150 "http://[Host_IP]:8088/superset/welcome/" "Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/107.0.0.0 Safari/537.36"

Please let me know, if I am missing anything while connecting to Azure Databricks.

Please help me to connect to Azure Databricks.

Thanks
Nagaraj M M

[rusackas]

Pinging @betodealmeida as someone who knows the intricacies of Databricks x Superset. Please feel free to redirect/shepherd this issue as you see fit :)

[nagarajmmu]

Hi Evan

Thank you for adding betodealmeida

[fullergalway]

Same error message trying to connect a hive database on preset.io

"test connection" reports, "Connection looks good!", but "connect" triggers the error.

{
  "errors": [
    {
      "message": "'str' object does not support item assignment",
      "error_type": "GENERIC_DB_ENGINE_ERROR",
      "level": "error",
      "extra": {
        "engine_name": "Apache Hive",
        "issue_codes": [
          {
            "code": 1002,
            "message": "Issue 1002 - The database returned an unexpected error."
          }
        ]
      }
    }
  ]

[fullergalway]

"test connection" reports, "Connection looks good!", but "connect" triggers the error.

The problem occurs in preset.io for me, but in my superset development environment there is no issue. (Reporting in preset)

[nagarajmmu]

Hi fullergalway and Elizabeth Thompson

Thanks for the update, I am waiting for the fix.

Thanks
Nagaraj M M

[vignesh-maersk2022]

Hi @eschutho I have tried your code by downloading locally and tested still the issue is not fixed. Please look into it again.

[rusackas]

I'm assuming this was fixed by now but never closed. Can anyone confirm on Preset and/or Superset 3.x?