[BUG] REDIRCT_URL mismatched with Databricks Cloud OAuth Application

[jasondamour]

Describe the bug

When trying to use Oauth for dbt-databricks, the browser returns this error:

{
"error_description": "redirect_uri 'http://localhost:8020' not registered for OAuth application 'dbt-databricks'",
"error": "invalid_request"
}

The mismatch is clear to see:

# curl https://accounts.cloud.databricks.com/api/2.0/accounts//oauth2/published-apps
{
  "app_id": "8521f912-fcca-11ed-be56-0242ac120002",
  "client_id": "dbt-databricks",
  "name": "dbt adapter for Databricks",
  "description": "dbt adapter for Databricks",
  "is_confidential_client": false,
  "redirect_urls": [
    "http://localhost:8050/"
  ],
  "scopes": [
    "sql",
    "offline_access"
  ]
}

vs

https://github.com/databricks/dbt-databricks/blob/main/dbt/adapters/databricks/connections.py?plain=1#L122

REDIRECT_URL = "http://localhost:8020"

Steps To Reproduce

1. Enable DBT OAuth app on account: databricks account published-app-integration create --app-id databricks-dbt-adapter
2. Create profiles.yml which uses auth_type: oauth
3. Run dbt debug to trigger oauth flow

Expected behavior

OAuth flow to succeed

Screenshots and log output

If applicable, add screenshots or log output to help explain your problem.

System information

The output of dbt --version:

➜  dbt --version 
Core:
  - installed: 1.7.7
  - latest:    1.7.7 - Up to date!

Plugins:
  - databricks: 1.7.7 - Up to date!
  - spark:      1.7.1 - Up to date!

The operating system you're using:
MacOS

The output of python --version:
3.11.7

Additional context

Probably easier/safer to update this connector than the databricks published app...

[benc-db]

@andrefurlan-db suggestions?

[jasondamour]

It's an easy PR to fix this, I'm happy to do the work. I just need help understanding the implications and the release versioning

[benc-db]

Absolutely, I'm just worried that we hard-coded that particular value to work with Azure, or similar, so I'm hoping to get someone who worked on the implementation to weigh in.

[benc-db]

@jasondamour: @andrefurlan-db is fixing this at the source so that we shouldn't need to make a change in the adapter. Will update this thread when fixed.

[jasondamour]

Awesome, ty. The scopes are also mismatched ("sql" vs "all-privileges")

[jasondamour]

@benc-db @andrefurlan-db Hi Guys, any update?

[benc-db]

I know we've been working on it (including the scope issue). Let me check with my team to see if we have an update on when it should be rolled out.

[rodrigorabioglio]

Hi guys! Do you have an idea when this fix is going to be rolled out ? Tks :)

[jasondamour]

@benc-db @andrefurlan-db any updates please.

[benc-db]

Apologies for how long this has dragged on. This is pretty far out of my area of expertise; my understanding is that it will break oauth for other users if we change the url in the adapter directly. I appreciate this PR that was filed: #632
however, my plan is to put out a PR that makes this user-configurable.
Will have that out today, release next week.

[benc-db]

See #640, I think there are still issues I'm trying to get resolved.