Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Cannot configure default credentials in Gitlab pipelines #1227

Open
ozw1z5rd opened this issue Feb 21, 2024 · 1 comment
Open

Cannot configure default credentials in Gitlab pipelines #1227

ozw1z5rd opened this issue Feb 21, 2024 · 1 comment
Labels
CLI CLI related issues Response Requested

Comments

@ozw1z5rd
Copy link

Describe the issue

I don't know if this is a "classic" bug, or a improvement request. The fact is that databricks-cli fails to recover the token from the configuration file when used inside a Gitlab pipeline. Anything works until you try to use the token.
BTW, the solution is don't initialize the databricks-cli, just pass TOKEN and WORKSPACE URL as env VARS.

Steps to reproduce the behavior

Of course, you need a Gitlab pipeline.

  1. obtain a person token from databricks
  2. In the Gitlab's repo settings -> CI/CD -> variables -> add a new VAR "TF_VAR_personal_token" to hold your PERSONAL TOKEN and set it Masked & Expanded. ( actuall, any var name is fine, but DATABRICKS_TOKEN )
  3. Use a YAML file like that where the databricks cli is initialized by writing the configuration file:
variables: 
  DATABRICKS_HOST: "<your workspace url>"

stages: 
  - deploy

deploy-job:    
  stage: deploy 
  stage: build
  environment: production
  script:
    # install databricks cli 
    - curl -fsSL https://raw.githubusercontent.com/databricks/setup-cli/main/install.sh | sh
    - echo $TF_VAR_personal_token | databricks configure 
    - databricks current-user  me

Expected Behavior

The Gitlab pipeline ends reporting the current user info

Actual Behavior

Anything is perfect until databricks current-user me is executed. You will get this message:

https://docs.databricks.com/en/dev-tools/auth.html#databricks-client-unified-authentication
to configure credentials for your preferred authentication method. Config: host=
<your workspace url>. Env: DATABRICKS_HOST

OS and CLI version

Linux, Databricks CLI V0.213.0

Debug Logs

$ if [ -z $TF_VAR_personal_token ]; then echo "Zero size TF_VAR_personal_token"; else echo $TF_VAR_personal_token | databricks configure $DEBUG; fi
12:06:19  INFO start pid=292 version=0.214.0 args="databricks, configure, --debug"
12:06:19  INFO Saving /root/.databrickscfg pid=292
12:06:19  INFO completed execution pid=292 exit_code=0
$ echo "DATABRICKS_HOST=$DATABRICKS_HOST, DATABRICKS_PATH=$DATABRICKS_PATH, token md5 will follow"
DATABRICKS_HOST=https://xxxxxx.cloud.databricks.com/, DATABRICKS_PATH=/Volumes/main/default/datalake, token md5 will follow
$ echo $TF_VAR_personal_token | md5sum
2b7425a1a31e169cce7e149de25635f8  - THIS IS CORRECT, THE TOKEN IS CORRECTLY PASSED ...
$ id
uid=0(root) gid=0(root) groups=0(root)
$ cat /root/.databrickscfg | sed s/$TF_VAR_personal_token/xxxxxxxxxxxxx/g
[DEFAULT]
host  = https://xxxxxx.cloud.databricks.com/
token = xxxxxxxxxxxxx ---- ... AND STORED
$ databricks auth env | sed s/$TF_VAR_personal_token/xxxxxxxxxxxxx/g
{
  "env": {
    "DATABRICKS_AUTH_TYPE": "pat",
    "DATABRICKS_CONFIG_PROFILE": "DEFAULT",
    "DATABRICKS_HOST": "https://xxxxxx.cloud.databricks.com/",
    "DATABRICKS_TOKEN": "xxxxxxxxxxxxx"
  }
} 
$ databricks current-user  me $DEBUG
12:06:20  INFO start pid=310 version=0.214.0 args="databricks, current-user, me, --debug"
Error: default auth: cannot configure default credentials, please check https://docs.databricks.com/en/dev-tools/auth.html#databricks-client-unified-authentication to configure credentials for your preferred authentication method. Config: host=https://xxxxxx.cloud.databricks.com/. Env: DATABRICKS_HOST
12:06:20 ERROR failed execution pid=310 exit_code=1 error="default auth: cannot configure default credentials, please check https://docs.databricks.com/en/dev-tools/auth.html#databricks-client-unified-authentication to configure credentials for your preferred authentication method. Config: host=https://xxxxxx.cloud.databricks.com/. Env: DATABRICKS_HOST"
@ozw1z5rd ozw1z5rd added the CLI CLI related issues label Feb 21, 2024
@andrewnester
Copy link
Contributor

Could you try to upgrade to 0.217.1 version and run databricks auth describe command and share the output?

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
CLI CLI related issues Response Requested
Projects
None yet
Development

No branches or pull requests

2 participants