[CT-1120] [Bug] BigQuery profile does not work when passing individual keyfile parameters using keyfile_json

[smithcb88]

Is this a new bug in dbt-core?

• I believe this is a new bug in dbt-core
• I have searched the existing issues, and I could not find an existing issue for this bug

Current Behavior

I have the below profiles.yml file in my environment (sensitive information replaced with root vegetables):

myprofile:
  target: dev
  outputs:
    prod:
      type: bigquery
      method: service-account
      project: my-project
      location: europe-west2
      dataset: core
      threads: 1
      keyfile_json:
        type: potato
        project_id: potato
        private_key_id: potato
        private_key: potato
        client_email: potato
        client_id: potato
        auth_uri: potato
        token_uri: potato
        auth_provider_x509_cert_url: potato
        client_x509_cert_url: potato

When I execute dbt run in my project, I get the below error:

Database Error
  expected str, bytes or os.PathLike object, not NoneType

Expected Behavior

The command would run my models, as it would (and does) do if I comment out everything under keyfile_json and replace with keyfile & a path to my JSON key file.

Steps To Reproduce

Run dbt_run using the above profiles.yml

Relevant log output

No response

Environment

- OS: Mac OS Monterey (12.3.1)
- Python: 3.10.4
- dbt: 1.2.0
- dbt-bigquery = 1.2.0

Which database adapter are you using with dbt?

bigquery

Additional Context

No response

[jtcohen6]

sensitive information replaced with root vegetables

This got a chuckle out of me :)

I remember running into a different error message, a while ago, when the private key is malformed (specifically containing \n characters instead of real newlines): #3044 (comment)

But I wouldn't have expected expected str, bytes or os.PathLike object, not NoneType.

@lostmygithubaccount assigned himself to this one, so I'll let him have it. Probably we want to transfer this issue to the dbt-bigquery repo, since that's the right place to keep investigating and spot the relevant code.

[smithcb88]

Glad it brightened someones day, @jtcohen6 !

I actually discovered that this is user error....! method: service-account should be service-account-json when passing individual keyfile paramaters. Frustratingly missed this from the documentation when debugging :( So this issue can be closed.

[jtcohen6]

Glad you were able to figure it out! We've seen the same issue come up before, and there's a good_first_issue to improve the error message that the dbt-bigquery plugin raises when it detects a mismatch between the specified method and the provided attribute (keyfile vs. keyfile_json): dbt-labs/dbt-bigquery#128

[lostmygithubaccount]

@smithcb88 also glad you figured it out! it would have taken me a while longer :)