Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

[Feature Request] support Databricks #244

Closed
frankivo opened this issue Feb 23, 2023 · 5 comments
Closed

[Feature Request] support Databricks #244

frankivo opened this issue Feb 23, 2023 · 5 comments

Comments

@frankivo
Copy link

Is your feature request related to a problem? Please describe.
Most test, but not all, work with Databricks. For example: expect_column_values_to_match_regex

Describe the solution you'd like
A valid implementation for those missing tests.

Describe alternatives you've considered
I'm writing custom tests for now.

Additional context
Willing to do some coding myself :)

@clausherther
Copy link
Contributor

Hi @frankivo - we don't have a way to test against databricks - or any platform other than Postgres, BigQuery or Snowflake - in our CI pipeline, so we don't currently support platforms outside of that core set.
However, as I've outlined in the answer to this question you can add support for other platforms via shims in their respective **-utils package, such as spark-utils. Hope that helps!

@frankivo
Copy link
Author

I'll try :)
databricks/dbt-databricks#280

@clausherther
Copy link
Contributor

@frankivo I wonder if they'll want you to implement this in spark-utils though, not in the adapater repo?

@frankivo
Copy link
Author

Afaik I'm using a databricks feature and not a spark feature. We'll see.

@clausherther
Copy link
Contributor

I don't think there is a databricks-utils package though, so spark-utils seems to be where the databricks/sparks macros go. My guess would be the macros in the adapter mirror what dbt-core supports, and since they don't have a regexp implementation, the databricks adapter shouldn't either.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

No branches or pull requests

2 participants