Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

feat: dbt 1.8 #10180

Merged
merged 1 commit into from
May 24, 2024
Merged

feat: dbt 1.8 #10180

merged 1 commit into from
May 24, 2024

Conversation

ZeRego
Copy link
Contributor

@ZeRego ZeRego commented May 23, 2024

Closes: #10034

This PR only offers partial support!

  • There is a regression in the postgres and redshift adaptor.
  • Databricks adaptor hasn't released a version compatible with 1.8.

We are going to monitor these adaptors and support them when possible.

Reviewer actions

  • I have manually tested the changes in the preview environment
  • I have reviewed the code
  • I understand that "request changes" will block this PR from merging

@ZeRego ZeRego requested a review from a team May 23, 2024 15:47
@ZeRego ZeRego self-assigned this May 23, 2024
@ZeRego ZeRego requested review from joaoviana and removed request for a team May 23, 2024 15:47
@owlas owlas requested a deployment to feat/dbt-1.8 - jaffle_db_pg_13 PR #10180 May 23, 2024 15:47 — with Render Abandoned
@owlas owlas temporarily deployed to feat/dbt-1.8 - headless-browser PR #10180 May 23, 2024 15:47 — with Render Destroyed
Copy link

netlify bot commented May 23, 2024

Deploy Preview for peaceful-bassi-cbf284 canceled.

Name Link
🔨 Latest commit 7e66ccc
🔍 Latest deploy log https://app.netlify.com/sites/peaceful-bassi-cbf284/deploys/664f6516fa1e7e0007240bc0

@owlas owlas temporarily deployed to feat/dbt-1.8 - lightdash PR #10180 May 23, 2024 15:48 — with Render Destroyed
Comment on lines +93 to +98
try {
return JSON.parse(line).unique_id;
} catch {
// ignore non-json lines
return '';
}
Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

dbt is throwing deprecation warnings, even with --quiet, which were causing an error.

"definitions": {
"LightdashNodeConfig": {
"allOf": [
{
Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Notice that here we usually extend the dbt definition.

They have changed the manifest json schema https://schemas.getdbt.com/dbt/manifest/v12.json to use draft 2020-12 https://json-schema.org/draft/2020-12/schema format.

I couldn't find a way to extend these definitions.
Let me know if there is a way. 🙏

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Yea, I don't think it's that straightforward unless specific referencing is done 👍

Comment on lines +27 to +29
const ajv = new Ajv2020();
ajv.addMetaSchema(draft7MetaSchema); // add backward compatibility with draft-07
ajv.addSchema([
Copy link
Contributor Author

@ZeRego ZeRego May 23, 2024

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

draft 2020 is not compatible with draft 07 https://ajv.js.org/json-schema.html#draft-2020-12
So we need to initialize AVJ2020 and add draft 07 meta schema.

Copy link
Contributor

@joaoviana joaoviana left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

🎉

@ZeRego ZeRego merged commit 379e902 into main May 24, 2024
45 of 51 checks passed
@ZeRego ZeRego deleted the feat/dbt-1.8 branch May 24, 2024 09:59
lightdash-bot pushed a commit that referenced this pull request May 24, 2024
# [0.1103.0](0.1102.3...0.1103.0) (2024-05-24)

### Features

* dbt 1.8 ([#10180](#10180)) ([379e902](379e902))
@lightdash-bot
Copy link
Collaborator

🎉 This PR is included in version 0.1103.0 🎉

The release is available on:

Your semantic-release bot 📦🚀

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
Projects
None yet
Development

Successfully merging this pull request may close these issues.

test support for dbt 1.8
4 participants