Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

DBT ephemeral models fail due to absence in the asset graph #19619

Closed
Pr0p1k opened this issue Feb 6, 2024 · 1 comment
Closed

DBT ephemeral models fail due to absence in the asset graph #19619

Pr0p1k opened this issue Feb 6, 2024 · 1 comment
Labels
integration: dbt Related to dagster-dbt type: bug Something isn't working

Comments

@Pr0p1k
Copy link

Pr0p1k commented Feb 6, 2024

Dagster version

1.5.14

What's the issue?

We have few ephemeral DBT models.
They are represented in Dagster asset graph as sources, which is fine.

The problem arises after execution:
dagster.core.errors.DagsterInvariantViolationError: Core compute for op "run_dbt_da74d" returned an output "eph***_dbt" that does not exist. The available outputs are [...]

The status in Dagster is "failed", but the models work successfully in our database.

So it seems, the ephemeral model is reported by dbt as successful, but Dagster can't find it, since it's not included in the graph.

There might be a potential issue that we have two dirs with models: "core" and "core_rel", and the missing comes from "core". But anyway table materialized models are shown from both dirs, and both dirs ephemeral models are missing in the available outputs.

What did you expect to happen?

Suppress the error as there's no way to display ephemeral models in Dagster

How to reproduce?

Create any upstream ephemeral model and run "Materialize all"

Deployment type

None

Deployment details

No response

Additional information

No response

Message from the maintainers

Impacted by this issue? Give it a 👍! We factor engagement into prioritization.

@Pr0p1k Pr0p1k added the type: bug Something isn't working label Feb 6, 2024
@garethbrickman garethbrickman added the integration: dbt Related to dagster-dbt label Feb 6, 2024
@rexledesma
Copy link
Member

I can't reproduce this using @dbt_assets. If you have a minimal reproduction exhibits your exact problem, please share it and re-open the issue.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
integration: dbt Related to dagster-dbt type: bug Something isn't working
Projects
None yet
Development

No branches or pull requests

3 participants