Open
Description
Is this your first time submitting a feature request?
- I have read the expectations for open source contributors
- I have searched the existing issues, and I could not find an existing issue for this feature
- I am requesting a straightforward extension of existing dbt functionality, rather than a Big Idea better suited to a discussion
Describe the feature
OpenLineage is an open-source project and standard designed to capture, collect, and visualize data lineage (the life-cycle and movement of data) across modern data ecosystems. It establishes a unified metadata model and an API for tools in the data pipeline ecosystem to report lineage and metadata in a standardized way.
In addition, OpenLineage captures data on pipeline runs which unlocks observability.
dbt should be fully integrated with openlineage like it's done now for Airflow.
Describe alternatives you've considered
N/A
Who will this benefit?
This will benefit users with the observability of their dbt pipelines. Openlineage (OL) is a standard. There are tools out there like Marquez that ingest and leverage those OL events.
It allows:
- lineage tracking of datasets
- observability of the pipeline executed
- observability of SQL queries executed in the data platform
Are you interested in contributing this feature?
Yes
Anything else?
No response