Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Auto populate View Lineage to Tables #2809

Closed
kevinbtai opened this issue Feb 16, 2022 · 2 comments · Fixed by #2977
Closed

Auto populate View Lineage to Tables #2809

kevinbtai opened this issue Feb 16, 2022 · 2 comments · Fixed by #2977
Assignees
Labels
Ingestion P1 High priority

Comments

@kevinbtai
Copy link

Is your feature request related to a problem? Please describe.
For data warehouses like Snowflake, we have a bunch of views created by other teams using our source data. It would be helpful if lineage can be automatically populated for views to their source tables so it is easier to track which users/queries rely on certain data.

Describe the solution you'd like
Lineage for views is automatically created and attributed to the source tables.

@harshach harshach added Ingestion P1 High priority labels Feb 16, 2022
@harshach harshach added this to To do in Release 0.9.0 via automation Feb 16, 2022
@harshach harshach added this to the Milestone 4 - 0.9.0 Release milestone Feb 16, 2022
@igor-kramer
Copy link
Contributor

Wow! I'll wait your MR and after I can add the same functionality for Postgres/Vertica.

@harshach
Copy link
Collaborator

@ayush-shah lets run the viewDefinition to lineage runner and we should be able to connect to the upstream tables here.
Only edgecase here is if the view is in database y and its depends on the database x tables. But if we are processing database y tables/views first we won't be able to link it. But this should resolve in the next run as all the tables are available in the next ingestion cycle

@ayush-shah ayush-shah assigned ulixius9 and unassigned ayush-shah Feb 24, 2022
@ulixius9 ulixius9 moved this from To do to In progress in Release 0.9.0 Feb 24, 2022
ulixius9 added a commit to ulixius9/OpenMetadata that referenced this issue Feb 24, 2022
Release 0.9.0 automation moved this from In progress to Done Feb 24, 2022
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
Ingestion P1 High priority
Projects
No open projects
Development

Successfully merging a pull request may close this issue.

5 participants