Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

When ingesting from multiple source with siblings, all siblings should be visible under 1 entity. #10551

Open
vijay-jangir opened this issue May 21, 2024 · 2 comments
Labels

Comments

@vijay-jangir
Copy link

When we ingest data from multiple sources (example dbt + hive, trino + hive). where sibling dataset is same (hive in our case). All the components (hive, dbt, trino) should be visible under 1 entity.

For dbt, it's composed of dbt and hive
image
similarly, for trino; it's composed of trino and hive.

However, it should come as composed of trino, dbt, and hive

Copy link

This issue is stale because it has been open for 30 days with no activity. If you believe this is still an issue on the latest DataHub release please leave a comment with the version that you tested it with. If this is a question/discussion please head to https://slack.datahubproject.io. For feature requests please use https://feature-requests.datahubproject.io

@github-actions github-actions bot added the stale label Jun 21, 2024
@vijay-jangir
Copy link
Author

This is still an issue.
this is causing issues when we have multiple systems using same underlying source.
We use DBT for our analytics etl jobs, and use trino for query engine.
but we cannot ingest both trino and dbt wihout messing up the lineage and siblings

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
Projects
None yet
Development

No branches or pull requests

1 participant