Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

docs: dataset usage and query history feature guide #5900

Merged
merged 4 commits into from
Oct 14, 2022

Conversation

treff7es
Copy link
Contributor

Checklist

  • The PR conforms to DataHub's Contributing Guideline (particularly Commit Message Format)
  • Links to related issues (if applicable)
  • Tests for the changes have been added/updated (if applicable)
  • Docs related to the changes have been added/updated (if applicable). If a new feature has been added a Usage Guide has been added for the same.
  • For any breaking change/potential downtime/deprecation/big changes an entry has been made in Updating DataHub

@github-actions
Copy link

github-actions bot commented Sep 10, 2022

Unit Test Results (build & test)

571 tests  +50   571 ✔️ +52   14m 7s ⏱️ -3s
141 suites +19       0 💤 ±  0 
141 files   +19       0  -   2 

Results for commit aade869. ± Comparison against base commit 4174669.

♻️ This comment has been updated with latest results.

<img width="70%" src="https://raw.githubusercontent.com/datahub-project/static-assets/main/imgs/source-snowflake-capabilities.png"/>
</p>

There are some sources where you have to use a different usage specific source for usage ingestion. In this
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This sentence is a bit confusing. Consider rephrasing to:

Some sources require a separate, usage-specific recipe to ingest Usage and Query History metadata. In this case, it is noted in the capabilities summary, like so:


## Using Dataset Usage & Query History

After successful ingestion, the Query tab will be enabled on datasets with any usage.
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

edit: the Queries tab

Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

After successful ingestion, will both the Queries and Stats tab be populated?

<img width="70%" src="https://raw.githubusercontent.com/datahub-project/static-assets/main/imgs/feature-queries-tab.png"/>
</p>

On the query tab, you can see the top 5 queries which referenced this dataset.
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

edit: On the Queries tab

Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Suggestion: can we add context about how we choose the top 5 queries? is it based on number of times executed? over what period of time?

<img width="70%" src="https://raw.githubusercontent.com/datahub-project/static-assets/main/imgs/feature-usage-stats-tab.png"/>
</p>

With the collected usage data, you can even see column-level usage statistics (on sources that support this):
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

(on sources that support this):

How can people tell which sources support column-level usage?

<img width="70%" src="https://raw.githubusercontent.com/datahub-project/static-assets/main/imgs/feature-usage-stats-tab.png"/>
</p>

With the collected usage data, you can even see column-level usage statistics (on sources that support this):
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Is this image correct? I don't see column level usage

@anshbansal anshbansal added the docs Issues and Improvements to docs label Sep 13, 2022
@laulpogan
Copy link
Contributor

@shirshanka @treff7es has this on his list for tomorrow morning- will revisit Thurs

Copy link
Collaborator

@maggiehays maggiehays left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Looks great!

@treff7es treff7es merged commit 5a2bdea into datahub-project:master Oct 14, 2022
anthonyhungnguyen pushed a commit to anthonyhungnguyen/zlp-datahub that referenced this pull request Nov 8, 2022
cccs-tom pushed a commit to CybercentreCanada/datahub that referenced this pull request Nov 18, 2022
cccs-tom pushed a commit to CybercentreCanada/datahub that referenced this pull request Nov 18, 2022
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
docs Issues and Improvements to docs docs-refresh
Projects
None yet
Development

Successfully merging this pull request may close these issues.

6 participants