Data Modeling: Datasets

[dpsutton]

Data Modeling: Datasets

Ability to used saved datasets as the source of a query. Initial deliverable is to use the saved-questions functionality and create a UX that promotes and eases their use.

Links

• product doc

Milestone 1

Backend

• Model and add in collection items API: Issue: Model and add in collection items API #18655 PR: Dataset schema #18629
• Add to query source API: Issue Add to query source API #18656 PR: tree view for datasets #18704
• Search: Issue datasets in search #18657 PR: Dataset search #18715
• Cleanup of changing to a dataset: revision history and verification issues. See Add ability to turn a saved question into a dataset #18702 (review)
• Ensure tree view only contains parents of datasets: Issue ensure dataset tree api only returns nodes which contain leaves that are datasets #18672 PR: Annotate collection tree with datasets and cards #18833
• mark metadata as from a dataset query PR: mark dataset queries in the qp #18900

Frontend

• Turn a saved question into a dataset and display it in collections (Add ability to turn a saved question into a dataset #18702)
• Add "Datasets" section to query builder's data picker (Update query builder data selector for datasets #18764)

Milestone 2

Frontend

• Distinct UI for dataset page (Simple mode for datasets, update dataset headers #18993)
• Ability to undo turning a question into a dataset (Undo turning a question info a dataset #18855)
• New UI for creating things inside a collection (plus icon) (Adding collection items from collection page #19009)
• Turn a dataset back into a saved question (Add ability to turn a dataset back into a question #18873)
• Distinct URLs for dataset pages (Use /dataset URLs for datasets #19032)
• Ensure dataset picker only shows collections with datasets (Hide collections without datasets in dataset picker #19153)

Backend

• Audit and correct analytics. Things like card-read, card-edit.
• dismissed modal state: Issue Record modal dismissal state per user #18658, PR: Nest qbnewb under /modal to allow for other modals #18926
• datasets in activity admin page: Dataset recent activity #18967

Milestone 3

Frontend

• Implement basic query editor (Dataset query editor #19103)
• Implement metadata editor (Metadata editor for models #19774)

Backend

• datasets preserve metadata (Datasets preserve metadata #19158)

Misc / Polish

• Fix revision history messages when a question is turned into a dataset and vice-versa (Add revision messages for datasets #18875)
• Display datasets correctly in recent searches (Display datasets in recent searches and on activity page #18987)
• Data selector: skip raw data step when there is only one database when navigating back from a selected table
• Data selector: when opening the selector with a dataset selected, "Saved Questions" opens instead of "Datasets"
• Show if a dataset is in an official collection (maybe also show if a question is based on an official dataset)
• Disallow creating datasets based on native questions using variables
• Add a warning icon to questions built on archived datasets
• List datasets together with questions when adding things to dashboards
• Cleanup data selector UI (comments from Maz)
• Verify dataset UI
• Handle databases without nested queries support
• Should be possible to pick a Dataset as a target when configuring click behavior on a dashboard card (Slack converation)

[rcronin]

I noticed that the datasets essentially turn the underlying query into a subselect when adding filters, joins, etc. to it. How is it going to be handled when the query for a dataset needs to be filtered from a variable? Specific scenario would be when the base of a dataset is a native sql question and there's a variable for a date filter?

[flamber]

@rcronin It's currently not supported to pass-thru variables: #6449

[rcronin]

@rcronin It's currently not supported to pass-thru variables: #6449

Will it be supported at some point? This would be extremely valuable feature and seems to be noted that way in the issue referenced.

[flamber]

@rcronin I cannot tell you if or when, but upvote the issue by clicking 👍 on the first post. And follow the roadmap: https://www.metabase.com/roadmap/