[Epic] Replace Data Selector component

[kulyk]

Our DataSelector component became pretty fragile and hard to work with time as new functionality was added. Here are a few things it's doing:

• letting users pick databases, schemas, tables, and fields (for both real and virtual data sources; virtual = saved questions represented as tables)
• doing all the necessary data loading + various states handling (loading, error, empty, lacking permissions, etc.)
• search (FE-only or via /api/search endpoint)
• virtualizing long lists of data
• scoping the picker to a particular database/schema/table

DataSelector became pretty hard to modify and extend. On top of that, it's pretty far from our code quality bar on the FE which is troubling as the component is pretty important and often used.

Proposal

During the Metabase 45 cycle, we've implemented a brand new picker for data app scaffolding. It doesn't support all the DataSelector use cases yet, but it's pretty close. It might be worth adding the missing features and replacing the legacy component with a new one.

The new component is called DataPicker and is stored under metabase/containers on the data-apps-main branch (source code).

Implementation Plan

Done before (for data apps)

• Add basic raw data picker for app scaffolding #25997
• Make data apps data picker controlled #26018
• Add data type picker to scaffolding data picker #26019
• Add safety checks to scaffolding data picker #26023
• Add items filters API to data app scaffolding data picker #26024
• Improve collection selection for scaffolding data picker #26033
• Add basic search to scaffolding data picker #26039
• Fix can't select nested collection in scaffolding data picker #26122

Blockers

No blockers on the product/design side

• Automatically select a single database in the data picker #27265
• Add empty states to data picker #27259
• Add loading states to data picker #27258
• Virtualize lists (databases, schemas, tables, fields)

Nice to haves to achieve full feature parity

• Allow picking databases / schemas (not only tables / models / questions) (needs design)
• Allow picking table fields (needs design)
• Allow configuring steps (e.g. don't let a user change a selected database, schema, table, etc.)
• Add FE search (should show up when there're many databases / schemas / tables/ fields) (needs design)
• Add popover variant (needs design)

Replacing Data Selector

Admin

• Replace DatabaseDataSelector in MetadataHeader
• Replace DatabaseSchemaAndTableDataSelector in FilteredToUrlTable
• Replace DatabaseSchemaAndTableDataSelector in GuiQueryEditor

Query Builder

• Replace SchemaTableAndFieldDataSelector in MappedFieldPicker (model metadata editor)
• Replace DataSelector in DataSourceSelectors (native query editor)
• Replace CollectionDatasetOrDataSourceSelector in DataStep (starting data picker in notebook editor) #40437
• Replace DataSourceSelector in JoinStep (notebook editor)
• Replace SchemaTableAndFieldDataSelector in TagEditorParam (native query editor)
• Replace DataSourceSelector in QuestionDataSelector

Wrapping up

• Remove DataSelector

Related Issues

• Data Selector opens with database field picker if no saved questions exist, but models exist #25142
• In the data picker, make the interaction for schema browsing more like the new interaction for browsing collections #16862
• When joining, the Raw Data selection shouldn't make me pick the database I'm using again #20406
• Update the data picker starting point when working with Models #27101
• Display Table Metadata when asking user to 'Pick your starting data' #14596
• Data selector not working in FullApp embedding if going directly from location that hasn't loaded /api/database yet #21511
• Saved Questions data selector lists all questions for collections with the same name #24660

[flamber]

Related #4874

[ranquild]

Closing in favor of the new "Entity Picker" project