[Meta][2.14] Support Multiple Data Source in OpenSearch Dashboards Plugins

[BionIT]

Context

Since 2.4.0, multiple data source feature is enabled as a core plugin(see RFC #1388 and PR #2334).

With multiple data source enabled, users of OpenSearch Dashboards are able to create data source connections and query local and remote OpenSearch clusters(see documentation https://opensearch.org/docs/latest/dashboards/management/data-sources/). Enabling multiple data source at dashboards plugin would ensure users that have been using the multiple data source feature to have a consistent experience in the dashboard, and allows users to use the same dashboard for different clusters.

This meta issue is used to track the PR/Issues for supporting multiple data source in OpenSearch Dashboards Plugins

• [FEATURE] Support Multiple Data Source in Query Workbench Dashboards Plugin dashboards-query-workbench#266
• [FEATURE] Support Multiple Data Source in Search Relevance Dashboards Plugin dashboards-search-relevance#371
• [FEATURE] Support Multiple Data Source in Security Dashboards Plugin security-dashboards-plugin#1782
• [FEATURE] Support Multiple Data Source in Notifications Dashboards Plugin dashboards-notifications#159
• [FEATURE] Support Multiple Data Source in Maps Dashboards Plugin dashboards-maps#584
• [FEATURE] Support Multiple Data Source in Index Management Dashboards Plugin index-management-dashboards-plugin#982
• [FEATURE] Support Multiple Data Source in Anomaly Detection Dashboards Plugin anomaly-detection-dashboards-plugin#686
• [FEATURE] Support Multiple Data Source in ML Commons Dashboards Plugin ml-commons-dashboards#304
• [FEATURE] Support Multiple Data Source in Trace Analytics Dashboards Plugin dashboards-observability#1440

Data source connection picker

In #5717, we introduced a picker to select data connection. This picker is within the data source plugin bundle, and when loaded, it will fetch data source connection which is a type of saved object and then populate the available data connections as single selected options in the drop down.

In order to consume this picker from a plugin, follow the steps as below:

1. in the opensearch_dashboards.json file, add dataSource as required bundle, eg. "requiredBundles": ["dataSourceManagement"]
2. import the component in the file where this picker is needed, eg. import { ClusterSelector } from '../../data_source_management/public'
3. this picker exposes onSelectedDataSource which can be used to get the selected option from the picker component

Data source client

Data source plugin was released in OpenSearch Dashboards 2.4.0 as part of the multi data source feature.
The plugin exposes OpenSearch client wrapper and this is the design https://github.com/opensearch-project/OpenSearch-Dashboards/blob/main/docs/multi-datasource/client_management_design.md, it can be acquired from context via the following:
client: OpenSearchClient = await context.dataSource.opensearch.getClient(dataSourceId);
For legacy client, we can initiate the client via the following
apiCaller: LegacyAPICaller = context.dataSource.opensearch.legacy.getClient(dataSourceId).callAPI;
Note that we need to pass in the data source id as argument, and this id can be acquired by passing a function to the child component by following instructions above.

PoC

PoC commit for query bench dashboards plugin to support MDS: opensearch-project/dashboards-query-workbench@main...BionIT:dashboards-query-workbench:mdinteg

A few things to notice:

1. to consume the client, we need to add "optionalPlugins": ["dataSource"] in opensearch_dashboards.json in the plugin repo
2. to consume the React component to pick data source, we need to add "requiredBundles": ["dataSourceManagement"]
3. during plugin set up, in order to register the plugin API schema/spec, we can do the following dataSource.registerCustomApiSchema(sqlPlugin)
4. most plugins might need to depend on the data source picker, please follow to set it up, and once we acquire the selected data source id, we need to pass it to plugin const {dataSourceId} = request.query; then the datasource Id can be passed to the data source client wrapper context.dataSource.opensearch.getClient(dataSourceId)

[hdhalter]

Is documentation needed for this?

[BionIT]

Created issue for documentation opensearch-project/documentation-website#7071 @hdhalter working on it now