Skip to content

Commit

Permalink
SQL Datasources: Reinstate SQL data source behavior around database s…
Browse files Browse the repository at this point in the history
…election when default configured databases already exist (#65659)

* badlm0mma/sql_datasource_update/ initial notes

* baldm0mma/sql_datasource_update/ solution

* baldm0mma/sql_datasource_update/ clean datasetSelector

* baldm0mma/sql_datasource_update/ clean up queryEditor.tsx

* baldm0mma/sql_datasource_update/ clewan up queryHeader.tsx

* baldm0mma/sql_datasource_update/ clean up tableSelector.tsx

* baldm0mma/sql_datasource_update/ clean up mysqlDatasource.ts

* baldm0mma/sql_datasource_update/ clean up configurationEditor.tsx

* baldm0mma/sql_datasource_update/ rem conlog from queryEditor.tsx

* baldm0mma/sql_datasource_update/ rem conlog from queryEditor.tsx

* baldm0mma/sql_datasource_update/ remove conlog in tableSelector.tsx

* baldm0mma/sql_datasource_update/ rem conlog in sqlDatasource.ts

* baldm0mma/sql_datasource_update/ update deafult database value in sqlDatasource.ts

* baldm0mma/sql_datasource_update/ update logic to accomidate no preconfig

* baldm0mma/sql_datasource_update/ update props

* baldm0mma/sql_datasource_update/ update prop names

* baldm0mma/sql_datasource_update/ update prop names in tableSelector

* baldm0mma/sql_datasource_update/ update annos in datasetSelector

* baldm0mma/sql_datasource_update/ update naming

* baldm0mma/sql_datasource_update/ update to standard langauae

* baldm0mma/sql_datasource_update/ update prop names

* baldm0mma/sql_datasource_update/ update annos in datasetSelector

* baldm0mma/sql_datasource_update/ remove unused import in tableSelector.tsx

* baldm0mma/sql_datasource_update/ remove addDefaultdataset

* baldm0mma/sql_datasource_update/ reset query when needed

* baldm0mma/sql_datasource_update/ update asymc return val

* baldm0mma/sql_datasource_update/ remove psql query editor

* baldm0mma/sql_datasource_update/ remove con logs in defaults.ts

* baldm0mma/sql_datasource_update/ revert postgres changes

* baldm0mma/sql_datasource_update/ update postgres naming

* baldm0mma/sql_datasource_update/ add altert

* baldm0mma/sql_datasource_update/ update annos and add alerts and alert logic

* baldm0mma/sql_datasource_update/ update postgres nomenclature

* baldm0mma/sql_datasource_update/ update annos and remove con logs

* baldm0mma/sql_datasource_update/ update nomenclature

* baldm0mma/sql_datasource_update/ drone fix

* baldm0mma/sql_datasource_update/ export and format

* baldm0mma/sql_datasource_update/ update docs for mysql datasource with dataset dropdown disable info

* baldm0mma/sql_datasource_update/ update mssql docs

* baldm0mma/sql_datasource_update/ update postgres docs and alert logic

* baldm0mma/sql_datasource_update/ add tests

* baldm0mma/sql_datasource_update/ update docs

* baldm0mma/sql_datasource_update/ update test names

* baldm0mma/sql_datasource_update/ update alert language

* baldm0mma/sql_datasource_update/ correct spelling

* baldm0mma/sql_datasource_update/ update postgres query builder image

* baldm0mma/sql_datasource_update/ update annos

* baldm0mma/sql_datasource_update/ update tests

* baldm0mma/sql_datasource_update/ docs updated

* baldm0mma/sql_datsource_update/ refactor determinePlaceholder

* Update public/app/features/plugins/sql/components/QueryEditor.tsx

spelling

Co-authored-by: Zoltán Bedi <zoltan.bedi@gmail.com>

* Update public/app/features/plugins/sql/components/QueryEditor.tsx

spelling

Co-authored-by: Zoltán Bedi <zoltan.bedi@gmail.com>

* baldm0mma/sql_datasource_update/ remove superfluous cleanup data from tests

* baldm0mma/sql_datasource_update/ update spelling

* Update public/app/features/plugins/sql/components/SqlComponents.test.tsx

spelling

Co-authored-by: Zoltán Bedi <zoltan.bedi@gmail.com>

* baldm0mma/sql_datasource_update/ add logic to prevent db call for tables if dataset it nullish

* baldm0mma/sql_datasource_update/ update alert tests

* baldm0mma/sql_datasource_update/ update button text

* baldm0mma/sql_datasource_update/ update tests to preferred standard

* baldm0mma/sql_datasource_update/add feature flag

* baldm0mma/sql_datasource_update/ update mssql docs

* baldm0mma/sql_datasource_update/ add feature flag to registry

* baldm0mma/sql_datasource_update/ adjust table and dataset dropdown logic

* baldm0mma/sql_datasource_update/ update testing to deal with feature flag

* baldm0mma/sql_datasource_update/ update wioth cascadeDisable

* baldm0mma/sql_datasource_update/ update naming

* baldm0mma/sql_datasource_update/ update tests to reflect enabled feature flag

* baldm0mma/sql_datasource_update/ update annotations

* baldm0mma/sql_datasource_update/ update annos in queryEd

* baldm0mma/sql_datasource_update/ update test names

* baldm0mma/sql_datasource_update/ update anno issues

* baldm0mma/slq_datasource_update/ add query to sqlDatasource with error

* baldm0mma/sql_datasource_update/ update docs language

* baldm0mma/sql_datasource_update/ remove notes to self

* baldm0mma/sql_datasource_update/ add QueryEditorFeatureFlag.utils.ts

* baldm0mma/sql_datasource_update/ update database into json

* baldm0mma/sql_datasource_update/ found file

* Update docs/sources/datasources/mssql/query-editor/index.md

Co-authored-by: lwandz13 <126723338+lwandz13@users.noreply.github.com>

* baldm0mma/sql_datasource_update/ update feature flag and toggles

* baldm0mma/sql_datasource_update/ add hasConfigIssue, update annos, rethink disable logic

* baldm0mma/sql_datasource_update/ update warning language

* baldm0mma/sql_datasource_update/ update button content

* baldm0mma/sql_datasource_update/ update jsonData logic in frontendsettings.go

* baldm0mma/sql_datasource_update/ update jsonData logic

* baldm0mma/sql_datasource_update/ update annos in frontendsettings.go

* baldm0mma/sql_datasource_update/ update sql editor docs

* baldm0mma/sql_datasource_update/ update mysql docs

* baldm0mma/sql_datasource_update/ update postgres docs

* baldm0mma/sql_datasource_update/ remove unused code in datasetSelector.tsx

* baldm0mma/sql_datasource_update/ update syntax conventions

* baldm0mma/sql_datasource_update/ add logs

* baldm0mma/sql_datasource_update/ remove unused code

* baldm0mma/sql_datasource_update/ remove conlogs

* baldm0mma/sql_datasource_update/ update tests

* baldm0mma/sql_datasource_update/ add second error to query

* baldm0mma/sql_datasource_update/ run make gen-cue

* baldm0mma/sql_database_update/ lint registry

* baldm0mma/sql_datasource_update/ update registry

* baldm0mma/sql_datasource_update/ upate datasource logic

* baldm0mma/sql_datasource_update/ add logs

* baldm0mma/sql_datasource_update/ add comms to self

* baldm0mma/sql_datasource_update/ comment out false pos tests, and add investigatory comments

* baldm0mma/sql_database_update/ update query error to only test for config change if query was made in "builder" mode

* baldm0mma/sql_datasource_update/ update annos in frontendsettings.go

* baldm0mma/sql_datasource_update/ update error logic to datasource

* baldm0mma/sql_datasource_update/ remove alerts from query editor

* baldm0mma/sql_datasource_update/ remove unused imports

* baldm0mma/sql_datasource_update/ update tests

* baldm0mma/sql_datasource_update/ remove comments

* baldm0mma/sql_datasource_update/ remove logs in queryGroup.tsx

* baldm0mma/sql_datasource_update/ remove outdated annotation in datasetSelector.tsx

* baldm0mma/sql_datasource_update/ remove superfluous test and update test description

* baldm0mma/sql_datasource_update/ remove feature flag

* baldm0mma/sql_datasource_update/ add back feature flag

* baldm0mma/sql_datasource_update/ update to enums

* baldm0mma/sql_datasource_update/ update panel caps

* baldm0mma/sql_datasource_update/ update dataset selector to default update the database correctly

* baldm0mma/sql_datasource_update/ move onChange into conditional

* baldm0mma/sql_datasource_update/ add logic for previous datasets choice

* baldm0mma/sql_datasource_update/ add back previous logic for assigning default datasets

* baldm0mma/sql_datasource_update/ update useEffect dep array

* baldm0mma/sql_datasource_update/ remove feature toggle

* baldm0mma/sql_datasource_update/ add feature toggle

---------

Co-authored-by: Zoltán Bedi <zoltan.bedi@gmail.com>
Co-authored-by: lwandz13 <126723338+lwandz13@users.noreply.github.com>
  • Loading branch information
3 people committed Jun 6, 2023
1 parent 52c4761 commit c0a1fc2
Show file tree
Hide file tree
Showing 23 changed files with 372 additions and 66 deletions.
8 changes: 4 additions & 4 deletions docs/sources/datasources/mssql/query-editor/index.md
Expand Up @@ -87,12 +87,12 @@ Code mode supports autocompletion of tables, columns, SQL keywords, standard SQL

In **Builder mode**, you can build queries using a visual interface.

### Select a dataset and table
### Dataset and table selection

In the **Dataset** dropdown, select the MS SQL database to query.
In the **Dataset** dropdown, select the MSSQL database to query. Grafana populates the dropdown with all databases that the user can access.
Once you select a database, Grafana populates the dropdown with all available tables.

Grafana populates the dropdown with the databases that the configured user can access.
When you select a dataset, Grafana populates the **Table** dropdown with available tables.
**Note:** If a default database has been configured through the Data Source Configuration page (or through a provisioning configuration file), the user will only be able to use that single preconfigured database for querying.

### Select columns and aggregation functions (SELECT)

Expand Down
2 changes: 2 additions & 0 deletions docs/sources/datasources/mysql/_index.md
Expand Up @@ -186,6 +186,8 @@ If your table or database name contains a reserved word or a [not permitted char
In the dataset dropdown, choose the MySQL database to query. The dropdown is be populated with the databases that the user has access to.
When the dataset is selected, the table dropdown is populated with the tables that are available.

**Note:** If a default database has been configured through the Data Source Configuration page (or through a provisioning configuration file), the user will only be able to use that single preconfigured database for querying.

### Columns and Aggregation functions (SELECT)

Using the dropdown, select a column to include in the data. You can also specify an optional aggregation function.
Expand Down
8 changes: 4 additions & 4 deletions docs/sources/datasources/postgres/_index.md
Expand Up @@ -88,18 +88,18 @@ Make sure the user does not get any unwanted privileges from the public role.

## Query builder

{{< figure src="/static/img/docs/v92/postgresql_query_builder.png" class="docs-image--no-shadow" caption="PostgreSQL query builder" >}}
{{< figure src="/static/img/docs/screenshot-postgres-query-editor.png" class="docs-image--no-shadow" caption="PostgreSQL query builder" >}}

The PostgreSQL query builder is available when editing a panel using a PostgreSQL data source. The built query can be run by pressing the `Run query` button in the top right corner of the editor.

### Format

The response from PostgreSQL can be formatted as either a table or as a time series. To use the time series format one of the columns must be named `time`.

### Dataset and Table selection
### Dataset and table selection

In the dataset dropdown, choose the PostgreSQL database to query. The dropdown is be populated with the databases that the user has access to.
When the dataset is selected, the table dropdown is populated with the tables that are available.
The dataset dropdown will be populated with the configured database to which the user has access.
The table dropdown is populated with the tables that are available within that database.

### Columns and Aggregation functions (SELECT)

Expand Down
Expand Up @@ -64,6 +64,7 @@ Some stable features are enabled by default. You can disable a stable feature by
| `faroDatasourceSelector` | Enable the data source selector within the Frontend Apps section of the Frontend Observability |
| `enableDatagridEditing` | Enables the edit functionality in the datagrid panel |
| `dataSourcePageHeader` | Apply new pageHeader UI in data source edit page |
| `sqlDatasourceDatabaseSelection` | Enables previous SQL data source dataset dropdown behavior |

## Alpha feature toggles

Expand Down
1 change: 1 addition & 0 deletions packages/grafana-data/src/types/featureToggles.gen.ts
Expand Up @@ -100,4 +100,5 @@ export interface FeatureToggles {
extraThemes?: boolean;
lokiPredefinedOperations?: boolean;
pluginsFrontendSandbox?: boolean;
sqlDatasourceDatabaseSelection?: boolean;
}
10 changes: 10 additions & 0 deletions pkg/api/frontendsettings.go
Expand Up @@ -375,6 +375,16 @@ func (hs *HTTPServer) getFSDataSources(c *contextmodel.ReqContext, availablePlug
}
}

// Update `jsonData.database` for outdated provisioned SQL datasources created WITHOUT the `jsonData` object in their configuration.
// In these cases, the `Database` value is defined (if at all) on the root level of the provisioning config object.
// This is done for easier warning/error checking on the front end.
if (ds.Type == datasources.DS_MSSQL) || (ds.Type == datasources.DS_MYSQL) || (ds.Type == datasources.DS_POSTGRES) {
// Only update if the value isn't already assigned.
if dsDTO.JSONData["database"] == nil || dsDTO.JSONData["database"] == "" {
dsDTO.JSONData["database"] = ds.Database
}
}

if (ds.Type == datasources.DS_INFLUXDB) || (ds.Type == datasources.DS_ES) {
dsDTO.Database = ds.Database
}
Expand Down
7 changes: 7 additions & 0 deletions pkg/services/featuremgmt/registry.go
Expand Up @@ -555,5 +555,12 @@ var (
FrontendOnly: true,
Owner: grafanaPluginsPlatformSquad,
},
{
Name: "sqlDatasourceDatabaseSelection",
Description: "Enables previous SQL data source dataset dropdown behavior",
FrontendOnly: true,
State: FeatureStateBeta,
Owner: grafanaBiSquad,
},
}
)
1 change: 1 addition & 0 deletions pkg/services/featuremgmt/toggles_gen.csv
Expand Up @@ -81,3 +81,4 @@ dataSourcePageHeader,beta,@grafana/enterprise-datasources,false,false,false,true
extraThemes,alpha,@grafana/grafana-frontend-platform,false,false,false,true
lokiPredefinedOperations,alpha,@grafana/observability-logs,false,false,false,true
pluginsFrontendSandbox,alpha,@grafana/plugins-platform-backend,false,false,false,true
sqlDatasourceDatabaseSelection,beta,@grafana/grafana-bi-squad,false,false,false,true
4 changes: 4 additions & 0 deletions pkg/services/featuremgmt/toggles_gen.go
Expand Up @@ -334,4 +334,8 @@ const (
// FlagPluginsFrontendSandbox
// Enables the plugins frontend sandbox
FlagPluginsFrontendSandbox = "pluginsFrontendSandbox"

// FlagSqlDatasourceDatabaseSelection
// Enables previous SQL data source dataset dropdown behavior
FlagSqlDatasourceDatabaseSelection = "sqlDatasourceDatabaseSelection"
)
73 changes: 52 additions & 21 deletions public/app/features/plugins/sql/components/DatasetSelector.tsx
Expand Up @@ -6,47 +6,78 @@ import { Select } from '@grafana/ui';

import { DB, ResourceSelectorProps, toOption } from '../types';

interface DatasetSelectorProps extends ResourceSelectorProps {
import { isSqlDatasourceDatabaseSelectionFeatureFlagEnabled } from './QueryEditorFeatureFlag.utils';

export interface DatasetSelectorProps extends ResourceSelectorProps {
db: DB;
value: string | null;
applyDefault?: boolean;
disabled?: boolean;
dataset: string | undefined;
preconfiguredDataset: string;
isPostgresInstance: boolean | undefined;
onChange: (v: SelectableValue) => void;
}

export const DatasetSelector = ({ db, value, onChange, disabled, className, applyDefault }: DatasetSelectorProps) => {
export const DatasetSelector = ({
dataset,
db,
isPostgresInstance,
onChange,
preconfiguredDataset,
}: DatasetSelectorProps) => {
/*
The behavior of this component - for MSSQL and MySQL datasources - is based on whether the user chose to create a datasource
with or without a default database (preconfiguredDataset). If the user configured a default database, this selector
should only allow that single preconfigured database option to be selected. If the user chose to NOT assign/configure a default database,
then the user should be able to use this component to choose between multiple databases available to the datasource.
*/
// `hasPreconfigCondition` is true if either 1) the sql datasource has a preconfigured default database,
// OR if 2) the datasource is Postgres. In either case the only option available to the user is the preconfigured database.
const hasPreconfigCondition = !!preconfiguredDataset || isPostgresInstance;

const state = useAsync(async () => {
if (isSqlDatasourceDatabaseSelectionFeatureFlagEnabled()) {
// If a default database is already configured for a MSSQL or MySQL data source, OR the data source is Postgres, no need to fetch other databases.
if (hasPreconfigCondition) {
// Set the current database to the preconfigured database.
onChange(toOption(preconfiguredDataset));
return [toOption(preconfiguredDataset)];
}
}

// If there is no preconfigured database, but there is a selected dataset, set the current database to the selected dataset.
if (dataset) {
onChange(toOption(dataset));
}

// Otherwise, fetch all databases available to the datasource.
const datasets = await db.datasets();
return datasets.map(toOption);
}, []);

useEffect(() => {
if (!applyDefault) {
return;
}
// Set default dataset when values are fetched
if (!value) {
if (state.value && state.value[0]) {
onChange(state.value[0]);
}
} else {
if (state.value && state.value.find((v) => v.value === value) === undefined) {
// if value is set and newly fetched values does not contain selected value
if (state.value.length > 0) {
if (!isSqlDatasourceDatabaseSelectionFeatureFlagEnabled()) {
// Set default dataset when values are fetched
if (!dataset) {
if (state.value && state.value[0]) {
onChange(state.value[0]);
}
} else {
if (state.value && state.value.find((v) => v.value === dataset) === undefined) {
// if value is set and newly fetched values does not contain selected value
if (state.value.length > 0) {
onChange(state.value[0]);
}
}
}
}
}, [state.value, value, applyDefault, onChange]);
}, [state.value, onChange, dataset]);

return (
<Select
className={className}
aria-label="Dataset selector"
value={value}
value={dataset}
options={state.value}
onChange={onChange}
disabled={disabled}
disabled={state.loading}
isLoading={state.loading}
menuShouldPortal={true}
/>
Expand Down
19 changes: 15 additions & 4 deletions public/app/features/plugins/sql/components/QueryEditor.tsx
Expand Up @@ -13,13 +13,23 @@ import { QueryHeader, QueryHeaderProps } from './QueryHeader';
import { RawEditor } from './query-editor-raw/RawEditor';
import { VisualEditor } from './visual-query-builder/VisualEditor';

interface Props extends QueryEditorProps<SqlDatasource, SQLQuery, SQLOptions> {
queryHeaderProps?: Pick<QueryHeaderProps, 'isDatasetSelectorHidden'>;
interface SqlQueryEditorProps extends QueryEditorProps<SqlDatasource, SQLQuery, SQLOptions> {
queryHeaderProps?: Pick<QueryHeaderProps, 'isPostgresInstance'>;
}

export function SqlQueryEditor({ datasource, query, onChange, onRunQuery, range, queryHeaderProps }: Props) {
export function SqlQueryEditor({
datasource,
query,
onChange,
onRunQuery,
range,
queryHeaderProps,
}: SqlQueryEditorProps) {
const [isQueryRunnable, setIsQueryRunnable] = useState(true);
const db = datasource.getDB();

const { preconfiguredDatabase } = datasource;
const isPostgresInstance = !!queryHeaderProps?.isPostgresInstance;
const { loading, error } = useAsync(async () => {
return () => {
if (datasource.getDB(datasource.id).init !== undefined) {
Expand Down Expand Up @@ -80,13 +90,14 @@ export function SqlQueryEditor({ datasource, query, onChange, onRunQuery, range,
<>
<QueryHeader
db={db}
preconfiguredDataset={preconfiguredDatabase}
onChange={onQueryHeaderChange}
onRunQuery={onRunQuery}
onQueryRowChange={setQueryRowFilter}
queryRowFilter={queryRowFilter}
query={queryWithDefaults}
isQueryRunnable={isQueryRunnable}
{...queryHeaderProps}
isPostgresInstance={isPostgresInstance}
/>

<Space v={0.5} />
Expand Down
@@ -0,0 +1,5 @@
import { config } from '@grafana/runtime';

export const isSqlDatasourceDatabaseSelectionFeatureFlagEnabled = () => {
return !!config.featureToggles.sqlDatasourceDatabaseSelection;
};
45 changes: 29 additions & 16 deletions public/app/features/plugins/sql/components/QueryHeader.tsx
Expand Up @@ -10,17 +10,19 @@ import { SQLQuery, QueryFormat, QueryRowFilter, QUERY_FORMAT_OPTIONS, DB } from

import { ConfirmModal } from './ConfirmModal';
import { DatasetSelector } from './DatasetSelector';
import { isSqlDatasourceDatabaseSelectionFeatureFlagEnabled } from './QueryEditorFeatureFlag.utils';
import { TableSelector } from './TableSelector';

export interface QueryHeaderProps {
db: DB;
query: QueryWithDefaults;
isPostgresInstance?: boolean;
isQueryRunnable: boolean;
onChange: (query: SQLQuery) => void;
onRunQuery: () => void;
onQueryRowChange: (queryRowFilter: QueryRowFilter) => void;
onRunQuery: () => void;
preconfiguredDataset: string;
query: QueryWithDefaults;
queryRowFilter: QueryRowFilter;
isQueryRunnable: boolean;
isDatasetSelectorHidden?: boolean;
}

const editorModes = [
Expand All @@ -30,13 +32,14 @@ const editorModes = [

export function QueryHeader({
db,
query,
queryRowFilter,
isPostgresInstance,
isQueryRunnable,
onChange,
onRunQuery,
onQueryRowChange,
isQueryRunnable,
isDatasetSelectorHidden,
onRunQuery,
preconfiguredDataset,
query,
queryRowFilter,
}: QueryHeaderProps) {
const { editorMode } = query;
const [_, copyToClipboard] = useCopyToClipboard();
Expand Down Expand Up @@ -86,9 +89,20 @@ export function QueryHeader({
sql: undefined,
rawSql: '',
};

onChange(next);
};

const datasetDropdownIsAvailable = () => {
// If the feature flag is DISABLED, && the datasource is Postgres (`isPostgresInstance`),
// we want to hide the dropdown - as per previous behavior.
if (!isSqlDatasourceDatabaseSelectionFeatureFlagEnabled() && isPostgresInstance) {
return false;
}

return true;
};

return (
<>
<EditorHeader>
Expand Down Expand Up @@ -205,24 +219,23 @@ export function QueryHeader({
<>
<Space v={0.5} />
<EditorRow>
{isDatasetSelectorHidden ? null : (
{datasetDropdownIsAvailable() && (
<EditorField label="Dataset" width={25}>
<DatasetSelector
db={db}
value={query.dataset === undefined ? null : query.dataset}
dataset={query.dataset}
isPostgresInstance={isPostgresInstance}
preconfiguredDataset={preconfiguredDataset}
onChange={onDatasetChange}
/>
</EditorField>
)}

<EditorField label="Table" width={25}>
<TableSelector
db={db}
query={query}
value={query.table === undefined ? null : query.table}
dataset={query.dataset || preconfiguredDataset}
table={query.table}
onChange={onTableChange}
forceFetch={isDatasetSelectorHidden}
applyDefault
/>
</EditorField>
</EditorRow>
Expand Down

0 comments on commit c0a1fc2

Please sign in to comment.