Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Plugins: Add deprecation notice for /api/tsdb/query endpoint #45238

Merged
merged 29 commits into from Mar 30, 2022
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
29 commits
Select commit Hold shift + click to select a range
286b75d
add deprecation notice for /api/tsdb/query
wbrowne Feb 8, 2022
b0d6f66
Merge branch 'main' into deprecate-tsdb-query
wbrowne Feb 8, 2022
bc11a75
Merge branch 'main' into deprecate-tsdb-query
wbrowne Feb 9, 2022
18ef8a8
fix linking
wbrowne Feb 9, 2022
0520d71
regenerate after gen-go
wbrowne Feb 9, 2022
c09cbd6
Merge branch 'main' into deprecate-tsdb-query
wbrowne Feb 9, 2022
5e83aa2
add newline
wbrowne Feb 9, 2022
0a02280
Merge branch 'main' into deprecate-tsdb-query
wbrowne Feb 10, 2022
038fd4c
add API docs for ds/query
wbrowne Feb 10, 2022
b5518dd
Merge branch 'main' into deprecate-tsdb-query
wbrowne Feb 10, 2022
ee31489
regenerate spec
wbrowne Feb 10, 2022
f21b975
Merge branch 'main' into deprecate-tsdb-query
wbrowne Feb 10, 2022
48863b7
Merge branch 'main' into deprecate-tsdb-query
wbrowne Feb 11, 2022
547fbe3
pr feedback
wbrowne Mar 1, 2022
bc06246
add helpful tip
wbrowne Mar 1, 2022
222dbc4
make sub heading
wbrowne Mar 1, 2022
a83b96f
Merge branch 'main' into deprecate-tsdb-query
wbrowne Mar 3, 2022
b9b8c99
add more data
wbrowne Mar 3, 2022
f6fa776
update spec
wbrowne Mar 3, 2022
19ec3ec
merge with main
wbrowne Mar 3, 2022
cad678a
resolve conflicts
wbrowne Mar 11, 2022
fa6a8e1
resolve conflicts
wbrowne Mar 16, 2022
6a4c9eb
Merge branch 'main' into deprecate-tsdb-query
wbrowne Mar 24, 2022
c8547a4
update wording
wbrowne Mar 24, 2022
da971d4
mention both from/to
wbrowne Mar 24, 2022
a89edfa
add suggestions
wbrowne Mar 24, 2022
e47d1e1
Merge branch 'main' into deprecate-tsdb-query
wbrowne Mar 24, 2022
e0b4156
docs feedback
wbrowne Mar 24, 2022
06a5b3d
resolve conflicts
wbrowne Mar 24, 2022
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Jump to
Jump to file
Failed to load files.
Diff view
Diff view
114 changes: 103 additions & 11 deletions docs/sources/http_api/data_source.md
Expand Up @@ -604,13 +604,106 @@ Content-Type: application/json

Proxies all calls to the actual data source.

## Query a data source by ID
## Query a data source

Queries a data source having backend implementation.
Queries a data source having a backend implementation.

`POST /api/ds/query`

> **Note:** Grafana's built-in data sources usually have a backend implementation.

**Example request for the Test data source**:

```http
POST /api/ds/query HTTP/1.1
Accept: application/json
Content-Type: application/json

{
"queries":[
{
"refId":"A",
"scenarioId":"csv_metric_values",
"datasource":{
"uid":"PD8C576611E62080A"
wbrowne marked this conversation as resolved.
Show resolved Hide resolved
},
"format": "table"
"maxDataPoints":1848,
"intervalMs":200,
"stringInput":"1,20,90,30,5,0",
}
],
"from":"now-5m",
"to":"now"
}
```

JSON Body schema:

- **from/to** – Specifies the time range for the queries. The time can be either epoch timestamps in milliseconds or relative using Grafana time units. For example, `now-5m`.
- **queries** – Specifies one or more queries. Must contain at least 1.
wbrowne marked this conversation as resolved.
Show resolved Hide resolved
- **queries.datasource.uid** – Specifies the UID of data source to be queried. Each query in the request must have a unique `datasource`.
wbrowne marked this conversation as resolved.
Show resolved Hide resolved
- **queries.refId** – Specifies an identifier of the query. Defaults to "A".
- **queries.format** – Specifies the format the data should be returned in. Valid options are `time_series` or `table` depending on the data source.
- **queries.maxDataPoints** - Species the maximum amount of data points that a dashboard panel can render. Defaults to 100.
- **queries.intervalMs** - Specifies the time series time interval in milliseconds. Defaults to 1000.

In addition, specific properties of each data source should be added in a request (for example **queries.stringInput** as shown in the request above). To better understand how to form a query for a certain data source, use the Developer Tools in your browser of choice and inspect the HTTP requests being made to `/api/ds/query`.

**Example Test data source time series query response:**

```json
{
"results": {
"A": {
"frames": [
{
"schema": {
"refId": "A",
"fields": [
{
"name": "time",
"type": "time",
"typeInfo": {
"frame": "time.Time"
}
},
{
"name": "A-series",
"type": "number",
"typeInfo": {
"frame": "int64",
"nullable": true
}
}
]
},
"data": {
"values": [
[1644488152084, 1644488212084, 1644488272084, 1644488332084, 1644488392084, 1644488452084],
[1, 20, 90, 30, 5, 0]
]
}
}
]
}
}
}
```

## Deprecated resources

The following resources have been deprecated. They will be removed in a future release.

### Query a data source by ID

> **Warning:** This API is deprecated since Grafana v8.5.0 and will be removed in a future release. Refer to the [new data source query API](#query-a-data-source-by-id).

Queries a data source having a backend implementation.

`POST /api/tsdb/query`

> **Note:** Most of Grafana's builtin data sources have backend implementation.
> **Note:** Grafana's built-in data sources usually have a backend implementation.

**Example Request**:

Expand All @@ -635,17 +728,16 @@ Content-Type: application/json
}
```

> **Note:** The `from`, `to`, and `queries` properties are required.

JSON Body schema:

- **from/to** – Should be either absolute in epoch timestamps in milliseconds or relative using Grafana time units. For example, `now-1h`.
- **queries.refId** – Specifies an identifier of the query. Is optional and default to "A".
- **queries.datasourceId** – Specifies the data source to be queried. Each `query` in the request must have an unique `datasourceId`.
- **queries.maxDataPoints** - Species maximum amount of data points that dashboard panel can render. Is optional and default to 100.
- **queries.intervalMs** - Specifies the time interval in milliseconds of time series. Is optional and defaults to 1000.
- **from/to** – Specifies the time range for the queries. The time can be either epoch timestamps in milliseconds or relative using Grafana time units. For example, `now-5m`.
- **queries.refId** – Specifies an identifier of the query. Defaults to "A".
- **queries.format** – Specifies the format the data should be returned in. Valid options are `time_series` or `table` depending on the data source.
- **queries.datasourceId** – Specifies the data source to be queried. Each `query` in the request must have a unique `datasourceId`.
- **queries.maxDataPoints** - Species the maximum amount of data points that a dashboard panel can render. Defaults to 100.
- **queries.intervalMs** - Specifies the time series time interval in milliseconds. Defaults to 1000.

In addition, each data source has its own specific properties that should be added in a request.
In addition, specific properties of each data source should be added in a request. To better understand how to form a query for a certain data source, use the Developer Tools in your browser of choice and inspect the HTTP requests being made to `/api/tsdb/query`.

**Example request for the MySQL data source:**

Expand Down
1 change: 1 addition & 0 deletions pkg/api/api.go
Expand Up @@ -399,6 +399,7 @@ func (hs *HTTPServer) registerRoutes() {
apiRoute.Get("/search/", routing.Wrap(hs.Search))

// metrics
// Deprecated: use /ds/query API instead.
marefr marked this conversation as resolved.
Show resolved Hide resolved
apiRoute.Post("/tsdb/query", authorize(reqSignedIn, ac.EvalPermission(datasources.ActionQuery)), routing.Wrap(hs.QueryMetrics))

// DataSource w/ expressions
Expand Down
4 changes: 4 additions & 0 deletions pkg/api/docs/definitions/datasources.go
Expand Up @@ -200,13 +200,17 @@ import (
//
// Query metrics.
//
// Please refer to [updated API](#/ds/queryMetricsWithExpressions) instead
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I think that we might need to update the queryMetricsWithExpressions definition to make it output proper response example in open api spec because the data frames conversion to JSON is custom, but I might be wrong?

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Do you mean when it returns encoded arrow? AFAIK everything is JSON nowadays https://github.com/grafana/grafana/blob/main/pkg/api/metrics.go#L98

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Was referring to JSON, but haven't checked swagger or the generated open api spec so not sure. Mainly referring to this https://github.com/grafana/grafana-plugin-sdk-go/blob/a6e537820b7b4554637de588e87ca2d77b8cad90/backend/data.go#L74-L82 that writes the JSON in a custom way by iterating the responses/frames.

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Checked using swagger now, http://localhost:3000/swagger-ui. This might be a bigger problem than just this endpoint and potentially something we might attack later. But here it goes.

The request body example includes the following:

{
  "debug": true,
  "from": "now-1h",
  "queries": [
    {
      "datasourceId": 86,
      "format": "table",
      "intervalMs": 86400000,
      "maxDataPoints": 1092,
      "rawSql": "SELECT 1 as valueOne, 2 as valueTwo",
      "refId": "A"
    }
  ],
  "to": "now"
}

The 200 response includes the following:

Details
{
  "Responses": {
    "additionalProp1": {
      "message": "string",
      "results": {
        "additionalProp1": {
          "dataframes": {},
          "error": "string",
          "meta": {},
          "refId": "string",
          "series": [
            {
              "name": "string",
              "points": [
                [
                  {
                    "Float64": 0,
                    "Valid": true
                  }
                ]
              ],
              "tags": {
                "additionalProp1": "string",
                "additionalProp2": "string",
                "additionalProp3": "string"
              }
            }
          ],
          "tables": [
            {
              "columns": [
                {
                  "text": "string"
                }
              ],
              "rows": [
                [
                  {}
                ]
              ]
            }
          ]
        },
        "additionalProp2": {
          "dataframes": {},
          "error": "string",
          "meta": {},
          "refId": "string",
          "series": [
            {
              "name": "string",
              "points": [
                [
                  {
                    "Float64": 0,
                    "Valid": true
                  }
                ]
              ],
              "tags": {
                "additionalProp1": "string",
                "additionalProp2": "string",
                "additionalProp3": "string"
              }
            }
          ],
          "tables": [
            {
              "columns": [
                {
                  "text": "string"
                }
              ],
              "rows": [
                [
                  {}
                ]
              ]
            }
          ]
        },
        "additionalProp3": {
          "dataframes": {},
          "error": "string",
          "meta": {},
          "refId": "string",
          "series": [
            {
              "name": "string",
              "points": [
                [
                  {
                    "Float64": 0,
                    "Valid": true
                  }
                ]
              ],
              "tags": {
                "additionalProp1": "string",
                "additionalProp2": "string",
                "additionalProp3": "string"
              }
            }
          ],
          "tables": [
            {
              "columns": [
                {
                  "text": "string"
                }
              ],
              "rows": [
                [
                  {}
                ]
              ]
            }
          ]
        }
      }
    },
    "additionalProp2": {
      "message": "string",
      "results": {
        "additionalProp1": {
          "dataframes": {},
          "error": "string",
          "meta": {},
          "refId": "string",
          "series": [
            {
              "name": "string",
              "points": [
                [
                  {
                    "Float64": 0,
                    "Valid": true
                  }
                ]
              ],
              "tags": {
                "additionalProp1": "string",
                "additionalProp2": "string",
                "additionalProp3": "string"
              }
            }
          ],
          "tables": [
            {
              "columns": [
                {
                  "text": "string"
                }
              ],
              "rows": [
                [
                  {}
                ]
              ]
            }
          ]
        },
        "additionalProp2": {
          "dataframes": {},
          "error": "string",
          "meta": {},
          "refId": "string",
          "series": [
            {
              "name": "string",
              "points": [
                [
                  {
                    "Float64": 0,
                    "Valid": true
                  }
                ]
              ],
              "tags": {
                "additionalProp1": "string",
                "additionalProp2": "string",
                "additionalProp3": "string"
              }
            }
          ],
          "tables": [
            {
              "columns": [
                {
                  "text": "string"
                }
              ],
              "rows": [
                [
                  {}
                ]
              ]
            }
          ]
        },
        "additionalProp3": {
          "dataframes": {},
          "error": "string",
          "meta": {},
          "refId": "string",
          "series": [
            {
              "name": "string",
              "points": [
                [
                  {
                    "Float64": 0,
                    "Valid": true
                  }
                ]
              ],
              "tags": {
                "additionalProp1": "string",
                "additionalProp2": "string",
                "additionalProp3": "string"
              }
            }
          ],
          "tables": [
            {
              "columns": [
                {
                  "text": "string"
                }
              ],
              "rows": [
                [
                  {}
                ]
              ]
            }
          ]
        }
      }
    },
    "additionalProp3": {
      "message": "string",
      "results": {
        "additionalProp1": {
          "dataframes": {},
          "error": "string",
          "meta": {},
          "refId": "string",
          "series": [
            {
              "name": "string",
              "points": [
                [
                  {
                    "Float64": 0,
                    "Valid": true
                  }
                ]
              ],
              "tags": {
                "additionalProp1": "string",
                "additionalProp2": "string",
                "additionalProp3": "string"
              }
            }
          ],
          "tables": [
            {
              "columns": [
                {
                  "text": "string"
                }
              ],
              "rows": [
                [
                  {}
                ]
              ]
            }
          ]
        },
        "additionalProp2": {
          "dataframes": {},
          "error": "string",
          "meta": {},
          "refId": "string",
          "series": [
            {
              "name": "string",
              "points": [
                [
                  {
                    "Float64": 0,
                    "Valid": true
                  }
                ]
              ],
              "tags": {
                "additionalProp1": "string",
                "additionalProp2": "string",
                "additionalProp3": "string"
              }
            }
          ],
          "tables": [
            {
              "columns": [
                {
                  "text": "string"
                }
              ],
              "rows": [
                [
                  {}
                ]
              ]
            }
          ]
        },
        "additionalProp3": {
          "dataframes": {},
          "error": "string",
          "meta": {},
          "refId": "string",
          "series": [
            {
              "name": "string",
              "points": [
                [
                  {
                    "Float64": 0,
                    "Valid": true
                  }
                ]
              ],
              "tags": {
                "additionalProp1": "string",
                "additionalProp2": "string",
                "additionalProp3": "string"
              }
            }
          ],
          "tables": [
            {
              "columns": [
                {
                  "text": "string"
                }
              ],
              "rows": [
                [
                  {}
                ]
              ]
            }
          ]
        }
      }
    }
  }
}

//
// Queries a data source having backend implementation.
//
// Most of Grafana’s builtin data sources have backend implementation.
//
// If you are running Grafana Enterprise and have Fine-grained access control enabled
// you need to have a permission with action: `datasources:query`.
//
// Deprecated: true
//
// Responses:
// 200: queryDatasourceResponse
// 401: unauthorisedError
Expand Down
2 changes: 1 addition & 1 deletion pkg/api/dtos/models.go
Expand Up @@ -63,7 +63,7 @@ type MetricRequest struct {
// queries.maxDataPoints - Species maximum amount of data points that dashboard panel can render. Is optional and default to 100.
// queries.intervalMs - Specifies the time interval in milliseconds of time series. Is optional and defaults to 1000.
// required: true
// example: [ { "refId": "A", "intervalMs": 86400000, "maxDataPoints": 1092, "datasourceId": 86, "rawSql": "SELECT 1 as valueOne, 2 as valueTwo", "format": "table" } ]
// example: [ { "refId": "A", "intervalMs": 86400000, "maxDataPoints": 1092, "datasource":{ "uid":"PD8C576611E62080A" }, "rawSql": "SELECT 1 as valueOne, 2 as valueTwo", "format": "table" } ]
Queries []*simplejson.Json `json:"queries"`
// required: false
Debug bool `json:"debug"`
Expand Down
1 change: 1 addition & 0 deletions pkg/api/metrics.go
Expand Up @@ -145,6 +145,7 @@ func (hs *HTTPServer) QueryMetricsFromDashboard(c *models.ReqContext) response.R
// POST /api/tsdb/query
//nolint: staticcheck // legacydata.DataResponse deprecated
//nolint: staticcheck // legacydata.DataQueryResult deprecated
// Deprecated: use QueryMetricsV2 instead.
func (hs *HTTPServer) QueryMetrics(c *models.ReqContext) response.Response {
reqDto := dtos.MetricRequest{}
if err := web.Bind(c.Req, &reqDto); err != nil {
Expand Down
7 changes: 5 additions & 2 deletions public/api-merged.json
Expand Up @@ -8299,10 +8299,11 @@
},
"/tsdb/query": {
"post": {
"description": "Queries a data source having backend implementation.\n\nMost of Grafana’s builtin data sources have backend implementation.\n\nIf you are running Grafana Enterprise and have Fine-grained access control enabled\nyou need to have a permission with action: `datasources:query`.",
"description": "Please refer to [updated API](#/ds/queryMetricsWithExpressions) instead\n\nQueries a data source having backend implementation.\n\nMost of Grafana’s builtin data sources have backend implementation.\n\nIf you are running Grafana Enterprise and have Fine-grained access control enabled\nyou need to have a permission with action: `datasources:query`.",
"tags": ["datasources"],
"summary": "Query metrics.",
"operationId": "queryDatasource",
"deprecated": true,
"parameters": [
{
"name": "Body",
Expand Down Expand Up @@ -13100,7 +13101,9 @@
"x-go-name": "Queries",
"example": [
{
"datasourceId": 86,
"datasource": {
"uid": "PD8C576611E62080A"
},
"format": "table",
"intervalMs": 86400000,
"maxDataPoints": 1092,
Expand Down
7 changes: 5 additions & 2 deletions public/api-spec.json
Expand Up @@ -6868,10 +6868,11 @@
},
"/tsdb/query": {
"post": {
"description": "Queries a data source having backend implementation.\n\nMost of Grafana’s builtin data sources have backend implementation.\n\nIf you are running Grafana Enterprise and have Fine-grained access control enabled\nyou need to have a permission with action: `datasources:query`.",
"description": "Please refer to [updated API](#/ds/queryMetricsWithExpressions) instead\n\nQueries a data source having backend implementation.\n\nMost of Grafana’s builtin data sources have backend implementation.\n\nIf you are running Grafana Enterprise and have Fine-grained access control enabled\nyou need to have a permission with action: `datasources:query`.",
"tags": ["datasources"],
"summary": "Query metrics.",
"operationId": "queryDatasource",
"deprecated": true,
"parameters": [
{
"name": "Body",
Expand Down Expand Up @@ -10425,7 +10426,9 @@
"x-go-name": "Queries",
"example": [
{
"datasourceId": 86,
"datasource": {
"uid": "PD8C576611E62080A"
},
"format": "table",
"intervalMs": 86400000,
"maxDataPoints": 1092,
Expand Down