Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Graphite: Expand metric names for variables #33694

Merged
merged 12 commits into from
May 11, 2021

Conversation

ifrost
Copy link
Contributor

@ifrost ifrost commented May 4, 2021

What this PR does / why we need it:

Variable query for Graphite allows to search for metrics, however it's quite limited because results contain possible values but occurring only at the last level of the query (please check the example below). This PR introduces "expand" function allowing to use expand API to get the full name of matching metrics. In combination with regex it will allow to extract any part of the metric name.

Example:

Metrics:

  • prod.servers.server001.cpu
  • prod.servers.server002.cpu
  • test.servers.server001.cpu

Search results: (normal query is already available, expanded query support is added in this PR)

normal query results expanded query expanded results
* prod, test expand(*) prod, test
*.servers servers expand(*.servers) prod.servers, test.servers
test.servers servers expand(test.servers) test.servers
*.servers.* server001,server002 expand(*.servers.*) prod.servers.server001, prod.servers.server002, test.servers.server001
test.servers.* server001 expand(test.servers.*) test.servers.server001
*.servers.*.cpu cpu expand(*.servers.*.cpu) prod.servers.server001.cpu, prod.servers.server002.cpu, test.servers.server001.cpu

⚠️ Note: "normal query" is the same as "expand query" with a regex matching the last part of the name.

Why is it important?

The available implementation allows to handle only simpler (though most common) cases where only the last part of the metric should become a variable value. However, it's not possible extract other parts of metric names.

Imagine you needed a dropdown to select the environment (prod or test in the example above). Normal search could capture the top level name but only of all metrics you have. Let's say besides prod.* and test.* metrics you had statsd.*. With normal query "statsd" would also be captured. With "expand" it's possible to write do this:

Query: expand(*.servers)
Regex: /^(\w+).*/

Expand will first find only metrics that have servers in the second level and then regex will capture only environment names.

How to test it?

Run: make devenv sources=graphite
Send some sample data: https://gist.github.com/ifrost/ada07138775d5295fba169d27abddf8d

Create a dashboard with following query: $env.servers.$server.cpu

Screenshot 2021-05-04 at 14 34 08

Create variables:

env:

Screenshot 2021-05-04 at 14 35 14

and server:

Screenshot 2021-05-04 at 14 35 24

Which issue(s) this PR fixes:

Fixes #2343

Special notes for your reviewer: Same example is included in updated docs.

@ifrost ifrost requested a review from a team as a code owner May 4, 2021 12:36
@ifrost ifrost requested review from a team, oddlittlebird, achatterjee-grafana, osg-grafana and gabor and removed request for a team May 4, 2021 12:36
@ifrost ifrost requested a review from a team May 4, 2021 12:37
Copy link
Contributor

@achatterjee-grafana achatterjee-grafana left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Is this for V8.0?

docs/sources/datasources/graphite.md Outdated Show resolved Hide resolved
docs/sources/datasources/graphite.md Outdated Show resolved Hide resolved
@ifrost ifrost added this to the 8.0.0 milestone May 4, 2021
ifrost and others added 2 commits May 4, 2021 19:31
Co-authored-by: achatterjee-grafana <70489351+achatterjee-grafana@users.noreply.github.com>
Co-authored-by: achatterjee-grafana <70489351+achatterjee-grafana@users.noreply.github.com>
@ifrost
Copy link
Contributor Author

ifrost commented May 4, 2021

Is this for V8.0?

Yes! Sorry, forgot about the milestone. Updated and adding "add to changelog" label.

Copy link
Contributor

@achatterjee-grafana achatterjee-grafana left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Added change suggestions.

docs/sources/datasources/graphite.md Show resolved Hide resolved
docs/sources/datasources/graphite.md Outdated Show resolved Hide resolved
docs/sources/datasources/graphite.md Show resolved Hide resolved
docs/sources/datasources/graphite.md Outdated Show resolved Hide resolved
docs/sources/datasources/graphite.md Outdated Show resolved Hide resolved
docs/sources/datasources/graphite.md Show resolved Hide resolved
ifrost and others added 7 commits May 5, 2021 08:44
Co-authored-by: achatterjee-grafana <70489351+achatterjee-grafana@users.noreply.github.com>
Co-authored-by: achatterjee-grafana <70489351+achatterjee-grafana@users.noreply.github.com>
Co-authored-by: achatterjee-grafana <70489351+achatterjee-grafana@users.noreply.github.com>
Co-authored-by: achatterjee-grafana <70489351+achatterjee-grafana@users.noreply.github.com>
Co-authored-by: achatterjee-grafana <70489351+achatterjee-grafana@users.noreply.github.com>
Copy link
Contributor

@achatterjee-grafana achatterjee-grafana left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Doc looks ok, we can refine it further if needed.

Copy link
Contributor

@gabor gabor left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

looks good to me 👍 !

@ifrost ifrost merged commit 60cf0c8 into main May 11, 2021
@ifrost ifrost deleted the ifrost/graphite-expand-metrics-for-variables branch May 11, 2021 07:39
ryantxu pushed a commit that referenced this pull request May 13, 2021
* Allow expanding metrics names in variable queries

* Simplify docs

* Update docs/sources/datasources/graphite.md

Co-authored-by: achatterjee-grafana <70489351+achatterjee-grafana@users.noreply.github.com>

* Update docs/sources/datasources/graphite.md

Co-authored-by: achatterjee-grafana <70489351+achatterjee-grafana@users.noreply.github.com>

* Update docs/sources/datasources/graphite.md

Co-authored-by: achatterjee-grafana <70489351+achatterjee-grafana@users.noreply.github.com>

* Update docs/sources/datasources/graphite.md

Co-authored-by: achatterjee-grafana <70489351+achatterjee-grafana@users.noreply.github.com>

* Update docs/sources/datasources/graphite.md

Co-authored-by: achatterjee-grafana <70489351+achatterjee-grafana@users.noreply.github.com>

* Update docs/sources/datasources/graphite.md

Co-authored-by: achatterjee-grafana <70489351+achatterjee-grafana@users.noreply.github.com>

* Update docs/sources/datasources/graphite.md

Co-authored-by: achatterjee-grafana <70489351+achatterjee-grafana@users.noreply.github.com>

* Clean up docs

Co-authored-by: achatterjee-grafana <70489351+achatterjee-grafana@users.noreply.github.com>
mortenaa pushed a commit to mortenaa/grafana that referenced this pull request May 25, 2021
* Allow expanding metrics names in variable queries

* Simplify docs

* Update docs/sources/datasources/graphite.md

Co-authored-by: achatterjee-grafana <70489351+achatterjee-grafana@users.noreply.github.com>

* Update docs/sources/datasources/graphite.md

Co-authored-by: achatterjee-grafana <70489351+achatterjee-grafana@users.noreply.github.com>

* Update docs/sources/datasources/graphite.md

Co-authored-by: achatterjee-grafana <70489351+achatterjee-grafana@users.noreply.github.com>

* Update docs/sources/datasources/graphite.md

Co-authored-by: achatterjee-grafana <70489351+achatterjee-grafana@users.noreply.github.com>

* Update docs/sources/datasources/graphite.md

Co-authored-by: achatterjee-grafana <70489351+achatterjee-grafana@users.noreply.github.com>

* Update docs/sources/datasources/graphite.md

Co-authored-by: achatterjee-grafana <70489351+achatterjee-grafana@users.noreply.github.com>

* Update docs/sources/datasources/graphite.md

Co-authored-by: achatterjee-grafana <70489351+achatterjee-grafana@users.noreply.github.com>

* Clean up docs

Co-authored-by: achatterjee-grafana <70489351+achatterjee-grafana@users.noreply.github.com>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
Development

Successfully merging this pull request may close these issues.

Templating: Regex on full graphite expended matches
4 participants