Loki Prometheus mixin: Fix incorrect selectors and simplify mixin usage #6189

irizzant · 2022-05-18T10:54:54Z

Signed-off-by: irizzant i.rizzante@gmail.com

What this PR does / why we need it:

This PR fixes some issues reported in #4838 with selectors and matchers.

It also adds the kube-prometheus mixin utils needed to automatically generate PrometheusRule objects and dashboards.

This way using the mixin become as easy as adding the following to the main.jsonnet:

{ // this is the main object in main.jsonnet
local addMixin = (import 'github.com/prometheus-operator/kube-prometheus/jsonnet/kube-prometheus/lib/mixin.libsonnet');
local lokiMixinRulesDashboards = addMixin({
  name: 'loki',
  mixin: (import 'github.com/grafana/loki/production/loki-mixin/mixin.libsonnet') + {
    _config+: {},  // mixin configuration object
  },

prometheusRules: lokiMixinRulesDashboards.prometheusRules {
        metadata+: k.meta.v1.objectMeta.withNamespace($.namespace),
      },
});

} + {
      // this adds the ConfigMap's with dashboards json
      local configMap = k.core.v1.configMap,

      [name]: configMap.new('loki-dashboard' + name) +
              configMap.withData({
                [name]: std.manifestJsonMinified(lokiMixinRulesDashboards.grafanaDashboards[name]),
              }) +
              configMap.metadata.withLabelsMixin({
                grafana_dashboard: '1',
              })

      for name in std.objectFields(lokiMixinRulesDashboards.grafanaDashboards)
    },

  }

Which issue(s) this PR fixes:
Fixes #4838

Special notes for your reviewer:

Checklist

Documentation added
Tests updated
Is this an important fix or new feature? Add an entry in the CHANGELOG.md.
Changes that require user attention or interaction to upgrade are documented in docs/sources/upgrading/_index.md

Signed-off-by: irizzant <i.rizzante@gmail.com>

grafanabot · 2022-05-18T10:59:50Z

./tools/diff_coverage.sh ../loki-main/test_results.txt test_results.txt ingester,distributor,querier,querier/queryrange,iter,storage,chunkenc,logql,loki

Change in test coverage per package. Green indicates 0 or positive change, red indicates that test coverage for a package fell.

+           ingester	0%
+        distributor	0.3%
+            querier	0%
+ querier/queryrange	0%
+               iter	0%
+            storage	0%
+           chunkenc	0%
+              logql	0%
+               loki	0%

grafanabot · 2022-05-18T11:03:45Z

./tools/diff_coverage.sh ../loki-main/test_results.txt test_results.txt ingester,distributor,querier,querier/queryrange,iter,storage,chunkenc,logql,loki

Change in test coverage per package. Green indicates 0 or positive change, red indicates that test coverage for a package fell.

+           ingester	0%
+        distributor	0%
+            querier	0%
+ querier/queryrange	0%
+               iter	0%
+            storage	0%
+           chunkenc	0%
+              logql	0%
+               loki	0%

grafanabot · 2022-05-18T11:03:45Z

./tools/diff_coverage.sh ../loki-main/test_results.txt test_results.txt ingester,distributor,querier,querier/queryrange,iter,storage,chunkenc,logql,loki

Change in test coverage per package. Green indicates 0 or positive change, red indicates that test coverage for a package fell.

+           ingester	0%
+        distributor	0%
+            querier	0%
+ querier/queryrange	0%
+               iter	0%
+            storage	0%
+           chunkenc	0%
+              logql	0%
+               loki	0%

Signed-off-by: irizzant <i.rizzante@gmail.com>

irizzant · 2022-05-18T11:45:01Z

@kavirajk can you please have a look at this too? I don't understand why CI is failing since I didn't change anything but jsonnet

grafanabot · 2022-05-18T11:53:03Z

./tools/diff_coverage.sh ../loki-main/test_results.txt test_results.txt ingester,distributor,querier,querier/queryrange,iter,storage,chunkenc,logql,loki

Change in test coverage per package. Green indicates 0 or positive change, red indicates that test coverage for a package fell.

+           ingester	0%
+        distributor	0%
+            querier	0%
+ querier/queryrange	0%
+               iter	0%
+            storage	0%
+           chunkenc	0%
+              logql	0%
+               loki	0%

irizzant · 2022-05-18T13:02:54Z

@kavirajk

I don't understand why CI is failing since I didn't change anything but jsonnet

never mind, it looks like it automagically fixed itself!

grafanabot · 2022-05-22T19:46:54Z

./tools/diff_coverage.sh ../loki-main/test_results.txt test_results.txt ingester,distributor,querier,querier/queryrange,iter,storage,chunkenc,logql,loki

Change in test coverage per package. Green indicates 0 or positive change, red indicates that test coverage for a package fell.

+           ingester	0%
+        distributor	0.3%
+            querier	0%
+ querier/queryrange	0%
+               iter	0%
+            storage	0%
+           chunkenc	0%
+              logql	0%
+               loki	0%

irizzant · 2022-05-27T14:59:29Z

Hey @kavirajk can you have a look at this please?

grafanabot · 2022-05-27T15:05:47Z

./tools/diff_coverage.sh ../loki-main/test_results.txt test_results.txt ingester,distributor,querier,querier/queryrange,iter,storage,chunkenc,logql,loki

Change in test coverage per package. Green indicates 0 or positive change, red indicates that test coverage for a package fell.

+           ingester	0%
-        distributor	-0.3%
+            querier	0%
+ querier/queryrange	0%
+               iter	0%
+            storage	0%
+           chunkenc	0%
+              logql	0%
+               loki	0%

production/loki-mixin/dashboards/dashboard-utils.libsonnet

irizzant · 2022-05-30T17:07:37Z

@kavirajk I resolved the discussion related to this PR , let me know if it's ok for you

irizzant · 2022-06-03T18:30:21Z

@kavirajk can you please let me know if you need anything else to merge this PR ?

Thank you

grafanabot · 2022-06-14T06:57:48Z

./tools/diff_coverage.sh ../loki-main/test_results.txt test_results.txt ingester,distributor,querier,querier/queryrange,iter,storage,chunkenc,logql,loki

Change in test coverage per package. Green indicates 0 or positive change, red indicates that test coverage for a package fell.

+           ingester	0%
-        distributor	-0.3%
+            querier	0%
+ querier/queryrange	0%
+               iter	0%
+            storage	0%
+           chunkenc	0%
+              logql	0%
+               loki	0%

grafanabot · 2022-06-14T20:08:49Z

./tools/diff_coverage.sh ../loki-main/test_results.txt test_results.txt ingester,distributor,querier,querier/queryrange,iter,storage,chunkenc,logql,loki

Change in test coverage per package. Green indicates 0 or positive change, red indicates that test coverage for a package fell.

+           ingester	0%
+        distributor	0%
+            querier	0%
+ querier/queryrange	0%
+               iter	0%
+            storage	0%
+           chunkenc	0%
+              logql	0%
+               loki	0%

Signed-off-by: irizzant <i.rizzante@gmail.com>

grafanabot · 2022-06-14T20:25:19Z

./tools/diff_coverage.sh ../loki-main/test_results.txt test_results.txt ingester,distributor,querier,querier/queryrange,iter,storage,chunkenc,logql,loki

Change in test coverage per package. Green indicates 0 or positive change, red indicates that test coverage for a package fell.

+           ingester	0%
+        distributor	0%
+            querier	0%
+ querier/queryrange	0%
+               iter	0%
+            storage	0%
+           chunkenc	0%
+              logql	0%
+               loki	0%

This reverts commit ba24006.

irizzant · 2022-06-15T06:37:51Z

Hi @trevorwhitney is it possible to get the PR merged? I had to solve some conflicts...

grafanabot · 2022-06-15T06:42:02Z

./tools/diff_coverage.sh ../loki-main/test_results.txt test_results.txt ingester,distributor,querier,querier/queryrange,iter,storage,chunkenc,logql,loki

Change in test coverage per package. Green indicates 0 or positive change, red indicates that test coverage for a package fell.

+           ingester	0%
+        distributor	0%
+            querier	0%
+ querier/queryrange	0%
+               iter	0%
+            storage	0%
+           chunkenc	0%
+              logql	0%
+               loki	0%

irizzant · 2022-06-27T09:05:21Z

Hi @trevorwhitney @kavirajk , can you please have a look? I fixed the conflicts and it should be ready to be merged now

grafanabot · 2022-06-27T09:09:21Z

./tools/diff_coverage.sh ../loki-main/test_results.txt test_results.txt ingester,distributor,querier,querier/queryrange,iter,storage,chunkenc,logql,loki

Change in test coverage per package. Green indicates 0 or positive change, red indicates that test coverage for a package fell.

+           ingester	0%
-        distributor	-0.3%
+            querier	0%
+ querier/queryrange	0%
+               iter	0%
+            storage	0%
+           chunkenc	0%
+              logql	0%
+               loki	0%

kavirajk

@irizzant Thanks for your patience. LGTM, approach with service monitor.

kavirajk · 2022-06-27T09:23:27Z

@irizzant Looks like mixin-check is failing. Can you take care of it? I can merge after that.

production/ksonnet/loki/config.libsonnet

grafanabot · 2022-06-27T12:41:21Z

./tools/diff_coverage.sh ../loki-main/test_results.txt test_results.txt ingester,distributor,querier,querier/queryrange,iter,storage,chunkenc,logql,loki

Change in test coverage per package. Green indicates 0 or positive change, red indicates that test coverage for a package fell.

+           ingester	0%
-        distributor	-0.3%
+            querier	0%
- querier/queryrange	-0.1%
+               iter	0%
+            storage	0%
+           chunkenc	0%
+              logql	0%
+               loki	0%

Signed-off-by: irizzant <i.rizzante@gmail.com>

grafanabot · 2022-06-28T12:25:12Z

./tools/diff_coverage.sh ../loki-main/test_results.txt test_results.txt ingester,distributor,querier,querier/queryrange,iter,storage,chunkenc,logql,loki

Change in test coverage per package. Green indicates 0 or positive change, red indicates that test coverage for a package fell.

+           ingester	0%
+        distributor	0%
+            querier	0%
+ querier/queryrange	0%
+               iter	0%
+            storage	0%
+           chunkenc	0%
+              logql	0%
+               loki	0%

irizzant · 2022-06-28T13:20:50Z

@kavirajk @trevorwhitney , the checks are passing.

I don't know why they failed before, I just merged the main branch.

fix(dashboards): fix matchers and selectors

5ccf17e

Signed-off-by: irizzant <i.rizzante@gmail.com>

irizzant requested a review from a team as a code owner May 18, 2022 10:54

pull-request-size bot added the size/S label May 18, 2022

doc: add entry in CHANGELOG

3aed910

pull-request-size bot added size/M and removed size/S labels May 18, 2022

Merge branch 'grafana:main' into tmp

54abda2

irizzant changed the title ~~Loki Prometheus mixin: fix incorrect selectors and matchers~~ Loki Prometheus mixin: Fix incorrect selectors and matchers May 18, 2022

irizzant changed the title ~~Loki Prometheus mixin: Fix incorrect selectors and matchers~~ Loki Prometheus mixin: Fix incorrect selectors and simplify mixin usage May 18, 2022

feat(dashboards): add kube-prometheus mixin

81689fd

Signed-off-by: irizzant <i.rizzante@gmail.com>

doc: update CHANGELOG

8d3b5b0

irizzant mentioned this pull request May 18, 2022

loki-mixin: Complex configuration needed compared to other mixins #4838

Closed

irizzant added 3 commits May 19, 2022 13:44

Merge branch 'grafana:main' into tmp

b3521a8

Merge branch 'grafana:main' into tmp

f14b6e3

Merge branch 'grafana:main' into tmp

980974e

Merge branch 'main' into tmp

370667e

kavirajk reviewed May 30, 2022

View reviewed changes

production/loki-mixin/dashboards/dashboard-utils.libsonnet Outdated Show resolved Hide resolved

Merge branch 'grafana:main' into tmp

1b1abe5

Merge branch 'grafana:main' into tmp

8805346

Merge branch 'main' into tmp

ed6e1d8

irizzant requested a review from trevorwhitney June 14, 2022 08:35

Merge branch 'main' into tmp

3c3e49a

fix(dashboards): add ServiceMonitor to Loki Simple Scalable

ba24006

Signed-off-by: irizzant <i.rizzante@gmail.com>

pull-request-size bot added size/L and removed size/M labels Jun 14, 2022

Revert "fix(dashboards): add ServiceMonitor to Loki Simple Scalable"

7a42b4b

This reverts commit ba24006.

pull-request-size bot added size/M and removed size/L labels Jun 14, 2022

Merge branch 'grafana:main' into tmp

619f77c

Merge branch 'main' into tmp

0b9a687

kavirajk approved these changes Jun 27, 2022

View reviewed changes

kavirajk reviewed Jun 27, 2022

View reviewed changes

production/ksonnet/loki/config.libsonnet Show resolved Hide resolved

Merge branch 'grafana:main' into tmp

39a210d

irizzant requested a review from kavirajk June 27, 2022 12:38

irizzant added 2 commits June 27, 2022 12:46

doc: add details about create_service_monitor` parameter

3dad062

Signed-off-by: irizzant <i.rizzante@gmail.com>

Merge branch 'grafana:main' into tmp

4002c39

trevorwhitney merged commit 002dd8a into grafana:main Jun 28, 2022

agermain mentioned this pull request Sep 5, 2022

Make loki-mixin dashboards work out of the box with Helm chart #6266

Closed

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Loki Prometheus mixin: Fix incorrect selectors and simplify mixin usage #6189

Loki Prometheus mixin: Fix incorrect selectors and simplify mixin usage #6189

irizzant commented May 18, 2022 •

edited

grafanabot commented May 18, 2022

grafanabot commented May 18, 2022

grafanabot commented May 18, 2022

irizzant commented May 18, 2022

grafanabot commented May 18, 2022

irizzant commented May 18, 2022 •

edited

grafanabot commented May 22, 2022

irizzant commented May 27, 2022

grafanabot commented May 27, 2022

irizzant commented May 30, 2022

irizzant commented Jun 3, 2022

grafanabot commented Jun 14, 2022

grafanabot commented Jun 14, 2022

grafanabot commented Jun 14, 2022

irizzant commented Jun 15, 2022

grafanabot commented Jun 15, 2022

irizzant commented Jun 27, 2022

grafanabot commented Jun 27, 2022

kavirajk left a comment

kavirajk commented Jun 27, 2022

grafanabot commented Jun 27, 2022

grafanabot commented Jun 28, 2022

irizzant commented Jun 28, 2022

Loki Prometheus mixin: Fix incorrect selectors and simplify mixin usage #6189

Loki Prometheus mixin: Fix incorrect selectors and simplify mixin usage #6189

Conversation

irizzant commented May 18, 2022 • edited

grafanabot commented May 18, 2022

grafanabot commented May 18, 2022

grafanabot commented May 18, 2022

irizzant commented May 18, 2022

grafanabot commented May 18, 2022

irizzant commented May 18, 2022 • edited

grafanabot commented May 22, 2022

irizzant commented May 27, 2022

grafanabot commented May 27, 2022

irizzant commented May 30, 2022

irizzant commented Jun 3, 2022

grafanabot commented Jun 14, 2022

grafanabot commented Jun 14, 2022

grafanabot commented Jun 14, 2022

irizzant commented Jun 15, 2022

grafanabot commented Jun 15, 2022

irizzant commented Jun 27, 2022

grafanabot commented Jun 27, 2022

kavirajk left a comment

Choose a reason for hiding this comment

kavirajk commented Jun 27, 2022

grafanabot commented Jun 27, 2022

grafanabot commented Jun 28, 2022

irizzant commented Jun 28, 2022

irizzant commented May 18, 2022 •

edited

irizzant commented May 18, 2022 •

edited