Bug 1768483: Limit Cardinality of Marketplace quay metrics #259

awgreene · 2019-10-28T16:49:46Z

This commit introduces a series of record rules that lump http status codes returned by Default AppRegistries into buckets of 100.

Reviewer Checklist

Implementation matches the proposed design, or proposal is updated to match implementation
Sufficient unit test coverage
Sufficient end-to-end test coverage
Docs updated or added to /docs
Commit messages sensible and descriptive

Solves #264

kevinrizza · 2019-10-28T17:25:09Z

/lgtm

awgreene · 2019-10-28T17:41:59Z

/hold

Waiting for someone from telemeter to comment before merging.

manifests/12_prometheus_rule.yaml

awgreene · 2019-10-30T15:47:18Z

@brancz should be all set for a review

manifests/12_prometheus_rule.yaml

awgreene · 2019-10-30T18:47:46Z

/retest

awgreene · 2019-10-30T19:51:22Z

/retest

awgreene · 2019-10-30T21:01:02Z

/retest

awgreene · 2019-10-31T10:29:11Z

/retest

awgreene · 2019-10-31T11:53:14Z

/retest

lilic

This file should be removed then? Or at least the content of it? https://github.com/operator-framework/operator-marketplace/blob/c72dc6c093fa8592b83a74ab2839d65fa59f58a3/deploy/prometheus_rule.yaml

Nice improvements!

manifests/12_prometheus_rule.yaml

awgreene · 2019-10-31T12:25:07Z

This file should be removed then? Or at least the content of it? https://github.com/operator-framework/operator-marketplace/blob/c72dc6c093fa8592b83a74ab2839d65fa59f58a3/deploy/prometheus_rule.yaml

Good point @lilic, I have updated the contents of the file.

brancz

One last thing, then lgtm.

brancz · 2019-10-31T12:39:31Z

deploy/prometheus_rule.yaml

+        - alert: CommunityOperatorConnectionErrors
+          annotations:
+            message: Unable to connect to the default OperatorSource community-operators AppRegistry for {{ $value }}% of requests.
+          expr: increase(app_registry_request_total{code!~"2..",opsrc="community-operators"}[5m]) / increase(app_registry_request_total{opsrc="community-operators"}[5m]) > 0.5


Increase is a rather unusual use for this, rate is typically used for the purpose that we use it here for.

@brancz Addressed in the latest version of this PR, but would you mind sharing the reasoning?

awgreene · 2019-11-04T13:15:32Z

/retest

openshift-ci-robot · 2019-11-04T14:48:00Z

@awgreene: This pull request references Bugzilla bug 1768483, which is invalid:

expected the bug to target the "4.3.0" release, but it targets "---" instead

Comment /bugzilla refresh to re-evaluate validity if changes to the Bugzilla bug are made, or edit the title of this pull request to link to a different bug.

In response to this:

Bug 1768483: Limit Cardinality of Marketplace quay metrics

Instructions for interacting with me using PR comments are available here. If you have questions or suggestions related to my behavior, please file an issue against the kubernetes/test-infra repository.

awgreene · 2019-11-04T14:48:08Z

/bugzilla refresh

openshift-ci-robot · 2019-11-04T14:48:11Z

@awgreene: This pull request references Bugzilla bug 1768483, which is invalid:

expected the bug to target the "4.3.0" release, but it targets "---" instead

Comment /bugzilla refresh to re-evaluate validity if changes to the Bugzilla bug are made, or edit the title of this pull request to link to a different bug.

In response to this:

/bugzilla refresh

Instructions for interacting with me using PR comments are available here. If you have questions or suggestions related to my behavior, please file an issue against the kubernetes/test-infra repository.

awgreene · 2019-11-04T14:49:23Z

/bugzilla refresh

openshift-ci-robot · 2019-11-04T14:49:28Z

@awgreene: This pull request references Bugzilla bug 1768483, which is valid. The bug has been moved to the POST state. The bug has been updated to refer to the pull request using the external bug tracker.

In response to this:

/bugzilla refresh

Instructions for interacting with me using PR comments are available here. If you have questions or suggestions related to my behavior, please file an issue against the kubernetes/test-infra repository.

awgreene · 2019-11-04T14:50:39Z

/retest

awgreene · 2019-11-04T16:36:02Z

/retest

awgreene · 2019-11-04T18:54:03Z

/retest

awgreene · 2019-11-04T22:01:27Z

/retest

awgreene · 2019-11-04T23:33:38Z

/retest

lilic · 2019-11-05T10:04:26Z

/retest

Think the upstream failure should be fixed already.

awgreene · 2019-11-05T11:23:07Z

/retest

anik120 · 2019-11-05T14:10:23Z

/lgtm

openshift-ci-robot · 2019-11-05T14:31:27Z

[APPROVALNOTIFIER] This PR is APPROVED

This pull-request has been approved by: awgreene, gallettilance, kevinrizza

The full list of commands accepted by this bot can be found here.

The pull request process is described here

Needs approval from an approver in each of these files:

~~OWNERS~~ [kevinrizza]

Approvers can indicate their approval by writing /approve in a comment
Approvers can cancel approval by writing /approve cancel in a comment

awgreene · 2019-11-05T16:46:49Z

/retest

level=info msg="Cluster operator ingress Available is False with IngressUnavailable: Not all ingress controllers are available."```

awgreene · 2019-11-05T18:13:30Z

/retest

1105 17:32:03.615061     825 factory_object_mapping.go:423] Failed to download OpenAPI (the server could not find the requested resource), falling back to swagger
6) Interacting with an `AllNamespaces` install mode Operator (Jaeger)
   ✖ displays subscription creation form for selected Operator (1 failure)
A Jasmine spec timed out. Resetting the WebDriver Control Flow.
   • selects all namespaces for Operator subscription
   • displays Operator as subscribed in OperatorHub
   • displays Operator in "Cluster Service Versions" view for "test-cttoo" namespace
   • creates Operator `Deployment`
   • recreates Operator `Deployment` if manually deleted

awgreene · 2019-11-05T18:15:12Z

/bugzilla refresh

openshift-ci-robot · 2019-11-05T18:15:15Z

@awgreene: This pull request references Bugzilla bug 1768483, which is valid.

In response to this:

/bugzilla refresh

Instructions for interacting with me using PR comments are available here. If you have questions or suggestions related to my behavior, please file an issue against the kubernetes/test-infra repository.

awgreene · 2019-11-05T18:19:41Z

/test e2e-aws-console-olm

openshift-ci-robot · 2019-11-05T19:16:23Z

@awgreene: All pull requests linked via external trackers have merged. Bugzilla bug 1768483 has been moved to the MODIFIED state.

In response to this:

Bug 1768483: Limit Cardinality of Marketplace quay metrics

Instructions for interacting with me using PR comments are available here. If you have questions or suggestions related to my behavior, please file an issue against the kubernetes/test-infra repository.

openshift-ci-robot added the size/M Denotes a PR that changes 30-99 lines, ignoring generated files. label Oct 28, 2019

openshift-ci-robot requested review from aravindhp and ecordell October 28, 2019 16:52

openshift-ci-robot assigned kevinrizza Oct 28, 2019

openshift-ci-robot added the lgtm Indicates that a PR is ready to be merged. label Oct 28, 2019

openshift-ci-robot added the do-not-merge/hold Indicates that a PR should not merge because someone has issued a /hold command. label Oct 28, 2019

brancz suggested changes Oct 29, 2019

View reviewed changes

manifests/12_prometheus_rule.yaml Outdated Show resolved Hide resolved

awgreene force-pushed the marketplace-prom-rules branch from 393319b to 9212643 Compare October 29, 2019 15:46

openshift-ci-robot removed the lgtm Indicates that a PR is ready to be merged. label Oct 29, 2019

awgreene force-pushed the marketplace-prom-rules branch from 9212643 to f0e58fd Compare October 29, 2019 16:41

brancz reviewed Oct 30, 2019

View reviewed changes

manifests/12_prometheus_rule.yaml Outdated Show resolved Hide resolved

manifests/12_prometheus_rule.yaml Outdated Show resolved Hide resolved

awgreene force-pushed the marketplace-prom-rules branch from f0e58fd to d0d2d58 Compare October 30, 2019 14:42

brancz reviewed Oct 30, 2019

View reviewed changes

manifests/12_prometheus_rule.yaml Outdated Show resolved Hide resolved

awgreene force-pushed the marketplace-prom-rules branch from d0d2d58 to c72dc6c Compare October 30, 2019 17:36

awgreene mentioned this pull request Oct 31, 2019

Alerting rules incomplete #264

Closed

lilic reviewed Oct 31, 2019

View reviewed changes

manifests/12_prometheus_rule.yaml Outdated Show resolved Hide resolved

manifests/12_prometheus_rule.yaml Outdated Show resolved Hide resolved

manifests/12_prometheus_rule.yaml Outdated Show resolved Hide resolved

awgreene force-pushed the marketplace-prom-rules branch from c72dc6c to 2a8ab23 Compare October 31, 2019 12:22

openshift-ci-robot added size/L Denotes a PR that changes 100-499 lines, ignoring generated files. and removed size/M Denotes a PR that changes 30-99 lines, ignoring generated files. labels Oct 31, 2019

brancz reviewed Oct 31, 2019

View reviewed changes

awgreene force-pushed the marketplace-prom-rules branch from 2a8ab23 to 4ce0777 Compare October 31, 2019 12:44

awgreene force-pushed the marketplace-prom-rules branch from 4ce0777 to 23a03fa Compare November 4, 2019 11:18

awgreene changed the title ~~[metrics] Update metrics reported by marketplace~~ Bug 1768483: Limit Cardinality of Marketplace quay metrics Nov 4, 2019

openshift-ci-robot added the bugzilla/invalid-bug Indicates that a referenced Bugzilla bug is invalid for the branch this PR is targeting. label Nov 4, 2019

openshift-ci-robot added the bugzilla/valid-bug Indicates that a referenced Bugzilla bug is valid for the branch this PR is targeting. label Nov 4, 2019

openshift-ci-robot removed the bugzilla/invalid-bug Indicates that a referenced Bugzilla bug is invalid for the branch this PR is targeting. label Nov 4, 2019

openshift-ci-robot assigned anik120 Nov 5, 2019

openshift-ci-robot added the lgtm Indicates that a PR is ready to be merged. label Nov 5, 2019

gallettilance approved these changes Nov 5, 2019

View reviewed changes

openshift-merge-robot merged commit 530c85d into operator-framework:master Nov 5, 2019

Bug 1768483: Limit Cardinality of Marketplace quay metrics #259

Bug 1768483: Limit Cardinality of Marketplace quay metrics #259

awgreene commented Oct 28, 2019 •

edited

kevinrizza commented Oct 28, 2019

awgreene commented Oct 28, 2019

awgreene commented Oct 30, 2019

awgreene commented Oct 30, 2019

awgreene commented Oct 30, 2019

awgreene commented Oct 30, 2019

awgreene commented Oct 31, 2019

awgreene commented Oct 31, 2019

lilic left a comment

awgreene commented Oct 31, 2019

brancz left a comment

brancz Oct 31, 2019

awgreene Oct 31, 2019

awgreene commented Nov 4, 2019

openshift-ci-robot commented Nov 4, 2019

awgreene commented Nov 4, 2019

openshift-ci-robot commented Nov 4, 2019

awgreene commented Nov 4, 2019

openshift-ci-robot commented Nov 4, 2019

awgreene commented Nov 4, 2019

awgreene commented Nov 4, 2019

awgreene commented Nov 4, 2019

awgreene commented Nov 4, 2019

awgreene commented Nov 4, 2019

lilic commented Nov 5, 2019

awgreene commented Nov 5, 2019

anik120 commented Nov 5, 2019

openshift-ci-robot commented Nov 5, 2019

awgreene commented Nov 5, 2019

awgreene commented Nov 5, 2019

awgreene commented Nov 5, 2019

openshift-ci-robot commented Nov 5, 2019

awgreene commented Nov 5, 2019

openshift-ci-robot commented Nov 5, 2019

Bug 1768483: Limit Cardinality of Marketplace quay metrics #259

Bug 1768483: Limit Cardinality of Marketplace quay metrics #259

Conversation

awgreene commented Oct 28, 2019 • edited

kevinrizza commented Oct 28, 2019

awgreene commented Oct 28, 2019

awgreene commented Oct 30, 2019

awgreene commented Oct 30, 2019

awgreene commented Oct 30, 2019

awgreene commented Oct 30, 2019

awgreene commented Oct 31, 2019

awgreene commented Oct 31, 2019

lilic left a comment

Choose a reason for hiding this comment

awgreene commented Oct 31, 2019

brancz left a comment

Choose a reason for hiding this comment

brancz Oct 31, 2019

Choose a reason for hiding this comment

awgreene Oct 31, 2019

Choose a reason for hiding this comment

awgreene commented Nov 4, 2019

openshift-ci-robot commented Nov 4, 2019

awgreene commented Nov 4, 2019

openshift-ci-robot commented Nov 4, 2019

awgreene commented Nov 4, 2019

openshift-ci-robot commented Nov 4, 2019

awgreene commented Nov 4, 2019

awgreene commented Nov 4, 2019

awgreene commented Nov 4, 2019

awgreene commented Nov 4, 2019

awgreene commented Nov 4, 2019

lilic commented Nov 5, 2019

awgreene commented Nov 5, 2019

anik120 commented Nov 5, 2019

openshift-ci-robot commented Nov 5, 2019

awgreene commented Nov 5, 2019

awgreene commented Nov 5, 2019

awgreene commented Nov 5, 2019

openshift-ci-robot commented Nov 5, 2019

awgreene commented Nov 5, 2019

openshift-ci-robot commented Nov 5, 2019

awgreene commented Oct 28, 2019 •

edited