Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Bug 2002281: 4.8: pkg/resourceprovider: guard from negative metrics #55

Merged
merged 2 commits into from Sep 10, 2021

Conversation

dgrisonnet
Copy link
Member

Cherry-pick of 0b3ac78 on release-4.8.

When serving the resource metrics API, prometheus-adapter may return
negative values for pods/nodes memory and CPU usage. This happens
because Prometheus sees counter resets which results in Prometheus
interpolating data incorrectly to avoid the counter value going down.
To prevent that, we need to add some guards in prometheus-adapter to
replace the negative value by zero whenever it detects one.

Signed-off-by: Damien Grisonnet <dgrisonn@redhat.com>
@openshift-ci openshift-ci bot added the bugzilla/severity-medium Referenced Bugzilla bug's severity is medium for the branch this PR is targeting. label Sep 8, 2021
@openshift-ci
Copy link

openshift-ci bot commented Sep 8, 2021

@dgrisonnet: This pull request references Bugzilla bug 2002281, which is valid. The bug has been moved to the POST state. The bug has been updated to refer to the pull request using the external bug tracker.

6 validation(s) were run on this bug
  • bug is open, matching expected state (open)
  • bug target release (4.8.z) matches configured target release for branch (4.8.z)
  • bug is in the state ASSIGNED, which is one of the valid states (NEW, ASSIGNED, ON_DEV, POST, POST)
  • dependent bug Bugzilla bug 1956830 is in the state VERIFIED, which is one of the valid states (VERIFIED, RELEASE_PENDING, CLOSED (ERRATA), CLOSED (CURRENTRELEASE))
  • dependent Bugzilla bug 1956830 targets the "4.9.0" release, which is one of the valid target releases: 4.9.0
  • bug has dependents

Requesting review from QA contact:
/cc @juzhao

In response to this:

Bug 2002281: 4.8: pkg/resourceprovider: guard from negative metrics

Instructions for interacting with me using PR comments are available here. If you have questions or suggestions related to my behavior, please file an issue against the kubernetes/test-infra repository.

@openshift-ci openshift-ci bot added the bugzilla/valid-bug Indicates that a referenced Bugzilla bug is valid for the branch this PR is targeting. label Sep 8, 2021
@openshift-ci openshift-ci bot added the approved Indicates a PR has been approved by an approver from all required OWNERS files. label Sep 8, 2021
@dgrisonnet
Copy link
Member Author

/assign @simonpasquier

Signed-off-by: Damien Grisonnet <dgrisonn@redhat.com>
@simonpasquier
Copy link

/lgtm

@openshift-ci openshift-ci bot added the lgtm Indicates that a PR is ready to be merged. label Sep 8, 2021
@juzhao
Copy link

juzhao commented Sep 9, 2021

/bugzilla cc-qa

@openshift-ci
Copy link

openshift-ci bot commented Sep 9, 2021

@juzhao: An error was encountered querying GitHub for users with public email (juzhao@redhat.com) for bug 2002281 on the Bugzilla server at https://bugzilla.redhat.com. No known errors were detected, please see the full error message for details.

Full error message. Post "http://ghproxy/graphql": dial tcp 172.30.229.2:80: i/o timeout

Please contact an administrator to resolve this issue, then request a bug refresh with /bugzilla refresh.

In response to this:

/bugzilla cc-qa

Instructions for interacting with me using PR comments are available here. If you have questions or suggestions related to my behavior, please file an issue against the kubernetes/test-infra repository.

@juzhao
Copy link

juzhao commented Sep 9, 2021

tested with the fix, watched for a while, did not see negative value for oc adm top nodes
/label qe-approved

@juzhao
Copy link

juzhao commented Sep 9, 2021

/lgtm

@openshift-ci openshift-ci bot added the qe-approved Signifies that QE has signed off on this PR label Sep 9, 2021
@openshift-ci
Copy link

openshift-ci bot commented Sep 9, 2021

[APPROVALNOTIFIER] This PR is APPROVED

This pull-request has been approved by: dgrisonnet, juzhao, simonpasquier

The full list of commands accepted by this bot can be found here.

The pull request process is described here

Needs approval from an approver in each of these files:
  • OWNERS [dgrisonnet,simonpasquier]

Approvers can indicate their approval by writing /approve in a comment
Approvers can cancel approval by writing /approve cancel in a comment

@mfojtik mfojtik added the cherry-pick-approved Indicates a cherry-pick PR into a release branch has been approved by the release branch manager. label Sep 10, 2021
@mfojtik
Copy link
Member

mfojtik commented Sep 10, 2021

[patch-manager] 🚀 Approved for z-stream by score: 0.20

adding cherry-pick approved

@openshift-merge-robot openshift-merge-robot merged commit bfffea3 into openshift:release-4.8 Sep 10, 2021
@openshift-ci
Copy link

openshift-ci bot commented Sep 10, 2021

@dgrisonnet: All pull requests linked via external trackers have merged:

Bugzilla bug 2002281 has been moved to the MODIFIED state.

In response to this:

Bug 2002281: 4.8: pkg/resourceprovider: guard from negative metrics

Instructions for interacting with me using PR comments are available here. If you have questions or suggestions related to my behavior, please file an issue against the kubernetes/test-infra repository.

@openshift-bot
Copy link

[ART PR BUILD NOTIFIER]

This PR has been included in build ose-prometheus-adapter-container-v4.8.0-202311261141.p0.gbfffea3.assembly.stream for distgit ose-prometheus-adapter.
All builds following this will include this PR.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
approved Indicates a PR has been approved by an approver from all required OWNERS files. bugzilla/severity-medium Referenced Bugzilla bug's severity is medium for the branch this PR is targeting. bugzilla/valid-bug Indicates that a referenced Bugzilla bug is valid for the branch this PR is targeting. cherry-pick-approved Indicates a cherry-pick PR into a release branch has been approved by the release branch manager. lgtm Indicates that a PR is ready to be merged. qe-approved Signifies that QE has signed off on this PR
Projects
None yet
Development

Successfully merging this pull request may close these issues.

None yet

6 participants