Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

No metrics gathered for rollbacks #5017

Closed
rombert opened this issue Dec 5, 2018 · 5 comments
Closed

No metrics gathered for rollbacks #5017

rombert opened this issue Dec 5, 2018 · 5 comments
Labels
bug Categorizes issue or PR as related to a bug. unconfirmed

Comments

@rombert
Copy link

rombert commented Dec 5, 2018

Output of helm version:

Client: &version.Version{SemVer:"v2.8.2", GitCommit:"2.8.2", GitTreeState:"clean"}
Server: &version.Version{SemVer:"v2.11.0", GitCommit:"2e55dbe1fdb5fdb96b75ff144a339489417b146b", GitTreeState:"clean"}

Output of kubectl version:

Client Version: version.Info{Major:"1", Minor:"12", GitVersion:"v1.12.0", GitCommit:"0ed33881dc4355495f623c6f22e7dd0b7632b7c0", GitTreeState:"archive", BuildDate:"2018-09-28T12:00:00Z", GoVersion:"go1.11.1", Compiler:"gc", Platform:"linux/amd64"}
Server Version: version.Info{Major:"1", Minor:"12", GitVersion:"v1.12.1", GitCommit:"4ed3216f3ec431b140b1d899130a69fc671678f4", GitTreeState:"clean", BuildDate:"2018-10-05T16:36:14Z", GoVersion:"go1.10.4", Compiler:"gc", Platform:"linux/amd64"}

Cloud Provider/Platform (AKS, GKE, Minikube etc.): Internally managed K8s ( but I don't manage it myself ).


I am using prometheus to gather metrics about releases. One good entry point for that is the /metrics endpoint exposed by Tiller. However, I am missing metrics for the RollbackRelease method, as it's not present in the output. I took a quick loot at the code from #2171 but I don't see why the metrics are not present.

@bacongobbler bacongobbler added bug Categorizes issue or PR as related to a bug. unconfirmed labels Dec 6, 2018
@rombert
Copy link
Author

rombert commented Dec 14, 2018

@bacongobbler - I see that you did a first triage of this issue. Is there anything I can do to at least help confirm it? It might be that "I'm holding it wrong", one can never know :-)

@bacongobbler
Copy link
Member

If you could provide your setup instructions that may be helpful :)

@rombert
Copy link
Author

rombert commented Dec 14, 2018

@bacongobbler - I did not set up my environment myself, but what I know is that:

  • we have a k10s cluster
  • we use helm for deployments
  • we use prometheus for gathering metrics
  • prometheus is successfully scraping metrics from tiller
  • tiller does not have anny metrics for the RollbackRelease method

To access the metrics entry point I did the following

$ kubectl port-forward tiller-deploy-679c587747-cx4vg 44135
$curl -s http://localhost:44135/metrics | grep Rollback

If I try to access other metrics, e.g. for releases they appear just fine

$ curl -s http://localhost:44135/metrics | grep InstallRelease
grpc_server_handled_total{grpc_code="OK",grpc_method="InstallRelease",grpc_service="hapi.services.tiller.ReleaseService",grpc_type="unary"} 156
grpc_server_handled_total{grpc_code="Unknown",grpc_method="InstallRelease",grpc_service="hapi.services.tiller.ReleaseService",grpc_type="unary"} 61
grpc_server_msg_received_total{grpc_method="InstallRelease",grpc_service="hapi.services.tiller.ReleaseService",grpc_type="unary"} 217
grpc_server_msg_sent_total{grpc_method="InstallRelease",grpc_service="hapi.services.tiller.ReleaseService",grpc_type="unary"} 156
grpc_server_started_total{grpc_method="InstallRelease",grpc_service="hapi.services.tiller.ReleaseService",grpc_type="unary"} 217

@bacongobbler - does that at least point you in the right direction?

@bacongobbler
Copy link
Member

Yes, thank you.

@bacongobbler
Copy link
Member

closing as inactive.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
bug Categorizes issue or PR as related to a bug. unconfirmed
Projects
None yet
Development

No branches or pull requests

2 participants