Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

How to expose ingress path of K8s as a target in prometheus #4218

Closed
bmv126 opened this Issue Jun 5, 2018 · 2 comments

Comments

Projects
None yet
2 participants
@bmv126
Copy link

bmv126 commented Jun 5, 2018

Bug Report

What did you do?
I have a K8s cluster (1.9 version) on which, I have a service running. I have enabled the jmx and have used the jmx exported to get the metrics in prometheus format.
For the user to access, I have created an ingress path for the jmx metrics, below is the k8s ingress resourse:

apiVersion: extensions/v1beta1
kind: Ingress
metadata:
name: jmx-ingress
annotations:
nginx.ingress.kubernetes.io/rewrite-target: /
kubernetes.io/ingress.class: nginx
spec:
rules:

  • http:
    paths:
    • path: /MyMetrics/
      backend:
      serviceName: service-jmx
      servicePort: 49115

Now using https://IP ADDRESS:PORT/MyMetrics/ , I am able to see the metrics.

But If set the target in prometheus.yml as below:
scrape_configs:
- job_name: prometheus
static_configs:
- targets:
- 10.194.11.35:80/MyMetrics/

I get below error:
Error loading config" err="couldn't load configuration (--config.file=/etc/config/prometheus.yml): "10.194.11.35:80/MyMetrics/" is not a valid hostname"

What did you expect to see?
I expect to see the metrics getting scraped on the prometheus server.

What did you see instead? Under which circumstances?
Prometheus server pod does not come up as it failing with the error.
level=error ts=2018-06-05T09:18:29.782548012Z caller=main.go:356 msg="Error loading config" err="couldn't load configuration (--config.file=/etc/config/prometheus.yml): "10.194.11.35:80/MyMetrics/" is not a valid hostname"

Environment
Kubernetes:
Server Version: version.Info{Major:"1", Minor:"9", GitVersion:"v1.9.6", GitCommit:"9f8ebd171479bec0ada837d7ee641dec2f8c6dd1", GitTreeState:"clean", BuildDate:"2018-03-21T15:13:31Z", GoVersion:"go1.9.3", Compiler:"gc", Platform:"linux/amd64"}

  • System information:
    Linux 3.10.0-693.17.1.el7.x86_64 x86_64

  • Prometheus version:
    2.0

@krasi-georgiev

This comment has been minimized.

Copy link
Member

krasi-georgiev commented Jun 5, 2018

This doesn't look like a bug so would need to ask in the users group 👍
https://groups.google.com/forum/#!forum/prometheus-users
The github issues are focused on bug reports 🐞

I would recommend looking through the official docs and examples as well as searching in the google groups as there is a good chance that you will find the answer immediately.
https://prometheus.io/docs/introduction/overview/

Feel free to reopen if you are 100% convinced that is a bug in Prometheus.

@lock

This comment has been minimized.

Copy link

lock bot commented Mar 22, 2019

This thread has been automatically locked since there has not been any recent activity after it was closed. Please open a new issue for related bugs.

@lock lock bot locked and limited conversation to collaborators Mar 22, 2019

Sign up for free to subscribe to this conversation on GitHub. Already have an account? Sign in.
You can’t perform that action at this time.