-
Notifications
You must be signed in to change notification settings - Fork 3.7k
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
serviceMonitorNamespaceSelector and serviceMonitorSelector don't work #1331
Comments
Which version of the Prometheus operator are you using? The |
@brancz Do'h! I forgot to mention it, my bad, I'm using |
@jescarri Thanks a lot for the detailed bug report. Your Prometheus CRD has a wrong serviceMonitorNamespaceSelector:
matchNames:
- foo
- bar
I would suggest adding a label to each namespace that you want to scan for ServiceMonitors (e.g. Then in your Prometheus CRD you would write: serviceMonitorNamespaceSelector:
matchLabels:
prometheus: front-end-team So now you might be asking yourself: "Why did this work with jesus1 and not jesus2?" Kubernetes ignores unknown fields. In this case it ignored the Let me know if this helps. |
Ha!, that made the difference, I ended up using a matchExpression. Thanks for the help!. |
@mxinden Have a question. I am not sure we are able to attach labels to namespaces created by Helm. Currently we have all our stack in helm. Can you enlighten me if |
@faheem-cliqz as far as I know you can only select labels via the LabelSelector, which sounds like a clean design to me. Maybe @gianrubio can give you more advice on how to add namespace labels via helm. @faheem-cliqz This sounds like a new question to me. In the future, please create a new Github issue for it. |
Found a lingering feature request in helm for this. Referred it. Moving on will create a new issue. May be @gianrubio knows of any hacks for now to get this. |
if anybody is curious, this is the code that causes this behavior (pinned @ ref a36a34f) https://github.com/coreos/prometheus-operator/blob/a36a34f/pkg/prometheus/operator.go#L1335-L1348 |
You can add labels to the namespaces and then reference those labels. If you want to use the namespace names, Kubernetes makes a label foreach namespace with its own name, matchExpressions:
- key: kubernetes.io/metadata.name
operator: In
values:
- monitoring
- prometheus
- default |
not working for default labels |
@trainingUI what Kubernetes version are you using? For older versions of k8s they don't have for ns in monitoring prometheus default; do
kubectl patch ns/$ns --patch "{ \"metadata\": { \"labels\": { \"name\": \"$ns\" } } }"
done which you can reference with: serviceMonitorNamespaceSelector:
matchExpressions:
- key: name
operator: In
values:
- monitoring
- prometheus
- default Make sure you include the service monitor labels also: serviceMonitorSelector:
matchExpressions:
- key: k8s-app
operator: In
values:
- registry |
It worked, Thanks! |
What did you do?
Deployed a prometheus with serviceMonitorNamespaceSelector and serviceMonitorSelector
What did you expect to see?
Scrape jobs for the matching namespace / serviceMonitor
What did you see instead? Under which circumstances?
Only one scrape job.
Environment
Kubernetes version information:
insert output of
kubectl version
hereServer Version: version.Info{Major:"1", Minor:"8+", GitVersion:"v1.8.9+coreos.1", GitCommit:"cd373fe93e046b0a0bc7e4045af1bf4171cea395", GitTreeState:"clean", BuildDate:"2018-03-13T21:28:21Z", GoVersion:"go1.8.3", Compiler:"gc", Platform:"linux/amd64"}
Kubernetes cluster kind:
Tectonic instaler
Manifests:
No errors.
Service monitors exist as follow:
Namespace foo:
Namespace bar:
Bot service monitors match healthy services.
However in prometheus jesus1 it only creates a matching scrape config for service monitor jesus1
The text was updated successfully, but these errors were encountered: