Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Update docs for service monitor integration #213

Merged
merged 9 commits into from
Jun 19, 2017
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
2 changes: 1 addition & 1 deletion api/monitor.go
Original file line number Diff line number Diff line change
Expand Up @@ -3,7 +3,7 @@ package api
import "fmt"

const (
AgentCoreosPrometheus = "COREOS_PROMETHEUS"
AgentCoreosPrometheus = "coreos-prometheus-operator"

MonitoringAgent = EngressKey + "/monitoring-agent" // COREOS_PROMETHEUS
PrometheusExporterPort = EngressKey + "/prometheus-exporter-port" // Kube NS where service monitors will be created
Expand Down
20 changes: 10 additions & 10 deletions docs/user-guide/ingress/README.md
Original file line number Diff line number Diff line change
Expand Up @@ -121,17 +121,17 @@ Below is the full list of supported annotation keys:

| Keys | Value | Default | Description |
|--------|-----------|----------|--------------|
| ingress.appscode.com/type | LoadBalancer, HostPort, NodePort | LoadBalancer | Indicates type of service used to expose HAProxy to the internet |
| ingress.appscode.com/replicas | integer | 1 | Indicates number of replicas of HAProxy pods |
| ingress.appscode.com/load-balaner-ip | string | x | For "gce" and "gke" cloud provider, if this value is set to a valid IPv4 address, it will be assigned to Google cloud network loadbalancer used to expose HAProxy. Usually this is set to a static IP to preserve DNS configuration |
| ingress.appscode.com/type | LoadBalancer, HostPort, NodePort | LoadBalancer | `Required`. Indicates type of service used to expose HAProxy to the internet |
| ingress.appscode.com/replicas | integer | 1 | `Optional`. Indicates number of replicas of HAProxy pods |
| ingress.appscode.com/load-balaner-ip | string | x | `Optional`. For "gce" and "gke" cloud provider, if this value is set to a valid IPv4 address, it will be assigned to Google cloud network loadbalancer used to expose HAProxy. Usually this is set to a static IP to preserve DNS configuration |
| ingress.appscode.com/node-selector | map | x | Indicates which hosts are selected to run HAProxy pods. This is a required annotation for `HostPort` type ingress. |
| ingress.appscode.com/sticky-session | bool | false | Indicates the session affinity for the traffic. If set, session affinity will apply to all the rulses. |
| ingress.appscode.com/annotations-service | map | x | Annotaiotns applied to service used to expose HAProxy |
| ingress.appscode.com/annotations-pod | map | x | Annotations applied to pods used to run HAProxy |
| ingress.appscode.com/keep-source-ip | bool | false | If set, preserves source IP for `LoadBalancer` type ingresses. The actual configuration generated depends on the underlying cloud provider. For gce, gke, azure: Adds annotation `service.beta.kubernetes.io/external-traffic: OnlyLocal` to services used to expose HAProxy. For aws, enforces the use of the PROXY protocol. |
| ingress.appscode.com/stats | bool | false | If set, HAProxy stats will be exposed |
| ingress.appscode.com/stats-port | integer | 1936 | Port used to expose HAProxy stats |
| ingress.appscode.com/stats-secret-name | string | x | Secret used to provide username & password to secure HAProxy stats endpoint. Secret must contain keys `username` and `password` |
| ingress.appscode.com/sticky-session | bool | false | `Optional`. Indicates the session affinity for the traffic. If set, session affinity will apply to all the rulses. |
| ingress.appscode.com/annotations-service | map | x | `Optional`. Annotaiotns applied to service used to expose HAProxy |
| ingress.appscode.com/annotations-pod | map | x | `Optional`. Annotations applied to pods used to run HAProxy |
| ingress.appscode.com/keep-source-ip | bool | false | `Optional`. If set, preserves source IP for `LoadBalancer` type ingresses. The actual configuration generated depends on the underlying cloud provider. For gce, gke, azure: Adds annotation `service.beta.kubernetes.io/external-traffic: OnlyLocal` to services used to expose HAProxy. For aws, enforces the use of the PROXY protocol. |
| ingress.appscode.com/stats | bool | false | `Optional`. If set, HAProxy stats will be exposed |
| ingress.appscode.com/stats-port | integer | 1936 | `Optional`. Port used to expose HAProxy stats |
| ingress.appscode.com/stats-secret-name | string | x | `Optional`. Secret used to provide username & password to secure HAProxy stats endpoint. Secret must contain keys `username` and `password` |
| ingress.appscode.com/ip | | | Removed since 1.5.6. Use `ingress.appscode.com/load-balaner-ip` |
| ingress.appscode.com/persist | | | Removed since 1.5.6. |
| ingress.appscode.com/daemon.nodeSelector | | | Removed since 1.5.6. Use `ingress.appscode.com/node-selector` |
Expand Down
8 changes: 4 additions & 4 deletions docs/user-guide/ingress/external-svc.md
Original file line number Diff line number Diff line change
Expand Up @@ -9,10 +9,10 @@ dns, [DNS resolvers](https://cbonte.github.io/haproxy-dconv/1.7/configuration.ht
| Keys | Value | Default | Description |
|--------|-----------|----------|-------------|
| ingress.appscode.com/use-dns-resolver | bool | false for L7, always true for L4 | If set, DNS resolution will be used |
| ingress.appscode.com/dns-resolver-nameservers | array | | If set to an array of DNS nameservers, these will be used HAProxy to periodically resolve DNS. If not set, HAProxy parses the server line definition and matches a host name at start up. |
| ingress.appscode.com/dns-resolver-retries | integer | | If set, this defines the number of queries to send to resolve a server name before giving up. If not set, default value pre-configured by HAProxy is used. |
| ingress.appscode.com/dns-resolver-timeout | map | | If set, defines timeouts related to name resolution. Define value as '{ "event": "time" }'. For a list of valid events, please consult [HAProxy documentation](https://cbonte.github.io/haproxy-dconv/1.7/configuration.html#5.3.2-timeout). |
| ingress.appscode.com/dns-resolver-hold | map | | If set, Defines period during which the last name resolution should be kept based on last resolution status. Define value as '{ "status": "period" }'. For a list of valid status, please consult [HAProxy documentation](https://cbonte.github.io/haproxy-dconv/1.7/configuration.html#5.3.2-hold). |
| ingress.appscode.com/dns-resolver-nameservers | array | | `Optional`. If set to an array of DNS nameservers, these will be used HAProxy to periodically resolve DNS. If not set, HAProxy parses the server line definition and matches a host name at start up. |
| ingress.appscode.com/dns-resolver-retries | integer | | `Optional`. If set, this defines the number of queries to send to resolve a server name before giving up. If not set, default value pre-configured by HAProxy is used. |
| ingress.appscode.com/dns-resolver-timeout | map | | `Optional`. If set, defines timeouts related to name resolution. Define value as '{ "event": "time" }'. For a list of valid events, please consult [HAProxy documentation](https://cbonte.github.io/haproxy-dconv/1.7/configuration.html#5.3.2-timeout). |
| ingress.appscode.com/dns-resolver-hold | map | | `Optional`. If set, Defines period during which the last name resolution should be kept based on last resolution status. Define value as '{ "status": "period" }'. For a list of valid status, please consult [HAProxy documentation](https://cbonte.github.io/haproxy-dconv/1.7/configuration.html#5.3.2-hold). |

Following example illustrates the scenario.

Expand Down
29 changes: 22 additions & 7 deletions docs/user-guide/ingress/stats-and-metrics.md
Original file line number Diff line number Diff line change
Expand Up @@ -4,20 +4,35 @@ To expose HAProxy stats, please use the following annotations:
### Stats annotations
| Keys | Value | Default | Description |
|--------|-----------|----------|-------------|
| ingress.appscode.com/stats | bool | false | If set, HAProxy stats will be exposed |
| ingress.appscode.com/stats-port | integer | 1936 | Port used to expose HAProxy stats |
| ingress.appscode.com/stats-secret-name | string | x | Secret used to provide username & password to secure HAProxy stats endpoint. Secret must contain keys `username` and `password` |
| ingress.appscode.com/stats-service-name | string | `<ingress-name>-stats` | ClusterIP type service used to expose HAproxy stats. This allows to avoid exposing stats to internet. |
| ingress.appscode.com/stats | bool | false | `Optional`. If set, HAProxy stats will be exposed |
| ingress.appscode.com/stats-port | integer | 1936 | `Optional`. Port used to expose HAProxy stats |
| ingress.appscode.com/stats-secret-name | string | x | `Optional`. Secret used to provide username & password to secure HAProxy stats endpoint. Secret must contain keys `username` and `password` |

Please note that a separate `ClusterIP` type service is used to expose stats. So, you can use expose unauthenticated stats endpoint without exposing them to the internet.
Please note that stats port is not exposed to the internet via the service running in front of HAProxy pods.

## Using Prometheus
Voyager operator exposes Prometheus ready metrics via the following endpoints on port `:8080`:

- `/metrics`: Scrape this to monitor operator.
- `/extensions/v1beta1/namespaces/:ns/ingresses/:name/pods/:ip/metrics` : Scrape this endpoint to monitor HAProxy running for a Kubernetes ingress
- `/voyager.appscode.com/v1beta1/namespaces/:ns/ingresses/:name/pods/:ip/metrics`: Scrape this endpoint to monitor HAProxy running for an AppsCode extended ingress
- `/voyager.appscode.com/v1beta1/namespaces/:ns/ingresses/:name/metrics`: Scrape this endpoint to monitor HAProxy running for an AppsCode extended ingress

To change the port, use `--address` flag on Voyager opreator.

Currently [further discussion is on-going](https://github.com/appscode/voyager/issues/154) on how to integrate this with CoreOS Prometheus Operator.
## Using [CoreOS Prometheus Operator](https://coreos.com/operators/prometheus/docs/latest/)
Voyager operator can create [service monitors](https://coreos.com/operators/prometheus/docs/latest/user-guides/running-exporters.html#create-a-matching-servicemonitor) for HAProxy pods. If enabled, a side-car exporter pod is run with HAProxy to expose Prometheus ready metrics via the following endpoints on port `:56789`:

- `/extensions/v1beta1/namespaces/:ns/ingresses/:name/pods/:ip/metrics` : Scrape this endpoint to monitor HAProxy running for a Kubernetes ingress
- `/voyager.appscode.com/v1beta1/namespaces/:ns/ingresses/:name/metrics`: Scrape this endpoint to monitor HAProxy running for an AppsCode extended ingress

To enable this feature, please use the following annotations:

| Keys | Value | Default | Description |
|--------|-----------|----------|-------------|
| ingress.appscode.com/monitoring-agent | string | | `Required`. Indicates the monitoring agent used. Only valid value currently is 'coreos-prometheus-operator' |
| ingress.appscode.com/prometheus-exporter-port| integer | 56789 | `Optional`. Indicates the port used by exporter side-car to expose Prometheus metrics endpoint. If the default port 56789 is used to expose traffic, change it to an unused port. |
| ingress.appscode.com/service-monitor-labels | map | | `Required`. Indicates labels applied to service monitor. |
| ingress.appscode.com/service-monitor-namespace| string | | `Required`. Indicates namespace where service monitors are created. This must be the same namespace of the Prometheus instance. |
| ingress.appscode.com/service-monitor-endpoint-scrape-interval | string | | `Optional`. Indicates the srace interval for HAProxy exporter endpoint

__Known Limitations:__ If the HAProxy stats password is updated, exporter must be restarted to use the new credentials. This issue is tracked [here](https://github.com/appscode/voyager/issues/212).