You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
Please vote on this issue by adding a 馃憤 reaction to the original issue to help the community and maintainers prioritize this request. Searching for pre-existing feature requests helps us consolidate datapoints for identical requirements into a single place, thank you!
Please do not leave "+1" or other comments that do not add relevant new information or questions, they generate extra noise for issue followers and do not help prioritize the request.
If you are interested in working on this issue or have submitted a pull request, please leave a comment.
Overview of the Issue
I'm attempting to configure ingress-nginx as the k8s ingress for services which are part of a newly deployed consul-connect service mesh. I understand that there is an existing issue wherein consul won't support multiple services that have selectors for the same pod(s).
To try overcome that, i've used the consul.hashicorp.com/connect-ignore=true label as suggested by #592 , However, the two services I attempt to exclude still appear to be (attempting to) register to consul.
Reproduction Steps
On EKS v1.18
Install consul-connect with the following deploy/config.yml:
helm upgrade --install -f deploy/config.yaml consul hashicorp/consul --create-namespace -n consul --version "0.37.0"
Add consul.hashicorp.com/connect-ignore=true label to the ingress-nginx-consul-controller-admission and ingress-nginx-consul-controller-metrics services
kubectl get svc -n ingress-nginx-consul
NAME TYPE CLUSTER-IP EXTERNAL-IP PORT(S) AGE
ingress-nginx-consul-controller LoadBalancer 10.100.211.64 <redacted>.elb.us-east-1.amazonaws.com 80:30986/TCP,443:31505/TCP 20h
ingress-nginx-consul-controller-admission ClusterIP 10.100.245.8 <none> 443/TCP 20h
ingress-nginx-consul-controller-metrics ClusterIP 10.100.8.80 <none> 9913/TCP 20h
kubectl get pod -n ingress-nginx-consul
NAME READY STATUS RESTARTS AGE
ingress-nginx-consul-controller-7f5997bd74-29l4w 0/2 Init:CrashLoopBackOff 173 20h
ingress-nginx-consul-controller-7f5997bd74-bqlqh 0/2 Init:CrashLoopBackOff 173 20h
kubectl logs -f ingress-nginx-consul-controller-7f5997bd74-29l4w -n ingress-nginx-consul -c consul-connect-inject-init
2021-12-08T11:22:43.177Z [INFO] Consul login complete
2021-12-08T11:22:43.180Z [INFO] Unable to find registered services; retrying
<snip>
2021-12-08T11:22:52.201Z [INFO] Check to ensure a Kubernetes service has been created for this application. If your pod is not starting also check the connect-inject deployment logs.
<snip>
2021-12-08T11:24:43.452Z [INFO] Unable to find registered services; retrying
2021-12-08T11:24:43.452Z [ERROR] Timed out waiting for service registration: error="did not find correct number of services: 3"
Logs from consul connect injector suggest that it is not ignoring the two services ingress-nginx-consul-controller-admission and ingress-nginx-consul-controller-metrics:
kubectl logs -f consul-connect-injector-webhook-deployment-69744dbd8c-6pphl -n consul
2021-12-08T08:19:39.277Z INFO controller.endpoints registering service with Consul {"name": "ingress-nginx-consul-controller-metrics", "id": "ingress-nginx-consul-controller-7f5997bd74-bqlqh-ingress-nginx-consul-controller-metrics", "agentIP": "100.96.132.75"}
2021-12-08T08:19:39.354Z INFO controller.endpoints registering proxy service with Consul {"name": "ingress-nginx-consul-controller-metrics-sidecar-proxy"}
2021-12-08T08:19:39.374Z INFO controller.endpoints updating health check status for service {"name": "ingress-nginx-consul-controller-metrics", "reason": "Pod \"ingress-nginx-consul/ingress-nginx-consul-controller-7f5997bd74-bqlqh\" is not ready", "status": "critical"}
2021-12-08T08:19:39.381Z INFO controller.endpoints registering service with Consul {"name": "ingress-nginx-consul-controller-metrics", "id": "ingress-nginx-consul-controller-7f5997bd74-29l4w-ingress-nginx-consul-controller-metrics", "agentIP": "100.96.140.174"}
2021-12-08T08:19:39.386Z INFO controller.endpoints registering proxy service with Consul {"name": "ingress-nginx-consul-controller-metrics-sidecar-proxy"}
2021-12-08T08:19:39.410Z INFO controller.endpoints updating health check status for service {"name": "ingress-nginx-consul-controller-metrics", "reason": "Pod \"ingress-nginx-consul/ingress-nginx-consul-controller-7f5997bd74-29l4w\" is not ready", "status": "critical"}
2021-12-08T08:19:39.459Z INFO controller.endpoints retrieved {"name": "ingress-nginx-consul-controller-admission", "ns": "ingress-nginx-consul"}
2021-12-08T08:19:39.459Z INFO controller.endpoints registering service with Consul {"name": "ingress-nginx-consul-controller-admission", "id": "ingress-nginx-consul-controller-7f5997bd74-29l4w-ingress-nginx-consul-controller-admission", "agentIP": "100.96.140.174"}
2021-12-08T08:19:39.476Z INFO controller.endpoints registering proxy service with Consul {"name": "ingress-nginx-consul-controller-admission-sidecar-proxy"}
2021-12-08T08:19:39.496Z INFO controller.endpoints updating health check status for service {"name": "ingress-nginx-consul-controller-admission", "reason": "Pod \"ingress-nginx-consul/ingress-nginx-consul-controller-7f5997bd74-29l4w\" is not ready", "status": "critical"}
2021-12-08T08:19:39.557Z INFO controller.endpoints registering service with Consul {"name": "ingress-nginx-consul-controller-admission", "id": "ingress-nginx-consul-controller-7f5997bd74-bqlqh-ingress-nginx-consul-controller-admission", "agentIP": "100.96.132.75"}
2021-12-08T08:19:39.561Z INFO controller.endpoints registering proxy service with Consul {"name": "ingress-nginx-consul-controller-admission-sidecar-proxy"}
2021-12-08T08:19:39.580Z INFO controller.endpoints updating health check status for service {"name": "ingress-nginx-consul-controller-admission", "reason": "Pod \"ingress-nginx-consul/ingress-nginx-consul-controller-7f5997bd74-bqlqh\" is not ready", "status": "critical"}
<snip>
2021-12-08T08:19:40.075Z INFO controller.endpoints retrieved {"name": "ingress-nginx-consul-controller", "ns": "ingress-nginx-consul"}
2021-12-08T08:19:40.076Z INFO controller.endpoints registering service with Consul {"name": "ingress-nginx-consul-controller", "id": "ingress-nginx-consul-controller-7f5997bd74-bqlqh-ingress-nginx-consul-controller", "agentIP": "100.96.132.75"}
2021-12-08T08:19:40.248Z INFO controller.endpoints registering proxy service with Consul {"name": "ingress-nginx-consul-controller-sidecar-proxy"}
2021-12-08T08:19:40.292Z INFO controller.endpoints updating health check status for service {"name": "ingress-nginx-consul-controller", "reason": "Pod \"ingress-nginx-consul/ingress-nginx-consul-controller-7f5997bd74-bqlqh\" is not ready", "status": "critical"}
2021-12-08T08:19:40.295Z INFO controller.endpoints registering service with Consul {"name": "ingress-nginx-consul-controller", "id": "ingress-nginx-consul-controller-7f5997bd74-29l4w-ingress-nginx-consul-controller", "agentIP": "100.96.140.174"}
2021-12-08T08:19:40.297Z INFO controller.endpoints registering proxy service with Consul {"name": "ingress-nginx-consul-controller-sidecar-proxy"}
2021-12-08T08:19:40.309Z INFO controller.endpoints updating health check status for service {"name": "ingress-nginx-consul-controller", "reason": "Pod \"ingress-nginx-consul/ingress-nginx-consul-controller-7f5997bd74-29l4w\" is not ready", "status": "critical"}
Expected behavior
I was hoping that by excluding the two ingress-nginx services (ingress-nginx-consul-controller-admission and ingress-nginx-consul-controller-metrics) that consul would successfully enable the ingress-controller pods to start and inject a transparent envoy sidecar as expected.
Environment details
as above.
Additional Context
The text was updated successfully, but these errors were encountered:
I built a docker image for the control-plane from the main branch and updated my environment. After applying the consul.hashicorp.com/service-ignore: "true" label to the two services, I could see them being de-registered in the consul logs and now the pods start up ok:
kubectl get pod -n ingress-nginx-consul
NAME READY STATUS RESTARTS AGE
ingress-nginx-consul-controller-ddb6988b8-kgn9b 2/2 Running 0 8m2s
ingress-nginx-consul-controller-ddb6988b8-xpg6b 2/2 Running 0 8m2s
Sorry for the noise - this unblocks me for now. I'll close the issue!
Community Note
Overview of the Issue
I'm attempting to configure ingress-nginx as the k8s ingress for services which are part of a newly deployed consul-connect service mesh. I understand that there is an existing issue wherein consul won't support multiple services that have selectors for the same pod(s).
To try overcome that, i've used the
consul.hashicorp.com/connect-ignore=true
label as suggested by #592 , However, the two services I attempt to exclude still appear to be (attempting to) register to consul.Reproduction Steps
On EKS v1.18
deploy/config.yml
:versions deployed by above chart:
Install ingress-nginx via helm chart.
Add
consul.hashicorp.com/connect-ignore=true
label to theingress-nginx-consul-controller-admission
andingress-nginx-consul-controller-metrics
servicesLogs
Ingress controller pods won't start:
Logs from consul connect injector suggest that it is not ignoring the two services
ingress-nginx-consul-controller-admission
andingress-nginx-consul-controller-metrics
:Expected behavior
I was hoping that by excluding the two ingress-nginx services (
ingress-nginx-consul-controller-admission
andingress-nginx-consul-controller-metrics
) that consul would successfully enable the ingress-controller pods to start and inject a transparent envoy sidecar as expected.Environment details
as above.
Additional Context
The text was updated successfully, but these errors were encountered: