Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Lost and don't reconnect app after consul client pod died #161

Closed
Veres72 opened this issue Nov 5, 2019 · 2 comments · Fixed by hashicorp/consul-helm#298 or #176
Closed

Lost and don't reconnect app after consul client pod died #161

Veres72 opened this issue Nov 5, 2019 · 2 comments · Fixed by hashicorp/consul-helm#298 or #176
Assignees
Labels
area/connect Related to Connect service mesh, e.g. injection type/bug Something isn't working

Comments

@Veres72
Copy link

Veres72 commented Nov 5, 2019

My installation:
kubernetes v1.14.3
consul within kubernetes cluster v1.6.1
consul connect with consul-k8s v0.9.0
I enabled HostNetwork: true for accessing my connect sidecars to my consul client on worker nodes.
Problem:
If consul client pod are dead and recreate, all pod with consul connect on this worker lost and don't reconnect to consul. All connections from this app to another app through consul connect didn't work.
How reproduce:
Install consul connect via helm and enable HostNetwork: true for consul client. Deploy several apps with consul connect and delete consul client pod.

@Veres72 Veres72 changed the title Lost and don't reconnect app when client pod deed Lost and don't reconnect app after consul client pod died Nov 5, 2019
@adilyse adilyse added area/connect Related to Connect service mesh, e.g. injection type/bug Something isn't working labels Nov 7, 2019
@lkysow lkysow self-assigned this Dec 2, 2019
@lkysow
Copy link
Member

lkysow commented Dec 10, 2019

Hi @Veres72 , the latest version of the consul-helm chart (v0.14.0) supports a client.dataDirectoryHostPath config that will mount Consul's data on the host disk. When the agent restarts, it will re-register its services. If you use that, be sure to also be running with PodSecurityPolicies so other Pods can't mount the same hostPath.

We're working on a better fix via a sidecar that will re-register the Pod if it is deregistered. Watch #171 for that.

@lkysow
Copy link
Member

lkysow commented Dec 17, 2019

This is fixed in consul-helm v0.15.0

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
area/connect Related to Connect service mesh, e.g. injection type/bug Something isn't working
Projects
None yet
3 participants