Sharing code to reproduce an issue with Kubernetes
To install the following code you'll need
- A kubernetes cluster.
Install steps to reproduce the issue
Deploying with skaffold
If you have skaffold installed.
$ skaffold run # or $ skaffold dev
Deploying with kubectl
$ kubectl apply -f kube
⚠️ Important note ⚠️
Some of the clients pod must be scheduled on a different node (it can be just one different node) to reproduce the issue.
We identified that when everything was scheduled on only one node the problem was harder to reproduce.
Identifying the issue
Check on Stackdriver logging the log from the server and client.
on the search bar you can set
text:'reset by peer' and you'll see the
# Error reported by the server E [info] 11#11: *21885 client 10.20.6.249 closed keepalive connection (104: Connection reset by peer) # Error reported by the client E curl: (56) Recv failure: Connection reset by peer
Both line should follow each other. If the client and the server are not
reporting at the same time a
Connection reset by peer you can ignore the
event as it might be something else.
If the clients seem stuck feel free to restart the pods like the following
kubectl delete pod -l app=client