Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Proxy seldomly not informed about lost LB after node reboot #409

Open
zolug opened this issue Apr 21, 2023 · 2 comments
Open

Proxy seldomly not informed about lost LB after node reboot #409

zolug opened this issue Apr 21, 2023 · 2 comments

Comments

@zolug
Copy link
Collaborator

zolug commented Apr 21, 2023

Describe the bug
It has been seen that seldomly some proxies are not informed about the disappearance of an LB that was hosted by a rebooted node.
In the most recent case the rebooted node also hosted an NSM k8s-registry (3 replicas), a spire server (3 replicas), Meridio IPAM and NSP.

It causes the old NSM interfaces to the unavailable LB POD to remain in the routing config thus causing traffic disturbance.

To Reproduce
Steps to reproduce the behavior:
???

  1. Setup a cluster with 7-8 proies and 2 LBs and 3 NSM k8s registries.
  2. Reboot a node hosting an LB and possibly a NSM k8s-registry.
  3. Check the proxy logs if they have received any notification about the lost LB NSE.

Expected behavior
Proxies should be notified through the NSM endpoint watcher about the lost LB NSE (and maybe via NSM heal as well about its unavailability).

Context

  • Network Service Mesh: [e.g. v1.6.2]
  • Meridio:1.1.0

Logs
Add logs here.

@zolug
Copy link
Collaborator Author

zolug commented Oct 16, 2023

Could be (partially) related to networkservicemesh/sdk-k8s#456

@zolug
Copy link
Collaborator Author

zolug commented Aug 13, 2024

Check again with NSM 1.13.2 and most recent Meridio.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
Status: 📋 To Do
Development

No branches or pull requests

1 participant