New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
daemon: fix panic when starting Cilium #12101
Conversation
It seems the we try to validate an endpoint without setting its allocator which causes Cilium to panic while restoring old endpoints: ``` goroutine 259 [running]: github.com/cilium/cilium/pkg/endpoint.(*Endpoint).identityLabelsChanged(0xc000b122c0, 0x280e920, 0xc0009d2840, 0x1, 0x0, 0x0, 0x0) /go/src/github.com/cilium/cilium/pkg/endpoint/endpoint.go:1818 +0x45c github.com/cilium/cilium/pkg/endpoint.(*Endpoint).runIdentityResolver(0xc000b122c0, 0x280e920, 0xc0009d2840, 0x1, 0x1, 0xc000cb2d80) /go/src/github.com/cilium/cilium/pkg/endpoint/endpoint.go:1748 +0x3db github.com/cilium/cilium/pkg/endpoint.(*Endpoint).UpdateLabels(0xc000b122c0, 0x280e920, 0xc0009d2840, 0xc000d2ca50, 0xc000d2cab0, 0x1, 0x6) /go/src/github.com/cilium/cilium/pkg/endpoint/endpoint.go:1709 +0x3d0 github.com/cilium/cilium/pkg/endpoint.(*Endpoint).RunMetadataResolver.func2(0x280e920, 0xc0009d2840, 0x3b4b200, 0xc000ce31b8) /go/src/github.com/cilium/cilium/pkg/endpoint/endpoint.go:1621 +0x46a github.com/cilium/cilium/pkg/controller.(*Controller).runController(0xc0000d6800) /go/src/github.com/cilium/cilium/pkg/controller/controller.go:205 +0xa2a created by github.com/cilium/cilium/pkg/controller.(*Manager).updateController /go/src/github.com/cilium/cilium/pkg/controller/manager.go:120 +0xb09 ``` With this change we set the allocator in the endpoint before we validate the endpoint, avoiding it to panic. Signed-off-by: André Martins <andre@cilium.io>
test-me-please |
retest-4.19 |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Is this a reliable failure? Just wondering when we broke it and whether it could affect earlier releases.
@joestringer I hit this by running #11903 (comment) and restarting Cilium pod. Not sure if it affects older versions. I have a feeling that #11714 might have introduced this regression. |
It seems the we try to validate an endpoint without setting its
allocator which causes Cilium to panic while restoring old endpoints:
With this change we set the allocator in the endpoint before we validate
the endpoint, avoiding it to panic.
Signed-off-by: André Martins andre@cilium.io