Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

subnet: handle unknown state in delete event #1000

Merged
merged 1 commit into from
Jul 9, 2018

Conversation

ggaaooppeenngg
Copy link
Contributor

@ggaaooppeenngg ggaaooppeenngg commented May 31, 2018

To fix panic (see log below), add a handler for
cache.DeletedFinalStateUnknown.

/usr/local/go/src/runtime/asm_amd64.s:2197
panic: interface conversion: interface {} is cache.DeletedFinalStateUnknown, not *v1.Node [recovered]
	panic: interface conversion: interface {} is cache.DeletedFinalStateUnknown, not *v1.Node [recovered]
	panic: interface conversion: interface {} is cache.DeletedFinalStateUnknown, not *v1.Node

goroutine 19 [running]:
github.com/coreos/flannel/vendor/k8s.io/apimachinery/pkg/util/runtime.HandleCrash(0x0, 0x0, 0x0)
	/go/src/github.com/coreos/flannel/vendor/k8s.io/apimachinery/pkg/util/runtime/runtime.go:58 +0x126
panic(0x1782360, 0xc420945e80)
	/usr/local/go/src/runtime/panic.go:489 +0x2cf
github.com/coreos/flannel/vendor/k8s.io/apimachinery/pkg/util/runtime.HandleCrash(0x0, 0x0, 0x0)
	/go/src/github.com/coreos/flannel/vendor/k8s.io/apimachinery/pkg/util/runtime/runtime.go:58 +0x126
panic(0x1782360, 0xc420945e80)
	/usr/local/go/src/runtime/panic.go:489 +0x2cf
github.com/coreos/flannel/subnet/kube.(*kubeSubnetManager).handleAddLeaseEvent(0xc420014f00, 0x1, 0x17e38a0, 0xc4204f61c0)
	/go/src/github.com/coreos/flannel/subnet/kube/kube.go:176 +0x2d3
github.com/coreos/flannel/subnet/kube.newKubeSubnetManager.func4(0x17e38a0, 0xc4204f61c0)
	/go/src/github.com/coreos/flannel/subnet/kube/kube.go:165 +0x47
github.com/coreos/flannel/vendor/k8s.io/client-go/tools/cache.ResourceEventHandlerFuncs.OnDelete(0xc4202ba6e0, 0xc4202ba6f0, 0xc4202ba700, 0x17e38a0, 0xc4204f61c0)
	/go/src/github.com/coreos/flannel/vendor/k8s.io/client-go/tools/cache/controller.go:206 +0x49
github.com/coreos/flannel/vendor/k8s.io/client-go/tools/cache.(*ResourceEventHandlerFuncs).OnDelete(0xc42018c780, 0x17e38a0, 0xc4204f61c0)
	<autogenerated>:56 +0x73
github.com/coreos/flannel/vendor/k8s.io/client-go/tools/cache.NewIndexerInformer.func1(0x1792520, 0xc4204f6680, 0x1792520, 0xc4204f6680)
	/go/src/github.com/coreos/flannel/vendor/k8s.io/client-go/tools/cache/controller.go:342 +0x519
github.com/coreos/flannel/vendor/k8s.io/client-go/tools/cache.(*DeltaFIFO).Pop(0xc4203120c0, 0xc4203aede0, 0x0, 0x0, 0x0, 0x0)
	/go/src/github.com/coreos/flannel/vendor/k8s.io/client-go/tools/cache/delta_fifo.go:451 +0x27e
github.com/coreos/flannel/vendor/k8s.io/client-go/tools/cache.(*controller).processLoop(0xc420067180)
	/go/src/github.com/coreos/flannel/vendor/k8s.io/client-go/tools/cache/controller.go:147 +0x40
github.com/coreos/flannel/vendor/k8s.io/client-go/tools/cache.(*controller).(github.com/coreos/flannel/vendor/k8s.io/client-go/tools/cache.processLoop)-fm()
	/go/src/github.com/coreos/flannel/vendor/k8s.io/client-go/tools/cache/controller.go:121 +0x2a
github.com/coreos/flannel/vendor/k8s.io/apimachinery/pkg/util/wait.JitterUntil.func1(0xc4205fcf70)
	/go/src/github.com/coreos/flannel/vendor/k8s.io/apimachinery/pkg/util/wait/wait.go:96 +0x5e
github.com/coreos/flannel/vendor/k8s.io/apimachinery/pkg/util/wait.JitterUntil(0xc420865f70, 0x3b9aca00, 0x0, 0x166ac01, 0x0)
	/go/src/github.com/coreos/flannel/vendor/k8s.io/apimachinery/pkg/util/wait/wait.go:97 +0xad
github.com/coreos/flannel/vendor/k8s.io/apimachinery/pkg/util/wait.Until(0xc4205fcf70, 0x3b9aca00, 0x0)
	/go/src/github.com/coreos/flannel/vendor/k8s.io/apimachinery/pkg/util/wait/wait.go:52 +0x4d
github.com/coreos/flannel/vendor/k8s.io/client-go/tools/cache.(*controller).Run(0xc420067180, 0x0)
	/go/src/github.com/coreos/flannel/vendor/k8s.io/client-go/tools/cache/controller.go:121 +0x237
github.com/coreos/flannel/subnet/kube.(*kubeSubnetManager).Run(0xc420014f00, 0x7f340ee4d3b0, 0xc4200161a8)
	/go/src/github.com/coreos/flannel/subnet/kube/kube.go:284 +0x8a
created by github.com/coreos/flannel/subnet/kube.NewSubnetManager
	/go/src/github.com/coreos/flannel/subnet/kube/kube.go:128 +0x592

Signed-off-by: Peng Gao peng.gao.dut@gmail.com

Description

Todos

  • Tests
  • Documentation
  • Release note

Release Note

None required

To fix panic (see log below), add a handler for
cache.DeletedFinalStateUnknown.

```
/usr/local/go/src/runtime/asm_amd64.s:2197
panic: interface conversion: interface {} is cache.DeletedFinalStateUnknown, not *v1.Node [recovered]
	panic: interface conversion: interface {} is cache.DeletedFinalStateUnknown, not *v1.Node [recovered]
	panic: interface conversion: interface {} is cache.DeletedFinalStateUnknown, not *v1.Node

goroutine 19 [running]:
github.com/coreos/flannel/vendor/k8s.io/apimachinery/pkg/util/runtime.HandleCrash(0x0, 0x0, 0x0)
	/go/src/github.com/coreos/flannel/vendor/k8s.io/apimachinery/pkg/util/runtime/runtime.go:58 +0x126
panic(0x1782360, 0xc420945e80)
	/usr/local/go/src/runtime/panic.go:489 +0x2cf
github.com/coreos/flannel/vendor/k8s.io/apimachinery/pkg/util/runtime.HandleCrash(0x0, 0x0, 0x0)
	/go/src/github.com/coreos/flannel/vendor/k8s.io/apimachinery/pkg/util/runtime/runtime.go:58 +0x126
panic(0x1782360, 0xc420945e80)
	/usr/local/go/src/runtime/panic.go:489 +0x2cf
github.com/coreos/flannel/subnet/kube.(*kubeSubnetManager).handleAddLeaseEvent(0xc420014f00, 0x1, 0x17e38a0, 0xc4204f61c0)
	/go/src/github.com/coreos/flannel/subnet/kube/kube.go:176 +0x2d3
github.com/coreos/flannel/subnet/kube.newKubeSubnetManager.func4(0x17e38a0, 0xc4204f61c0)
	/go/src/github.com/coreos/flannel/subnet/kube/kube.go:165 +0x47
github.com/coreos/flannel/vendor/k8s.io/client-go/tools/cache.ResourceEventHandlerFuncs.OnDelete(0xc4202ba6e0, 0xc4202ba6f0, 0xc4202ba700, 0x17e38a0, 0xc4204f61c0)
	/go/src/github.com/coreos/flannel/vendor/k8s.io/client-go/tools/cache/controller.go:206 +0x49
github.com/coreos/flannel/vendor/k8s.io/client-go/tools/cache.(*ResourceEventHandlerFuncs).OnDelete(0xc42018c780, 0x17e38a0, 0xc4204f61c0)
	<autogenerated>:56 +0x73
github.com/coreos/flannel/vendor/k8s.io/client-go/tools/cache.NewIndexerInformer.func1(0x1792520, 0xc4204f6680, 0x1792520, 0xc4204f6680)
	/go/src/github.com/coreos/flannel/vendor/k8s.io/client-go/tools/cache/controller.go:342 +0x519
github.com/coreos/flannel/vendor/k8s.io/client-go/tools/cache.(*DeltaFIFO).Pop(0xc4203120c0, 0xc4203aede0, 0x0, 0x0, 0x0, 0x0)
	/go/src/github.com/coreos/flannel/vendor/k8s.io/client-go/tools/cache/delta_fifo.go:451 +0x27e
github.com/coreos/flannel/vendor/k8s.io/client-go/tools/cache.(*controller).processLoop(0xc420067180)
	/go/src/github.com/coreos/flannel/vendor/k8s.io/client-go/tools/cache/controller.go:147 +0x40
github.com/coreos/flannel/vendor/k8s.io/client-go/tools/cache.(*controller).(github.com/coreos/flannel/vendor/k8s.io/client-go/tools/cache.processLoop)-fm()
	/go/src/github.com/coreos/flannel/vendor/k8s.io/client-go/tools/cache/controller.go:121 +0x2a
github.com/coreos/flannel/vendor/k8s.io/apimachinery/pkg/util/wait.JitterUntil.func1(0xc4205fcf70)
	/go/src/github.com/coreos/flannel/vendor/k8s.io/apimachinery/pkg/util/wait/wait.go:96 +0x5e
github.com/coreos/flannel/vendor/k8s.io/apimachinery/pkg/util/wait.JitterUntil(0xc420865f70, 0x3b9aca00, 0x0, 0x166ac01, 0x0)
	/go/src/github.com/coreos/flannel/vendor/k8s.io/apimachinery/pkg/util/wait/wait.go:97 +0xad
github.com/coreos/flannel/vendor/k8s.io/apimachinery/pkg/util/wait.Until(0xc4205fcf70, 0x3b9aca00, 0x0)
	/go/src/github.com/coreos/flannel/vendor/k8s.io/apimachinery/pkg/util/wait/wait.go:52 +0x4d
github.com/coreos/flannel/vendor/k8s.io/client-go/tools/cache.(*controller).Run(0xc420067180, 0x0)
	/go/src/github.com/coreos/flannel/vendor/k8s.io/client-go/tools/cache/controller.go:121 +0x237
github.com/coreos/flannel/subnet/kube.(*kubeSubnetManager).Run(0xc420014f00, 0x7f340ee4d3b0, 0xc4200161a8)
	/go/src/github.com/coreos/flannel/subnet/kube/kube.go:284 +0x8a
created by github.com/coreos/flannel/subnet/kube.NewSubnetManager
	/go/src/github.com/coreos/flannel/subnet/kube/kube.go:128 +0x592
```

Signed-off-by: Peng Gao <peng.gao.dut@gmail.com>
Copy link
Contributor

@tomdee tomdee left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM

@tomdee tomdee merged commit c7ee1fc into flannel-io:master Jul 9, 2018
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

None yet

2 participants