panic in BgpServer.AddNeighbor?

[murali-reddy]

Kube-router project use GoBGP as library. We ran into an issue where AddNeighbor() call seems resulting in panic. We are currently using v1.23

Appriciate any help any pointers to figure out whats going wrong.

1 @ 0x42cb6a 0x42cc4e 0x404e31 0x404a65 0x1347d87 0x457518 0x42a92f 0x4297de 0x43fe1f 0x131eb19 0x132e0c8 0x142c7dc 0x142edee 0xf60dea 0xf4633a 0x459c61
#	0x1347d86	github.com/cloudnativelabs/kube-router/vendor/github.com/osrg/gobgp/server.(*BgpServer).mgmtOperation.func1+0x56	/home/travis/gopath/src/github.com/cloudnativelabs/kube-router/vendor/github.com/osrg/gobgp/server/server.go:173
#	0x457517	runtime.call32+0x47													/home/travis/.gimme/versions/go1.8.5.linux.amd64/src/runtime/asm_amd64.s:514
#	0x42a92e	runtime.gopanic+0x2ce													/home/travis/.gimme/versions/go1.8.5.linux.amd64/src/runtime/panic.go:489
#	0x4297dd	runtime.panicmem+0x5d													/home/travis/.gimme/versions/go1.8.5.linux.amd64/src/runtime/panic.go:63
#	0x43fe1e	runtime.sigpanic+0x29e													/home/travis/.gimme/versions/go1.8.5.linux.amd64/src/runtime/signal_unix.go:290
#	0x131eb19	github.com/cloudnativelabs/kube-router/vendor/github.com/osrg/gobgp/server.(*BgpServer).mgmtOperation+0xe9		/home/travis/gopath/src/github.com/cloudnativelabs/kube-router/vendor/github.com/osrg/gobgp/server/server.go:174
#	0x132e0c7	github.com/cloudnativelabs/kube-router/vendor/github.com/osrg/gobgp/server.(*BgpServer).AddNeighbor+0x87		/home/travis/gopath/src/github.com/cloudnativelabs/kube-router/vendor/github.com/osrg/gobgp/server/server.go:1809
#	0x142c7db	github.com/cloudnativelabs/kube-router/app/controllers.(*NetworkRoutingController).syncInternalPeers+0x5ab		/home/travis/gopath/src/github.com/cloudnativelabs/kube-router/app/controllers/network_routes_controller.go:1044
#	0x142eded	github.com/cloudnativelabs/kube-router/app/controllers.(*NetworkRoutingController).OnNodeUpdate+0xfd			/home/travis/gopath/src/github.com/cloudnativelabs/kube-router/app/controllers/network_routes_controller.go:1221
#	0xf60de9	github.com/cloudnativelabs/kube-router/app/watchers.(*nodeWatcher).RegisterHandler.func1+0x49				/home/travis/gopath/src/github.com/cloudnativelabs/kube-router/app/watchers/node_watcher.go:57
#	0xf46339	github.com/cloudnativelabs/kube-router/utils.ListenerFunc.OnUpdate+0x39							/home/travis/gopath/src/github.com/cloudnativelabs/kube-router/utils/utils.go:14

[fujita]

I can't reproduce. Always happens? What argument is passed to AddNeighbor()?

[murali-reddy]

thanks @fujita for your response. Its actually simple call. It does not happen always.

https://github.com/cloudnativelabs/kube-router/blob/v0.1.0-rc1/app/controllers/network_routes_controller.go#L1216-L1240

I am not sure if its GoBGP issue, but irrespective as caller of the function, it would be ideal if the function returns gracefully with error. I can easily recreate the error. Let me know if you need other informatin.

[murali-reddy]

Problem turned out to be somewhere else. Sorry for the noise.

[fujita]

That's good to hear. Out of curiosity, what was the problem?

[murali-reddy]

@fujita basically i have hit this issue
golang/go#20060

basically i had map to hold current set of BGP neighbours and two go routines which update this map

go func() {
    // go routine 1
    // lock the mutex
    // update the map
    // call gobgp add nieghbour call
}

go func() {
    // go routine 2
    // lock the mutex
    // update the map
    // call gobgp add nieghbour call
}

Strangley panic was always shown to be inside gobgp library in stack trace. which lead to opening this issue.