Skip to content

Commit

Permalink
devices: Fix panic in tests when logger used after stopping
Browse files Browse the repository at this point in the history
The netlink unsubscribing is async and the error callback is sometimes
called with an error after unsubscribing:

  panic: Log in goroutine after TestDevicesController has completed:
  time=2024-05-14T15:40:53.086Z level=WARN
  msg="Netlink error received, restarting" module=devices-controller
  error="Receive failed: Receive called on a closed socket"

Ignore the error if we've cancelled the context.

Before this fix:
$ PRIVILEGED_TESTS=1 sudo -E stress go test . -test.v -test.run Devices -test.count 2
20s: 32 runs so far, 25 failures (78.12%)

After this fix:
$ PRIVILEGED_TESTS=1 sudo -E stress go test . -test.v -test.run Devices -test.count 2
30s: 51 runs so far, 0 failures

Fixes: 18ed625 ("devices: Convert from logrus to slog")
Signed-off-by: Jussi Maki <jussi@isovalent.com>
  • Loading branch information
joamaki committed May 16, 2024
1 parent a672336 commit 66b31eb
Showing 1 changed file with 7 additions and 0 deletions.
7 changes: 7 additions & 0 deletions pkg/datapath/linux/devices_controller.go
Original file line number Diff line number Diff line change
Expand Up @@ -195,6 +195,13 @@ func (dc *devicesController) subscribeAndProcess(ctx context.Context) {
// It cancels the context to unsubscribe from netlink updates
// which stops the processing.
errorCallback := func(err error) {
if ctx.Err() != nil {
// The netlink unsubscribe can lead to errorCallback being called after
// context cancellation with a "receive called on closed socket".
// Thus ignore the error if the context was cancelled.
return
}

dc.log.Warn("Netlink error received, restarting", logfields.Error, err)

// Cancel the context to stop the subscriptions.
Expand Down

0 comments on commit 66b31eb

Please sign in to comment.