Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

hubble/observer: increment 'numObservedFlows' atomically #11835

Merged
merged 1 commit into from Jun 2, 2020
Merged

hubble/observer: increment 'numObservedFlows' atomically #11835

merged 1 commit into from Jun 2, 2020

Conversation

aanm
Copy link
Member

@aanm aanm commented Jun 2, 2020

Fixes: a1d0430 ("hubble/server: Populate total number of flows and uptime in status")
Signed-off-by: André Martins andre@cilium.io

Fixes:

2020-06-02T15:51:20.650478448Z WARNING: DATA RACE
2020-06-02T15:51:20.650502009Z Read at 0x00c001123ae0 by goroutine 394:
2020-06-02T15:51:20.650505428Z   github.com/cilium/cilium/pkg/hubble/observer.(*LocalObserverServer).ServerStatus()
2020-06-02T15:51:20.650507539Z       /go/src/github.com/cilium/cilium/pkg/hubble/observer/local_observer.go:227 +0x1b7
2020-06-02T15:51:20.650514316Z   github.com/cilium/cilium/daemon/cmd.(*Daemon).getHubbleStatus()
2020-06-02T15:51:20.650516454Z       /go/src/github.com/cilium/cilium/daemon/cmd/hubble.go:60 +0x14e
2020-06-02T15:51:20.650518465Z   github.com/cilium/cilium/daemon/cmd.(*Daemon).startStatusCollector.func22()
2020-06-02T15:51:20.650520429Z       /go/src/github.com/cilium/cilium/daemon/cmd/status.go:672 +0x55
2020-06-02T15:51:20.650525783Z   github.com/cilium/cilium/pkg/status.(*Collector).runProbe.func1()
2020-06-02T15:51:20.650527926Z       /go/src/github.com/cilium/cilium/pkg/status/status.go:181 +0x63
2020-06-02T15:51:20.650529909Z 
2020-06-02T15:51:20.650549673Z Previous write at 0x00c001123ae0 by goroutine 258:
2020-06-02T15:51:20.650552615Z   github.com/cilium/cilium/pkg/hubble/observer.(*LocalObserverServer).Start()
2020-06-02T15:51:20.650554648Z       /go/src/github.com/cilium/cilium/pkg/hubble/observer/local_observer.go:173 +0x5d5
2020-06-02T15:51:20.650560059Z 
2020-06-02T15:51:20.650562138Z Goroutine 394 (running) created at:
2020-06-02T15:51:20.650567399Z   github.com/cilium/cilium/pkg/status.(*Collector).runProbe()
2020-06-02T15:51:20.65056961Z       /go/src/github.com/cilium/cilium/pkg/status/status.go:180 +0x37b
2020-06-02T15:51:20.650574231Z   github.com/cilium/cilium/pkg/status.(*Collector).spawnProbe.func1()
2020-06-02T15:51:20.650576391Z       /go/src/github.com/cilium/cilium/pkg/status/status.go:145 +0x5d
2020-06-02T15:51:20.65058082Z 
2020-06-02T15:51:20.650592687Z Goroutine 258 (running) created at:
2020-06-02T15:51:20.650595061Z   github.com/cilium/cilium/daemon/cmd.(*Daemon).launchHubble()
2020-06-02T15:51:20.650597075Z       /go/src/github.com/cilium/cilium/daemon/cmd/hubble.go:107 +0x774
2020-06-02T15:51:20.650601616Z   github.com/cilium/cilium/daemon/cmd.runDaemon()
2020-06-02T15:51:20.650603687Z       /go/src/github.com/cilium/cilium/daemon/cmd/daemon_main.go:1425 +0x1410
2020-06-02T15:51:20.65060812Z   github.com/cilium/cilium/daemon/cmd.NewDaemon()
2020-06-02T15:51:20.650610675Z       /go/src/github.com/cilium/cilium/daemon/cmd/daemon.go:496 +0x2bb6
2020-06-02T15:51:20.650615142Z   github.com/cilium/cilium/daemon/cmd.runDaemon()
2020-06-02T15:51:20.650617173Z       /go/src/github.com/cilium/cilium/daemon/cmd/daemon_main.go:1281 +0x354
2020-06-02T15:51:20.650623955Z   github.com/cilium/cilium/daemon/cmd.glob..func1()
2020-06-02T15:51:20.650626146Z       /go/src/github.com/cilium/cilium/daemon/cmd/daemon_main.go:116 +0xab
2020-06-02T15:51:20.650628123Z   github.com/cilium/cilium/daemon/cmd.glob..func1()
2020-06-02T15:51:20.650630075Z       /go/src/github.com/cilium/cilium/daemon/cmd/daemon_main.go:114 +0x91
2020-06-02T15:51:20.650635547Z   github.com/spf13/cobra.(*Command).execute()
2020-06-02T15:51:20.650637602Z       /go/src/github.com/cilium/cilium/vendor/github.com/spf13/cobra/command.go:846 +0x8e0
2020-06-02T15:51:20.650639616Z   github.com/spf13/cobra.(*Command).ExecuteC()
2020-06-02T15:51:20.650641541Z       /go/src/github.com/cilium/cilium/vendor/github.com/spf13/cobra/command.go:950 +0x499
2020-06-02T15:51:20.650645978Z   github.com/spf13/cobra.(*Command).Execute()
2020-06-02T15:51:20.650648019Z       /go/src/github.com/cilium/cilium/vendor/github.com/spf13/cobra/command.go:887 +0x1eb
2020-06-02T15:51:20.650652508Z   github.com/cilium/cilium/daemon/cmd.Execute()
2020-06-02T15:51:20.65065455Z       /go/src/github.com/cilium/cilium/daemon/cmd/daemon_main.go:147 +0x1cc
2020-06-02T15:51:20.650659036Z   main.main()
2020-06-02T15:51:20.650661094Z       /go/src/github.com/cilium/cilium/daemon/main.go:22 +0x2f

@aanm
hubble/observer: increment 'numObservedFlows' atomically
6978bde 
Fixes: a1d0430 ("hubble/server: Populate total number of flows and uptime in status")
Signed-off-by: André Martins <andre@cilium.io>
@aanm aanm added kind/bug This is a bug in the Cilium logic. needs-backport/1.8 labels Jun 2, 2020
@aanm aanm requested a review from a team June 2, 2020 17:18
@maintainer-s-little-helper
Copy link

Please set the appropriate release note label.

2 similar comments
@maintainer-s-little-helper
Copy link

Please set the appropriate release note label.

@maintainer-s-little-helper
Copy link

Please set the appropriate release note label.

@maintainer-s-little-helper maintainer-s-little-helper bot added this to In progress in 1.8.0 Jun 2, 2020
@maintainer-s-little-helper maintainer-s-little-helper bot added this to Needs backport from master in 1.8.0 Jun 2, 2020
@aanm aanm added the release-note/misc This PR makes changes that have no direct user impact. label Jun 2, 2020
@aanm
Copy link
Member Author

aanm commented Jun 2, 2020

test-me-please

@rolinh rolinh added the sig/hubble Impacts hubble server or relay label Jun 2, 2020
@maintainer-s-little-helper maintainer-s-little-helper bot added the ready-to-merge This PR has passed all tests and received consensus from code owners to merge. label Jun 2, 2020
@borkmann borkmann merged commit 9f2a0ad into master Jun 2, 2020
1.8.0 automation moved this from In progress to Merged Jun 2, 2020
@borkmann borkmann deleted the pr/fix-datarace-hubble-status branch June 2, 2020 20:46
@tklauser tklauser mentioned this pull request Jun 3, 2020
Merged
@maintainer-s-little-helper maintainer-s-little-helper bot moved this from Needs backport from master to Backport pending to v1.8 in 1.8.0 Jun 3, 2020
@maintainer-s-little-helper maintainer-s-little-helper bot moved this from Needs backport from master to Backport pending to v1.8 in 1.8.0 Jun 3, 2020
@joestringer joestringer moved this from Backport pending to v1.8 to Backport done to v1.8 in 1.8.0 Jun 6, 2020
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Reviewers

@gandro gandro gandro approved these changes

@glibsm glibsm glibsm approved these changes

@rolinh rolinh rolinh approved these changes

@michi-covalent michi-covalent michi-covalent approved these changes

Assignees
No one assigned
Labels
kind/bug This is a bug in the Cilium logic. ready-to-merge This PR has passed all tests and received consensus from code owners to merge. release-note/misc This PR makes changes that have no direct user impact. sig/hubble Impacts hubble server or relay
Projects
No open projects
1.8.0
  
Merged
1.8.0
Backport done to v1.8
Milestone
No milestone
Development

Successfully merging this pull request may close these issues.

None yet
7 participants
@aanm @gandro @glibsm @rolinh @michi-covalent @tklauser @borkmann