Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

[Race Detector] Race in proxylib between (*ClientSuite).TestRequestAllResources() and (*ClientSuite).UpsertNetworkPolicy #16315

Closed
jibi opened this issue May 26, 2021 · 2 comments · Fixed by #17141
Closed

[Race Detector] Race in proxylib between (*ClientSuite).TestRequestAllResources() and (*ClientSuite).UpsertNetworkPolicy #16315

jibi opened this issue May 26, 2021 · 2 comments · Fixed by #17141
Labels
area/CI Continuous Integration testing issue or flake area/proxy Impacts proxy components, including DNS, Kafka, Envoy and/or XDS servers. ci/flake This is a known failure that occurs in the tree. Please investigate me! kind/bug/race-detector

Comments

@jibi
Copy link
Member

jibi commented May 26, 2021

Found in Travis CI:

WARNING: DATA RACE
Read at 0x00c000459fe0 by goroutine 24:
  github.com/cilium/cilium/proxylib/npds.(*ClientSuite).TestRequestAllResources()
      /home/travis/gopath/src/github.com/cilium/cilium/proxylib/npds/client_test.go:108 +0x2b7
  runtime.call16()
      /home/travis/.gimme/versions/go1.16.4.linux.amd64/src/runtime/asm_amd64.s:550 +0x3d
  reflect.Value.Call()
      /home/travis/.gimme/versions/go1.16.4.linux.amd64/src/reflect/value.go:337 +0xd8
  gopkg.in/check%2ev1.(*suiteRunner).forkTest.func1()
      /home/travis/gopath/src/github.com/cilium/cilium/vendor/gopkg.in/check.v1/check.go:775 +0xb3b
  gopkg.in/check%2ev1.(*suiteRunner).forkCall.func1()
      /home/travis/gopath/src/github.com/cilium/cilium/vendor/gopkg.in/check.v1/check.go:669 +0xe1

Previous write at 0x00c000459fe0 by goroutine 56:
  github.com/cilium/cilium/proxylib/npds.(*ClientSuite).UpsertNetworkPolicy.func1()
      /home/travis/gopath/src/github.com/cilium/cilium/proxylib/npds/client_test.go:67 +0x19a
  github.com/cilium/cilium/pkg/completion.(*Completion).Complete()
      /home/travis/gopath/src/github.com/cilium/cilium/pkg/completion/completion.go:166 +0x2b5
  github.com/cilium/cilium/pkg/envoy/xds.(*AckingResourceMutatorWrapper).HandleResourceVersionAck()
      /home/travis/gopath/src/github.com/cilium/cilium/pkg/envoy/xds/ack.go:339 +0xbb4
  github.com/cilium/cilium/pkg/envoy/xds.(*Server).processRequestStream()
      /home/travis/gopath/src/github.com/cilium/cilium/pkg/envoy/xds/server.go:353 +0x1093
  github.com/cilium/cilium/pkg/envoy/xds.(*Server).HandleRequestStream()
      /home/travis/gopath/src/github.com/cilium/cilium/pkg/envoy/xds/server.go:184 +0x395
  github.com/cilium/cilium/pkg/envoy.(*xdsGRPCServer).StreamNetworkPolicies()
      /home/travis/gopath/src/github.com/cilium/cilium/pkg/envoy/grpc.go:95 +0xc9
  github.com/cilium/proxy/go/cilium/api._NetworkPolicyDiscoveryService_StreamNetworkPolicies_Handler()
      /home/travis/gopath/src/github.com/cilium/cilium/vendor/github.com/cilium/proxy/go/cilium/api/npds.pb.go:1548 +0xcf
  google.golang.org/grpc.(*Server).processStreamingRPC()
      /home/travis/gopath/src/github.com/cilium/cilium/vendor/google.golang.org/grpc/server.go:1329 +0x1627
  google.golang.org/grpc.(*Server).handleStream()
      /home/travis/gopath/src/github.com/cilium/cilium/vendor/google.golang.org/grpc/server.go:1409 +0x131e
  google.golang.org/grpc.(*Server).serveStreams.func1.1()
      /home/travis/gopath/src/github.com/cilium/cilium/vendor/google.golang.org/grpc/server.go:746 +0xe6

Goroutine 24 (running) created at:
  gopkg.in/check%2ev1.(*suiteRunner).forkCall()
      /home/travis/gopath/src/github.com/cilium/cilium/vendor/gopkg.in/check.v1/check.go:666 +0x459
  gopkg.in/check%2ev1.(*suiteRunner).forkTest()
      /home/travis/gopath/src/github.com/cilium/cilium/vendor/gopkg.in/check.v1/check.go:757 +0x1c4
  gopkg.in/check%2ev1.(*suiteRunner).runTest()
      /home/travis/gopath/src/github.com/cilium/cilium/vendor/gopkg.in/check.v1/check.go:812 +0x23e
  gopkg.in/check%2ev1.(*suiteRunner).run()
      /home/travis/gopath/src/github.com/cilium/cilium/vendor/gopkg.in/check.v1/check.go:618 +0x1dc
  gopkg.in/check%2ev1.Run()
      /home/travis/gopath/src/github.com/cilium/cilium/vendor/gopkg.in/check.v1/run.go:92 +0x5a
  gopkg.in/check%2ev1.RunAll()
      /home/travis/gopath/src/github.com/cilium/cilium/vendor/gopkg.in/check.v1/run.go:84 +0x136
  gopkg.in/check%2ev1.TestingT()
      /home/travis/gopath/src/github.com/cilium/cilium/vendor/gopkg.in/check.v1/run.go:72 +0x5ef
  github.com/cilium/cilium/proxylib/npds.Test()
      /home/travis/gopath/src/github.com/cilium/cilium/proxylib/npds/client_test.go:37 +0x38
  testing.tRunner()
      /home/travis/.gimme/versions/go1.16.4.linux.amd64/src/testing/testing.go:1193 +0x202

Goroutine 56 (running) created at:
  google.golang.org/grpc.(*Server).serveStreams.func1()
      /home/travis/gopath/src/github.com/cilium/cilium/vendor/google.golang.org/grpc/server.go:744 +0xb8
  google.golang.org/grpc/internal/transport.(*http2Server).operateHeaders()
      /home/travis/gopath/src/github.com/cilium/cilium/vendor/google.golang.org/grpc/internal/transport/http2_server.go:442 +0x1850
  google.golang.org/grpc/internal/transport.(*http2Server).HandleStreams()
      /home/travis/gopath/src/github.com/cilium/cilium/vendor/google.golang.org/grpc/internal/transport/http2_server.go:483 +0x49c
  google.golang.org/grpc.(*Server).serveStreams()
      /home/travis/gopath/src/github.com/cilium/cilium/vendor/google.golang.org/grpc/server.go:742 +0x1c7
  google.golang.org/grpc.(*Server).handleRawConn.func1()
      /home/travis/gopath/src/github.com/cilium/cilium/vendor/google.golang.org/grpc/server.go:703 +0x4c

Might be related to #16296

/cc @jrajahalme
@jibi jibi added area/CI Continuous Integration testing issue or flake area/proxy Impacts proxy components, including DNS, Kafka, Envoy and/or XDS servers. ci/flake This is a known failure that occurs in the tree. Please investigate me! kind/bug/race-detector labels May 26, 2021
@stale
Copy link

stale bot commented Jul 30, 2021

This issue has been automatically marked as stale because it has not had recent activity. It will be closed if no further activity occurs.

@stale stale bot added the stale The stale bot thinks this issue is old. Add "pinned" label to prevent this from becoming stale. label Jul 30, 2021
@gandro
Copy link
Member

gandro commented Aug 11, 2021

Still present on master https://app.travis-ci.com/github/cilium/cilium/jobs/530618686

@stale stale bot removed the stale The stale bot thinks this issue is old. Add "pinned" label to prevent this from becoming stale. label Aug 11, 2021
@gandro gandro mentioned this issue Aug 11, 2021
Merged
gandro added a commit to gandro/cilium that referenced this issue Aug 12, 2021
@gandro
proxylib: Fix data race in unit test
56a2952 
This fixes a data race where we were accessing raw integers in unit
tests from two different go routines without using atomics.

Fixes: cilium#16315

Signed-off-by: Sebastian Wicki <sebastian@isovalent.com>
tklauser pushed a commit that referenced this issue Aug 19, 2021
@gandro @tklauser
proxylib: Fix data race in unit test
c7cc24f 
This fixes a data race where we were accessing raw integers in unit
tests from two different go routines without using atomics.

Fixes: #16315

Signed-off-by: Sebastian Wicki <sebastian@isovalent.com>
tklauser pushed a commit to tklauser/cilium that referenced this issue Aug 23, 2021
@gandro @tklauser
proxylib: Fix data race in unit test
5c6ef30 
[ upstream commit c7cc24f ]

This fixes a data race where we were accessing raw integers in unit
tests from two different go routines without using atomics.

Fixes: cilium#16315

Signed-off-by: Sebastian Wicki <sebastian@isovalent.com>
Signed-off-by: Tobias Klauser <tobias@cilium.io>
tklauser pushed a commit that referenced this issue Aug 24, 2021
@gandro @tklauser
proxylib: Fix data race in unit test
7168de1 
[ upstream commit c7cc24f ]

This fixes a data race where we were accessing raw integers in unit
tests from two different go routines without using atomics.

Fixes: #16315

Signed-off-by: Sebastian Wicki <sebastian@isovalent.com>
Signed-off-by: Tobias Klauser <tobias@cilium.io>
krishgobinath pushed a commit to krishgobinath/cilium that referenced this issue Oct 20, 2021
@gandro
proxylib: Fix data race in unit test
e2f03ae 
This fixes a data race where we were accessing raw integers in unit
tests from two different go routines without using atomics.

Fixes: cilium#16315

Signed-off-by: Sebastian Wicki <sebastian@isovalent.com>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Assignees
No one assigned
Labels
area/CI Continuous Integration testing issue or flake area/proxy Impacts proxy components, including DNS, Kafka, Envoy and/or XDS servers. ci/flake This is a known failure that occurs in the tree. Please investigate me! kind/bug/race-detector
Projects
None yet
Milestone
No milestone
Development

Successfully merging a pull request may close this issue.

proxylib: Fix data races in unit tests gandro/cilium
2 participants
@gandro @jibi