Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

Data race with protoparse.Parser.ParseFiles #236

Closed
bufdev opened this issue Jul 17, 2019 · 1 comment
Closed

Data race with protoparse.Parser.ParseFiles #236

bufdev opened this issue Jul 17, 2019 · 1 comment

Comments

@bufdev
Copy link
Contributor

bufdev commented Jul 17, 2019

There's some data race with protoparse.Parser.ParseFiles:

Note: I'm using two completely separately instantiated protoparse.Parsers, ie I'm not even using the same protoparse.Parser across threads, so it has to be something global.

Here's two traces. Both of these were done by compiling https://github.com/googleapis/googleapis on a per-directory basis, that is I call ParseFiles for the files in each directory using a separate protoparse.Parser on a separate thread.

Thoughts:

  • Maybe something to do with the globals, I don't know what but it has to do with modifying *DescriptorProto objects so maybe one of the caches? Assumedly this concerns imports since I'm not calling ParseFiles directly on the same file at any time.
  • Maybe something to do with the well-known types?
WARNING: DATA RACE
Read at 0x00c0077de7b0 by goroutine 151:
  github.com/jhump/protoreflect/desc.(*FieldDescriptor).resolve()
      /go/pkg/mod/github.com/golang/protobuf@v1.3.2/protoc-gen-go/descriptor/descriptor.pb.go:887 +0xb29
  github.com/jhump/protoreflect/desc.(*MessageDescriptor).resolve()
      /go/pkg/mod/github.com/jhump/protoreflect@v1.4.5-0.20190717021903-efa672cbb5ff/desc/descriptor.go:326 +0x73b
  github.com/jhump/protoreflect/desc.(*MessageDescriptor).resolve()
      /go/pkg/mod/github.com/jhump/protoreflect@v1.4.5-0.20190717021903-efa672cbb5ff/desc/descriptor.go:316 +0x38b
  github.com/jhump/protoreflect/desc.createFileDescriptor()
      /go/pkg/mod/github.com/jhump/protoreflect@v1.4.5-0.20190717021903-efa672cbb5ff/desc/convert.go:88 +0x1c3b
  github.com/jhump/protoreflect/desc/protoparse.(*linker).linkFile()
      /go/pkg/mod/github.com/jhump/protoreflect@v1.4.5-0.20190717021903-efa672cbb5ff/desc/convert.go:19 +0x647
  github.com/jhump/protoreflect/desc/protoparse.(*linker).linkFile()
      /go/pkg/mod/github.com/jhump/protoreflect@v1.4.5-0.20190717021903-efa672cbb5ff/desc/protoparse/linker.go:640 +0x514
  github.com/jhump/protoreflect/desc/protoparse.(*linker).createdLinkedDescriptors()
      /go/pkg/mod/github.com/jhump/protoreflect@v1.4.5-0.20190717021903-efa672cbb5ff/desc/protoparse/linker.go:602 +0x34e
  github.com/jhump/protoreflect/desc/protoparse.(*linker).linkFiles()
      /go/pkg/mod/github.com/jhump/protoreflect@v1.4.5-0.20190717021903-efa672cbb5ff/desc/protoparse/linker.go:49 +0x98
  github.com/jhump/protoreflect/desc/protoparse.Parser.ParseFiles()
      /go/pkg/mod/github.com/jhump/protoreflect@v1.4.5-0.20190717021903-efa672cbb5ff/desc/protoparse/parser.go:183 +0x17d

Previous write at 0x00c0077de7b0 by goroutine 53:
  github.com/jhump/protoreflect/desc/protoparse.(*linker).resolveFieldTypes()
      /go/pkg/mod/github.com/jhump/protoreflect@v1.4.5-0.20190717021903-efa672cbb5ff/desc/protoparse/linker.go:381 +0xed5
  github.com/jhump/protoreflect/desc/protoparse.(*linker).resolveMessageTypes()
      /go/pkg/mod/github.com/jhump/protoreflect@v1.4.5-0.20190717021903-efa672cbb5ff/desc/protoparse/linker.go:303 +0x651
  github.com/jhump/protoreflect/desc/protoparse.(*linker).resolveMessageTypes()
      /go/pkg/mod/github.com/jhump/protoreflect@v1.4.5-0.20190717021903-efa672cbb5ff/desc/protoparse/linker.go:293 +0x3b1
  github.com/jhump/protoreflect/desc/protoparse.(*linker).resolveReferences()
      /go/pkg/mod/github.com/jhump/protoreflect@v1.4.5-0.20190717021903-efa672cbb5ff/desc/protoparse/linker.go:239 +0x1af
  github.com/jhump/protoreflect/desc/protoparse.(*linker).linkFiles()
      /go/pkg/mod/github.com/jhump/protoreflect@v1.4.5-0.20190717021903-efa672cbb5ff/desc/protoparse/linker.go:40 +0x74
  github.com/jhump/protoreflect/desc/protoparse.Parser.ParseFiles()
      /go/pkg/mod/github.com/jhump/protoreflect@v1.4.5-0.20190717021903-efa672cbb5ff/desc/protoparse/parser.go:183 +0x17d

WARNING: DATA RACE
Read at 0x00c00776e4e0 by goroutine 50:
  github.com/jhump/protoreflect/desc/protoparse.(*linker).resolveFieldTypes()
      /go/pkg/mod/github.com/golang/protobuf@v1.3.2/protoc-gen-go/descriptor/descriptor.pb.go:887 +0x157b
  github.com/jhump/protoreflect/desc/protoparse.(*linker).resolveMessageTypes()
      /go/pkg/mod/github.com/jhump/protoreflect@v1.4.5-0.20190717021903-efa672cbb5ff/desc/protoparse/linker.go:303 +0x651
  github.com/jhump/protoreflect/desc/protoparse.(*linker).resolveMessageTypes()
      /go/pkg/mod/github.com/jhump/protoreflect@v1.4.5-0.20190717021903-efa672cbb5ff/desc/protoparse/linker.go:293 +0x3b1
  github.com/jhump/protoreflect/desc/protoparse.(*linker).resolveReferences()
      /go/pkg/mod/github.com/jhump/protoreflect@v1.4.5-0.20190717021903-efa672cbb5ff/desc/protoparse/linker.go:239 +0x1af
  github.com/jhump/protoreflect/desc/protoparse.(*linker).linkFiles()
      /go/pkg/mod/github.com/jhump/protoreflect@v1.4.5-0.20190717021903-efa672cbb5ff/desc/protoparse/linker.go:40 +0x74
  github.com/jhump/protoreflect/desc/protoparse.Parser.ParseFiles()
      /go/pkg/mod/github.com/jhump/protoreflect@v1.4.5-0.20190717021903-efa672cbb5ff/desc/protoparse/parser.go:183 +0x17d

Previous write at 0x00c00776e4e0 by goroutine 73:
  github.com/jhump/protoreflect/desc/protoparse.(*linker).resolveFieldTypes()
      /go/pkg/mod/github.com/jhump/protoreflect@v1.4.5-0.20190717021903-efa672cbb5ff/desc/protoparse/linker.go:381 +0xed5
  github.com/jhump/protoreflect/desc/protoparse.(*linker).resolveMessageTypes()
      /go/pkg/mod/github.com/jhump/protoreflect@v1.4.5-0.20190717021903-efa672cbb5ff/desc/protoparse/linker.go:303 +0x651
  github.com/jhump/protoreflect/desc/protoparse.(*linker).resolveMessageTypes()
      /go/pkg/mod/github.com/jhump/protoreflect@v1.4.5-0.20190717021903-efa672cbb5ff/desc/protoparse/linker.go:293 +0x3b1
  github.com/jhump/protoreflect/desc/protoparse.(*linker).resolveReferences()
      /go/pkg/mod/github.com/jhump/protoreflect@v1.4.5-0.20190717021903-efa672cbb5ff/desc/protoparse/linker.go:239 +0x1af
  github.com/jhump/protoreflect/desc/protoparse.(*linker).linkFiles()
      /go/pkg/mod/github.com/jhump/protoreflect@v1.4.5-0.20190717021903-efa672cbb5ff/desc/protoparse/linker.go:40 +0x74
  github.com/jhump/protoreflect/desc/protoparse.Parser.ParseFiles()
      /go/pkg/mod/github.com/jhump/protoreflect@v1.4.5-0.20190717021903-efa672cbb5ff/desc/protoparse/parser.go:183 +0x17d
This was referenced Jul 17, 2019
Closed
Closed
@bufdev
Copy link
Contributor Author

bufdev commented Jul 18, 2019

#238 fixes this

@bufdev bufdev closed this as completed Jul 18, 2019
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Assignees
No one assigned
Labels
None yet
Projects
None yet
Milestone
No milestone
1 participant
@bufdev