x/tools/go/analysis: request some form of static thread safety analysis for Go

[g-talbot]

In issue #24889 there's a short paragraph about static thread safety analysis that clang does for C++ programs:

clang supports thread safety analysis for C++ (https://clang.llvm.org/docs/ThreadSafetyAnalysis.html) based on annotations in the source code. This would clearly be applicable to Go code.

I'm pretty agnostic as to how some form of static thread safety analysis enters the Go ecosystem, but I would like to have it. I've used this at a previous employer (Google) to great effect, and IMO it would be of great benefit to Go programs as well.

Here's a strawman proposal, more to spur discussion than be a "real" solution. What if a struct is defined with a lock in it like so:

type MyStruct struct {
  sync.Mutex

  field1 int
  field2 string
  field3 []string
}

Let's annotate MyStruct to require locking on all of its members:

//go: guarded
type MyStruct struct {
  sync.RWMutex

  // by having //go: guarded above this implies that MyStruct must be locked
  // to access the members.  Reads of any of the fields would require RLock() and
  // writes would require Lock().
  field1 int
  field2 string
  field3 []string
}

So if I defined a function that accessed the members and didn't lock, compilation would fail:

func (m *MyStruct) GetField1() int {
  return m.field1 // does not compile
}

But this would work:

func (m *MyStruct) GetField1() int {
  m.RLock()
  defer m.RUnlock()
  return m.field1
}

And this too:

func (m *MyStruct) GetField1() int {
  m.RLock()
  defer m.RUnlock()
  return m.GetField1WithLock()
}

//go: with-rlock: m
func (m *MyStruct) GetField1WithRLock() int {
  return m.field1
}

I know folks don't love the comment pragmas, but I didn't have a better idea. Perhaps the basic gist of tying the lock to a specific struct that there could be some way that's "neater" and "more go like" than the complex clang annotations? With anonymous struct fields making for clean composition?

[prattmic]

The gVisor project has recently created checklocks, a go vet style analysis tool which fits in the vein of what you are suggesting here.

IIRC, the tool is fairly immature, so I don't believe it is full-featured and simplified in places for typical conventions used in gVisor, but I think it provides a good starting point, or potentially could be directly used by other projects.

cc @timothy-king @amscanne @hbhasker

[g-talbot]

The gVisor project has recently created checklocks, a go vet style analysis tool which fits in the vein of what you are suggesting here.

IIRC, the tool is fairly immature, so I don't believe it is full-featured and simplified in places for typical conventions used in gVisor, but I think it provides a good starting point, or potentially could be directly used by other projects.

cc @timothy-king @amscanne @hbhasker

This is pretty cool, though two things might be showstoppers for us using it right now.

1. Doesn't distinguish between sync.Mutex and sync.RWMutex.
2. Defer isn't supported. i.e. defer x.Unlock() which is pretty much required if you want panic to work in presence of locking.

Are there plans to address these? Also, one suggestion--could there be " // +checklocks: all-guarded-by: mu" that specifies that every field in a struct is guarded by the mutex?

[timothy-king]

My plan at the moment is to have an internship project look into lock annotations starting at the beginning of September. It will likely start from checklocks. We would also add support for more code patterns (defer, RWMutex, sync.Locker?). The goal of the project would be to evaluate the ergonomics of the checker in real code. I am also curious if we can mix struct tags and comments. If all goes well, this could turn into a proposal or an experimental tool.

No promises, but giving you full transparency that this is on my radar.

So if I defined a function that accessed the members and didn't lock, compilation would fail:

This would be a breaking changing. This would require a Go2 proposal. A static analyzer is a much smaller lift.

  type MyStruct struct {
  sync.RWMutex

Embeddings are definitely something that needs more thought. No concrete ideas from me at the moment though.

Are there plans to address these? Also, one suggestion--could there be " // +checklocks: all-guarded-by: mu" that specifies that every field in a struct is guarded by the mutex?

I don't really see why not. That would just be syntactic sugar. There is also a possibility of trying to capture the "mutex hat" idiom in an annotation. That is more flexible, but it is not clear if users would find this too brittle/confusing. Real code examples would really help here.

[bcmills]

Note that you can get a lot of the benefits of static annotation by switching from sync.Mutex to a 1-buffered channel:

type MyStruct struct {
	fields chan *myStructFields // 1-buffered
}

func NewMyStruct() *MyStruct {
	fields := make(chan *myStructFields, 1)
	fields <- &myStructFields{}
	return &MyStruct{fields: fields}
}

type myStructFields struct {
	field1 int
	field2 string
	field3 []string
}

Then the scope of the fields is more closely tied to the synchronization operations for those fields:

func (m *MyStruct) GetField1() int {
	return m.field1 // Does not compile — field1 is a member of myStructFields, not MyStruct.
}

func (m *MyStruct) GetField1() int {
	fields := <-m.fields                   // acquire
	defer func() { m.fields <- fields }()  // release
 	return fields.field1
}

[cagedmantis]

I added a package for this issue. Please feel free to change it from x/tools/go/analysis if it belongs somewhere else.