Join GitHub today
GitHub is home to over 50 million developers working together to host and review code, manage projects, and build software together.Sign up
crypto/rand: mutex around /dev/urandom is unnecessarily broad #9221
(Possibly related to #7000)
We were trying to read concurrently (on Linux, go 1.3) from the Reader provided by crypto/rand, but were seeing a lot of unexpected contention. Taking a look at the implementation, it appears to take a mutex around the entire call just in order to protect the initialization step: https://github.com/golang/go/blob/master/src/crypto/rand/rand_unix.go#L51
This seems unnecessarily broad, since the underlying
I looked into this more as I was fixing #9205. The problem is that we use a bufio.Reader around the *os.File reading from /dev/urandom, and it's for the bufio.Reader that the mutex is important.
If we were to fix the mutex contention issue for random, we'd probably want to do a per-processor buffer of /dev/urandom data that we tried to read from first, only falling back to re-filling the per-processor buffer from a shared (without locking) *os.File that we then do ~large reads occasionally. Then we wouldn't need bufio.Reader at all. We could use sync.Pool for the per-P buffer, but using that is contentious.
Except on Plan9 for some reason?
I'm not sure why the standard library should be buffering here at all, honestly. I understand that it will aid performance in the non-concurrent case, but my first instinct would be to let the caller do the buffering (if they so desire) and by default to just do an unlocked unbuffered Read on the underlying fd.
Most threading libraries (pthreads etc) provide a thread-local storage API that doesn't content; since go does it's own scheduler I imagine (pure speculation here) that you pin threads to processors at the OS level to avoid scheduler fights, so you could cheat and use thread-local storage to get your per-P buffers? If I'm wildly off-base feel free to ignore this :)