why use 'Lock' in "get_batch()" while using queue to communication between threads

[huzhoudaxia]

The codes is in douzero.dmc.utils, line 5 to 52. I am confused about the usage of lock. To my knowledge, queue module can achieve thread synchronization, then why use Lock mechanism again.
I don't know whether my question is clearly. Hope it's worthy.
Yours sincerely.

[daochenzha]

@huzhoudaxia Very good question. Basically, the get_batch is getting a batch of samples instead of one sample. After getting all the samples, it will send the signals to the free_queue. Since there are multiple threads, there could be some weird cases. Let me give you an example:

Suppose we use 4 threads, the batch size is 32, and the number of buffers is 50. The four threads all request 32 samples (and they will only send free signals after getting 32 samples). Suppose at some time step, they get, 12, 12, 12, and 14 samples, respectively. At this time step, the program will get stuck since the buffers have not been freed.

Thus, we add a lock here to make getting 32 samples atomic.

[huzhoudaxia]

@daochenzha Thank you for your time. Your answer is very helpful.

[yffbit]

similar to the Dining Philosophers Problem