Avoid producer deadlock on connection closing

[merlimat]

Motivation

There's a condition in which a producer can remain deadlocked in the event of a connection failure.

The sequence goes like:

1. Producer (or multiple producers) have several outstanding request to write on a connection
2. The channel (used to pass buffers to write) has the buffer full and thus it's blocking
3. When the connection is closed the channel is not drained
4. The connection tries to notify the producers that it's time to reconnect
5. The producer go-routine is not able to process that notification, since it's blocked on the connection channel

[wolfstudy]

In here, Is it necessary for us to c.Lock() and c.Unlock() actions on an atomic operation?

[merlimat]

I think it's still needed to ensure that the mutex condition is broadcasted. To trigger a condition you need to have a lock on the associated mutex.

[wolfstudy]

I am wondering if there are other problems with this approach, because for an atomic primitive, we rarely see the operation of Lock and Unlock it.

atomic.StoreInt32(&c.state, int32(state))
c.Lock()
c.cond.Broadcast()
c.Unlock()

[merlimat]

There are 2 different usages here:

• On one hand we want other go routines to be able to check the state, without taking a lock
• On the other hand, we still need to maintain the atomic state update and notification

[wolfstudy]

yes, i agree with you, The main difference between us here is whether we need to lock to protect the atomic primitive, I think it is not needed, atomic itself is a synchronization primitive, so here, we can reduce the scope of the lock and only lock sync.cond. The code example is as follows:

atomic.StoreInt32(&c.state, int32(state))
c.Lock()
c.cond.Broadcast()
c.Unlock()

[wolfstudy]

Please ignore me

[wolfstudy]

LGTM +1