lib/connections: More fine grained locking (fixes #3066)

[calmh]

Purpose

This fixes the deadlock by reducing where we hold the various locks. To
start with it splits up the existing "mut" into a "listenersMut" and a
"curConMut" as these are the two things being protected and I can see no
relation between them that requires a shared lock. It also moves all
model calls outside of the lock, as I see no reason to hold the lock
while calling the model (and it's risky, as proven).

Testing

Things connect, now.

[AudriusButkevicius]

So this potentially brings back the connected to already connected issue, if we don't lock around AddConnection and ConnectedTo

[calmh]

How does that happen, again?

[AudriusButkevicius]

T1 in redial checks connectionType map, no value
T2 does model.AddConnection in handle
T2 sets connectionType map
T1 checks model.ConnectedTo and prints message

[calmh]

So if one connection comes in at the same time as we're dialing, one will get hit by "already connected" which is fine? We could reduce the window a bit though as connect() grabs ct quite early and the looks at it much later. There is also some optimizations to be done I think, as we could skip devices that are already connected with the maximum dialer preference and so. We keep doing discovery for devices as well, even though we are already connected...

[AudriusButkevicius]

I am as always on my phone so hard to have a proper discussion, though what you are saying sounds like true.

I think the caching discovery mux absorbs the frequent lookups?

[calmh]

Some of them, but I think it still lets through a global discovery query per five or ten minutes or something. But that's an easy optimization for another commit. :)

[AudriusButkevicius]

So whats the plan? Are we pushing this and following it up or whats the plan.

[AudriusButkevicius]

@st-review merge

[calmh]

From my POV yes as it fixes the problem I see, and I don't think I see the other problem if it's here. :)

[st-review]

👌 Merged as 31f6418. Thanks, @calmh!

[AudriusButkevicius]

Well the other problem is definately there, as thats what I tried to fix by using wider scope for the locks.

[calmh]

I remember that issue being about us not switching connections properly. I'll see if I can find it later.