[RFC] limiter: smarter perPeerLimit #81
Conversation
ghost
assigned 
magik6k
changed the title
|
(Will work on fixing tests once the idea is approved) |
| @@ -91,27 +97,41 @@ func (dl *dialLimiter) freeFDToken() { | |||
| } | |||
| } | |||
|
|
|||
| func (dl *dialLimiter) updateWaitingPerPeer() { | |||
| for p := range dl.activePerPeer { | |||
There was a problem hiding this comment.
These two loops are going to be very CPU heavy.
|
So, this looks like it decreases per-peer dials as we run out of file descriptors? We should make sure this doesn't lead to a situation where everything is slow and therefore none of the dials really finish. |
bigs
added
need/community-input
|
i think this change should hold off until we get a better idea of where we're getting bogged down when dialing DHT nodes |
|
This idea is interesting. Choking the peer limit as the FD allowance runs out. However, I worry that this will bring unfairness to the system. Also won't work well with the address pollution problem of the DHT (e.g. we have peers with 200 addresses), unless we implement an address scoring system. Right now we're pretty dumb and dial addresses in the order returned by the peerstore, but if we can prioritise certain addresses, even in a choked scenario we could end up dialling successfully. FYI @magik6k – we traced down the source of many woes to the DHT tripping over the FD limit in the swarm, and we are now pacing dials dynamically: libp2p/go-libp2p-kad-dht#237. Are you still interested in discussing ideas around this? |
This makes the
limiter.perPeerLimitdepend onfdConsuming.graph of
perPeerLimitvsfdConsuming: