Backport connection stream counter to v1.17

[lijunwangs]

Problem

There are two issues in v1.17 which are bugs in the QOS accounting

1. The stream counts are not counted for a stake across all connections, causing over allocation of stream bandwidth.
2. The stream counts are not counted for the unstaked node across all connections -- incentivizing the unstaked to spam connections.

In master this is well encapsulated in ConnectionStreamCounter which track the stream counts across connections for a key (pubkey or IpAddr).

Summary of Changes

Back port ConnectionStreamCounter, following 3 issues are solved:

1. Over allocations of PPS for staked nodes.
2. Over allocations of PPS for unstaked nodes.
3. Backport of sleep instead of drop when stream rate exceeded limit; #939: use sleep instead of drop when throttling streams.

Fixes #

[t-nelson]

can this be done without the refactor? this is stable branch. we need the absolutely most concise effective diff

[lijunwangs]

This is not exactly a refactoring -- The ConnectionStreamCounter is needed to do the counting across connections. Even though I understand the desire to keep changes minimum, I think do special handling for the backport itself to keep it small is bad practice as if there are two different implementations. The divergence makes maintaining these two branches very difficult and cause it easier to slip in new bugs.

[t-nelson]

this file was not introduced by either of the PRs that this one is backporting. no tests? straight to mb?

you can try to get @CriesofCarrots or @sakridge to sign off

[codecov-commenter]

Codecov Report

Attention: Patch coverage is 70.58824% with 15 lines in your changes are missing coverage. Please review.

Project coverage is 81.6%. Comparing base (4f9c393) to head (c0ef443).
Report is 1 commits behind head on v1.17.

Additional details and impacted files

@@            Coverage Diff            @@
##            v1.17     #991     +/-   ##
=========================================
- Coverage    81.6%    81.6%   -0.1%     
=========================================
  Files         806      807      +1     
  Lines      219348   219370     +22     
=========================================
+ Hits       179078   179093     +15     
- Misses      40270    40277      +7     

[sakridge]

I think I'm ok with the refactor. I would just like to see the testing it has gone through. unit test which stresses the streams, maybe creating connections which might get dropped and then recreated so maybe we could have a bunch of tasks pending while we are sleeping? And then also time on testnet and nodes experiencing mainnet traffic and checking of the counter values to see that it's working as expected.

[lijunwangs]

I have the jito-relayers one with latest v1.17 and the one with v1.17 + the changes in this PR. You can see the throttled streams with this change is much more stable compared to that without it -- indicating less thrashing in the system.

[lijunwangs]

The changed has been running on 3gG. packets sent down and streams throttled.

[lijunwangs]

Regarding sleep vs drop -- I do not think the change makes it any worse in worst case scenario. The worst case is bounded by the max_uni_streams per connections -- the maximum concurrent streams can be in open state for a connection. Even without the sleep, a malicious client can drag on feeding the data for the stream to simulate the scenario of keeping the stream open for "long" time. In regard of opening new connections while in sleep, it also a not net new change. While we are sleeping, we hold reference to the connection object -- so even the client close it, on the server side the connection is still counted in the connection cache table and is limited by how many concurrent connections per IP/pubkey. Regarding the just pure QUIC connection spamming, we already have the issue even without this change. It has to be addressed by early filtering from Quinn enhancement. So I do not think it make it any worse.