Limit size of outbound write buffer

[ajsutton]

Avoids excessive memory usage when a peer requests data faster than it can read it. There is probably a need to make the values used configurable and possibly vary them based on the stream state (ie multi stream negotiation should have little to no write buffer beyond what the OS provides, but things like Eth2 request for blocks might need more). Not entirely sure how that should fit into the API for jvm-libp2p though.

Changes made:

• Disable autoread when the channel become unwritable. This ensures we stop processing incoming requests when responses can't be read anyway.
• Adds a WriteTimeoutHandler to fail writes if the buffer remains full for 30 seconds. Auto-close then causes these connections to be closed.
• Limit size of write buffer to 1k instead of 64k. Ensures these limits apply quickly.

Prior to these changes a server handling a constant stream of ls multistream requests where the responses are not read, would use 5-6Gb of resident memory and all available CPU. Afterwards resident memory only grows by a few Mb, the CPU usage is negligible and the attacking connection is disconnected.

[Nashatyrev]

Interesting tweaks 👍
The only thing I'm concerned about is pushing all these tweaks by default, especially WriteTimeoutHandler. Let me think a bit on how to make this configurable

[ajsutton]

Yeah agreed - maybe they are defaults, though I'd want to spend more time making sure the number really do make sense, but they really do need to be configurable. I spent a bunch of time trying to find a way to just apply these settings within Teku without having to change jvm-libp2p but couldn't find a way to get hold of the netty classes. It does feel like having DOS protections like this fairly easy to apply would be useful.

May also be worth looking at what other libp2p implementations do out of the box for this kind of thing too.

[Nashatyrev]

Right now I'm thinking on adding some kind of 'advanced option': connectionPreHandler which would be invoked right after connection establishing. That would be the right place to add your tweaks on a client side.
That would also be the right place to add a 'firewall' either basic libp2p (which I'm planning to create) or any other custom one.
Does it sound reasonable?

[Nashatyrev]

@ajsutton what do you think on implementing this on Teku side: Consensys/teku#2478 ?
builder.getDebug().getBeforeSecureHandler() is probably not the most user-friendly place. I would adjust the Libp2p Builder later

[ajsutton]

Yep that sounds good to me.