I/O performance issue when subscribing to sync committee

[twoeths]

Describe the bug

A lot of attestations were submitted late from Jun 29 to Jul 01

This happened when the node subscribed to sync committee topic (notice that job wait time was too big for sync committee messages)

topic peers spiked because a lot of nodes with big long lived subnets connected to us at that time

there are more messages sent to us

and we have to published to too many peers also

this caused I/O lag issue (see #5740)

node received too many PRUNE messages

mesh peers were dropped (including core topics)

this caused gossip blocks being late, node had to process a lot of attestations at the same time which push pressure to our BLS worker pool (see #5739)

Expected behavior

No I/O performance issue, network thread (useWorker=true) could resolve this

Steps to reproduce

No response

Additional context

No response

Operating system

Linux

Lodestar version or commit hash

v1.8.0

[twoeths]

When the issue happens, actually most of the peers are outbound peers

This is when we have sync committee duty

But most of the time we already had enough subnet peers

When we have enough subnet peers, we should not dial peers