limit streams per tunnel connection

[kyessenov]

Envoy implementation is limited by the fact that internal listeners handle streams on the same worker as the tunnel listener. Therefore, we cannot handle a very large number of streams on a single tunnel.

We should ensure that we set a low max number of streams (~100 as a first estimate).

[hzxuzhonghu]

/not stale

[keithmattix]

Not stale

[GregHanson]

not stale

[kevinmingtarja]

Hi, I'm interested in picking this up if no one's working on it!

[bleggett]

So in istio/ztunnel#931

we added a pooling impl that considers streamcount-per-conn as criteria for starting a new connection, and exposes a ztunnel env var POOL_MAX_STREAMS_PER_CONNECTION to control that (which currently defaults to 100).

Our waypoint config is still hardcoded at 100 streams-per-conn though, which makes customizing POOL_MAX_STREAMS_PER_CONNECTION on the ztunnel side to a number higher than 100 a bad idea if you are using waypoints: https://github.com/istio/istio/blob/master/pilot/pkg/networking/core/listener_waypoint.go#L112

I think this will be mooted by sandwiching, however.