Proxy monitors its own load/health and throttles connections when required

[alnikola]

Add self-diagnostics capability to YARP and implement throttling policies which can be applied if some thresholds have been exceeded.

Indicators to watch for:

• CPU load
• Memory pressure
• Available TCP ports
• Average/mean/p95 client requests latency

Examples of throttling policies:

• Stop accepting new HTTP 1.1 requests by immediately returning 503
• Stop accepting new active streams on HTTP/2 connections
• Stop accepting new TCP connections

[davidni]

I'd advise caution for any solution that might have the explicit or implicit assumption that the current process is the only resource-intensive process running on the machine and / or that you can accurately determine being CPU limited. E.g. an instance of YARP running with a job object limiting cpu may be CPU-limited while the machine may have plenty of capacity to spare.