proxy: reintroduce dynamic limiter for compute lock

[conradludgate]

Problem

Computes that are healthy can manage many connection attempts at a time. Unhealthy computes cannot. We initially handled this with a fixed concurrency limit, but it seems this inhibits pgbench.

Summary of changes

Support AIMD for connect_to_compute lock to allow varying the concurrency limit based on compute health

Checklist before requesting a review

• I have performed a self-review of my code.
• If it is a core feature, I have added thorough tests.
• Do we need to implement analytics? if so did you add the relevant metrics to the dashboard?
• If this PR requires public announcement, mark it with /release-notes label and add several sentences in this section.

Checklist before merging

• Do not forget to reformat commit message to not include the above checklist

[conradludgate]

I took our original AIMD impl but rewrote it considerably to be simpler. It should still work the same

[github-actions]

3168 tests run: 3034 passed, 1 failed, 133 skipped (full report)

---

Failures on Postgres 15

• test_sharding_split_failures[failure19]: debug

# Run all failed tests locally:
scripts/pytest -vv -n $(nproc) -k "test_sharding_split_failures[debug-pg15-failure19]"

Flaky tests (2)

Postgres 15

• test_vm_bit_clear_on_heap_lock: debug

Postgres 14

• test_pageserver_restarts_under_worload: release

Code coverage* (full report)

• functions: 31.4% (6490 of 20663 functions)
• lines: 48.4% (50217 of 103724 lines)

* collected from Rust tests only

---

_{The comment gets automatically updated with the latest test results
e4d62ca at 2024-05-29T09:00:16.286Z :recycle:}

[conradludgate]

Tested it in staging, seems to work fine. Will follow up with a PR to change the defaults in staging as appropriate