memory, config: reject connections when memory usage is high (#1120)

[ti-chi-bot]

This is an automated cherry-pick of #1120

What problem does this PR solve?

Issue Number: close #1119

Problem Summary:
TiProxy may OOM when the connection count is too large

What is changed and how it works:

• Reject new connections when memory usage is too high
• Report unhealthy on the status port when the memory usage is too high

Check List

Tests

• Unit test
• Integration test
• Manual test (add detailed scripts or steps below)
• No code

The memory usage and connection count increased to a fixed value:

The server shows connections are rejected:

The client failed to connect anymore:

After restarting the client, the client can connect again

Notable changes

• Has configuration change
• Has HTTP API interfaces change
• Has tiproxyctl change
• Other user behavior changes

Release note

Please refer to Release Notes Language Style Guide to write a quality release note.

- Reject connections and report unhealthy when the memory usage is too high

[ti-chi-bot]

[APPROVALNOTIFIER] This PR is APPROVED

This pull-request has been approved by: djshow832

The full list of commands accepted by this bot can be found here.

The pull request process is described here

Details

Needs approval from an approver in each of these files:

• OWNERS [djshow832]

Approvers can indicate their approval by writing /approve in a comment
Approvers can cancel approval by writing /approve cancel in a comment

[ti-chi-bot]

[LGTM Timeline notifier]

Timeline:

• 2026-04-21 08:20:42.563456841 +0000 UTC m=+2067647.768816899: ☑️ agreed by djshow832.

[codecov-commenter]

Codecov Report

❌ Patch coverage is 77.24138% with 33 lines in your changes missing coverage. Please review.
⚠️ Please upload report for BASE (release-nextgen-202603@d34a518). Learn more about missing BASE report.

Files with missing lines	Patch %	Lines
pkg/manager/memory/memory.go	67.77%	19 Missing and 10 partials ⚠️
pkg/proxy/net/packetio.go	0.00%	2 Missing and 1 partial ⚠️
pkg/proxy/proxy.go	97.67%	1 Missing ⚠️

Additional details and impacted files

@@                    Coverage Diff                    @@
##             release-nextgen-202603    #1135   +/-   ##
=========================================================
  Coverage                          ?   67.37%           
=========================================================
  Files                             ?      149           
  Lines                             ?    16106           
  Branches                          ?        0           
=========================================================
  Hits                              ?    10852           
  Misses                            ?     4516           
  Partials                          ?      738           

Flag	Coverage Δ
unit	67.37% <77.24%> (?)

Flags with carried forward coverage won't be shown. Click here to find out more.

☔ View full report in Codecov by Sentry.
📢 Have feedback on the report? Share it here.

🚀 New features to boost your workflow:

• ❄️ Test Analytics: Detect flaky tests, report on failures, and find test suite problems.