Skip to content

Loadtest: make MySQL max open conns tunable (#44802)#48472

Merged
getvictor merged 3 commits into
mainfrom
victor/44802-mysql-conns-tunable
Jul 4, 2026
Merged

Loadtest: make MySQL max open conns tunable (#44802)#48472
getvictor merged 3 commits into
mainfrom
victor/44802-mysql-conns-tunable

Conversation

@getvictor

@getvictor getvictor commented Jun 30, 2026

Copy link
Copy Markdown
Member

Replace the hardcoded FLEET_MYSQL_MAX_OPEN_CONNS=10 in the loadtest terraform (both the root ecs.tf stack and the infra/ stack) with a mysql_max_open_conns variable, applied to both the writer and read-replica pools.

Also document per-tier connection pool sizing in the AWS reference architecture: 20 for R-class instances (25k+ hosts), 10 for the smaller T-class instances (5k/10k hosts) whose low default max_connections (~90/135) leaves little headroom.

Related issue: Resolves #44802

Summary by CodeRabbit

  • New Features
    • Added a configurable setting for MySQL connection limits in the load testing environment.
    • Connection settings now use a shared input value instead of fixed defaults, making it easier to tune database load behavior across container types.

Replace the hardcoded FLEET_MYSQL_MAX_OPEN_CONNS=10 in the loadtest
terraform (both the root ecs.tf stack and the infra/ stack) with a
mysql_max_open_conns variable defaulting to 20, applied to both the
writer and read-replica pools. The loadtest runs on R-class Aurora
instances with ample connection headroom, so 20 is safe and reduces
enrollment connection-pool contention.

Also document per-tier connection pool sizing in the AWS reference
architecture: 20 for R-class instances (25k+ hosts), 10 for the smaller
T-class instances (5k/10k hosts) whose low default max_connections
(~90/135) leaves little headroom.
…ative default

Address review feedback on #44802: drop the explanatory connection-pool
section from the reference architecture in favor of a per-tier
FLEET_MYSQL_MAX_OPEN_CONNS column (10 for T-class 5k/10k tiers, 20 for
R-class 25k+ tiers), trim the terraform variable descriptions, and keep
the loadtest default at the conservative 10 (override per run).
@getvictor getvictor changed the title Loadtest: make MySQL max open conns tunable, default 20 (#44802) Loadtest: make MySQL max open conns tunable (#44802) Jun 30, 2026
@getvictor getvictor marked this pull request as ready for review June 30, 2026 09:44
Copilot AI review requested due to automatic review settings June 30, 2026 09:44

@claude claude Bot left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Claude Code Review

This repository is configured for manual code reviews. Comment @claude review to trigger a review and subscribe this PR to future pushes, or @claude review once for a one-time review.

Tip: disable this comment in your organization's Code Review settings.

Copilot AI left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Warning

  • Copilot's review of this pull request may be incomplete because some of the changed files are excluded by your Copilot content exclusion settings. See Excluding content from Copilot for details.

Pull request overview

Updates the loadtesting Terraform stacks to make Fleet’s MySQL connection pool size configurable instead of hardcoded, so operators can tune per-container DB connections based on Aurora capacity and loadtesting scale.

Changes:

  • Adds a mysql_max_open_conns Terraform variable (default 10) to both loadtesting Terraform stacks.
  • Wires mysql_max_open_conns into both writer and read-replica Fleet MySQL pool env vars (FLEET_MYSQL_MAX_OPEN_CONNS and FLEET_MYSQL_READ_REPLICA_MAX_OPEN_CONNS).
  • Updates AWS reference architecture documentation (content excluded by policy).

Reviewed changes

Copilot reviewed 4 out of 5 changed files in this pull request and generated 2 comments.

Show a summary per file
File Description
infrastructure/loadtesting/terraform/variables.tf Adds mysql_max_open_conns variable for the root loadtesting stack.
infrastructure/loadtesting/terraform/infra/variables.tf Adds mysql_max_open_conns variable for the infra/ loadtesting stack.
infrastructure/loadtesting/terraform/infra/locals.tf Applies the variable to both writer and read-replica pool env vars.
infrastructure/loadtesting/terraform/ecs.tf Applies the variable to ECS task env vars for both writer and read-replica pools.
docs/Deploy/Reference-Architectures.md Updates connection pool sizing guidance (content excluded).
Files excluded by content exclusion policy (1)
  • docs/Deploy/Reference-Architectures.md

💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.

Comment thread infrastructure/loadtesting/terraform/variables.tf
Comment thread infrastructure/loadtesting/terraform/infra/variables.tf
@coderabbitai

coderabbitai Bot commented Jun 30, 2026

Copy link
Copy Markdown
Contributor

Review Change Stack

Important

Review skipped

Auto reviews are limited based on label configuration.

🏷️ Required labels (at least one) (1)
  • :ai

Please check the settings in the CodeRabbit UI or the .coderabbit.yaml file in this repository. To trigger a single review, invoke the @coderabbitai review command.

⚙️ Run configuration

Configuration used: Repository UI

Review profile: CHILL

Plan: Pro

Run ID: 1875de04-65ff-4d41-8f27-cab3ab364144

You can disable this status message by setting the reviews.review_status to false in the CodeRabbit configuration file.

Use the checkbox below for a quick retry:

  • 🔍 Trigger review
📝 Walkthrough

Walkthrough

A new Terraform input variable mysql_max_open_conns (type number, default 10) is declared in both infrastructure/loadtesting/terraform/variables.tf and infrastructure/loadtesting/terraform/infra/variables.tf. The previously hardcoded "10" values for FLEET_MYSQL_MAX_OPEN_CONNS and FLEET_MYSQL_READ_REPLICA_MAX_OPEN_CONNS in ecs.tf and infra/locals.tf are replaced with tostring(var.mysql_max_open_conns).

🚥 Pre-merge checks | ✅ 3 | ❌ 2

❌ Failed checks (1 warning, 1 inconclusive)

Check name Status Explanation Resolution
Description check ⚠️ Warning The description is missing most required template sections and checklist items, so it is largely incomplete. Fill out the required template sections, especially the checklist and testing items, or remove non-applicable lines.
Linked Issues check ❓ Inconclusive The Terraform changes satisfy the variable update, but the docs update in docs/Deploy/Reference-Architectures.md is excluded by !**/*.md, so that objective can't be verified. Re-run review with the markdown file included or confirm the reference-architecture update separately.
✅ Passed checks (3 passed)
Check name Status Explanation
Title check ✅ Passed The title concisely states the main change: making loadtest MySQL connection limits tunable.
Out of Scope Changes check ✅ Passed No clearly unrelated code changes are evident; the edits all support tuning loadtest MySQL connection limits.
Docstring Coverage ✅ Passed No functions found in the changed files to evaluate docstring coverage. Skipping docstring coverage check.
✨ Finishing Touches
🧪 Generate unit tests (beta)
  • Create PR with unit tests
  • Commit unit tests in branch victor/44802-mysql-conns-tunable

Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out.

❤️ Share

Comment @coderabbitai help to get the list of available commands.

@coderabbitai coderabbitai Bot left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Actionable comments posted: 2

🤖 Prompt for all review comments with AI agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.

Inline comments:
In `@infrastructure/loadtesting/terraform/infra/variables.tf`:
- Around line 46-50: The `mysql_max_open_conns` variable description is using
the wrong sizing term and an outdated formula. Update the description in
`variables.tf` to reference `fleet_task_count` instead of `fleet_containers`,
and make it clear that this setting applies to both the writer and read-replica
pools so the worst-case connection math reflects the total impact. Keep the
change localized to the `variable "mysql_max_open_conns"` block and adjust the
wording to match the stack’s actual budget.

In `@infrastructure/loadtesting/terraform/variables.tf`:
- Around line 35-39: Update the mysql_max_open_conns variable description to
reflect the correct worst-case connection budget: it should mention that this
setting applies to both the writer and read-replica pools, and that a single
Aurora instance can see roughly 2 * fleet_containers * mysql_max_open_conns when
reader traffic collapses onto the writer. Keep the change limited to the
variable description in variables.tf so the wording matches the actual
connection behavior.
🪄 Autofix (Beta)

Fix all unresolved CodeRabbit comments on this PR:

  • Push a commit to this branch (recommended)
  • Create a new PR with the fixes

ℹ️ Review info
⚙️ Run configuration

Configuration used: Path: .coderabbit.yaml

Review profile: CHILL

Plan: Pro

Run ID: 7c095ea4-e2b9-432d-8023-735de43b1014

📥 Commits

Reviewing files that changed from the base of the PR and between 21c3a3f and e79934d.

⛔ Files ignored due to path filters (1)
  • docs/Deploy/Reference-Architectures.md is excluded by !**/*.md
📒 Files selected for processing (4)
  • infrastructure/loadtesting/terraform/ecs.tf
  • infrastructure/loadtesting/terraform/infra/locals.tf
  • infrastructure/loadtesting/terraform/infra/variables.tf
  • infrastructure/loadtesting/terraform/variables.tf

Comment thread infrastructure/loadtesting/terraform/infra/variables.tf
Comment thread infrastructure/loadtesting/terraform/variables.tf
@getvictor

Copy link
Copy Markdown
Member Author

@rachaelshaw Please approve the doc part of this PR

@getvictor getvictor merged commit 426de4e into main Jul 4, 2026
12 of 13 checks passed
@getvictor getvictor deleted the victor/44802-mysql-conns-tunable branch July 4, 2026 05:48
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

MySQL connection pool tuning for loadtest environment

4 participants