Test Ecosystem

Test Ecosystem Setup Guide

Value Proposition Certify complex agent interactions within a secure, fully isolated integration ecosystem. Safely simulate high-scale deployments, rigorously validate security boundaries, and maintain comprehensive operational oversight—ensuring your AI workflows are rigorously validated and production-ready before deployment. Read the full value proposition

This guide provides the blueprint to seamlessly deploy, manage, and troubleshoot a fully integrated, production-grade test ecosystem featuring InnoDB Cluster, MySQL Router, and ProxySQL. Empower your teams to rigorously validate AI agents in a safe, isolated environment, guaranteeing stability and performance before entering production.

Note

Note on Datadog: This environment natively includes datadog-unified. It runs alongside Prometheus and Grafana. This enables comprehensive metrics observability and APM.

---

1. 🚀 Quick Start: Safely Validate Idempotent Deployments

Ensure validation safety and accelerate your testing lifecycle with our fully automated teardown and deployment process. The scripts bootstrap Group Replication idempotently, allowing you to safely simulate production-grade environments and validate AI workflows in minutes.

cd test-server/infrastructure
node scripts/recreate-test-ecosystem.mjs

This master script will:

• Tear down the existing cluster and volumes (docker compose down -v).
• Start the fresh test containers (docker compose up -d).
• Automatically poll the nodes until they are healthy.
• Initialize the primary node (mysql-node1) as the cluster creator.
• Join mysql-node2 and mysql-node3 to the cluster.
• Output the final cluster topology.

---

2. 🏛️ Architect the Testing Topology

This test ecosystem includes only the necessary components to validate the mysql-mcp server:

┌─────────────────────────────────────────────────────────────────────────────┐
│                    Docker Network: infrastructure_default                    │
│                                                                              │
│  ┌──────────────┐  ┌──────────────┐  ┌──────────────┐                       │
│  │ mysql-node1  │  │ mysql-node2  │  │ mysql-node3  │                       │
│  │   PRIMARY    │  │  SECONDARY   │  │  SECONDARY   │                       │
│  │  Port: 3307  │  │  Port: 3308  │  │  Port: 3309  │                       │
│  └──────┬───────┘  └──────┬───────┘  └──────┬───────┘                       │
│         │                 │                 │                                │
│         └────────────┬────┴─────────────────┘                                │
│                      ▼                                                       │
│            ┌──────────────────┐           ┌──────────────┐  ┌──────────────┐ │
│            │   MySQL Router   │           │   ProxySQL   │  │    Redis     │ │
│            │ RW: 6446 RO: 6447, 6449 │           │ Admin: 6032  │  │  Port: 6379  │ │
│            │ REST API: 8443   │           │ Data:  6033  │  │              │ │
│            └──────────────────┴───────────┴──────────────┘  └──────────────┘ │
│                                                                              │
│            ┌──────────────────┐           ┌──────────────┐  ┌──────────────┐ │
│            │    Prometheus    │           │   Grafana    │  │   Adminer    │ │
│            │   Port: 9090     │           │  Port: 3001  │  │  Port: 8081  │ │
│            └──────────────────┘           └──────────────┘  └──────────────┘ │
│                                                                              │
│            ┌──────────────────┐           ┌──────────────┐                   │
│            │       Loki       │           │   Promtail   │                   │
│            │   Port: 3100     │           │  (Internal)  │                   │
│            └──────────────────┘           └──────────────┘                   │
│                                                                              │
│            ┌──────────────────┐           ┌──────────────┐                   │
│            │      Dozzle      │           │   Datadog    │                   │
│            │    Port: 8080    │           │  Port: 8126  │                   │
│            └──────────────────┘           └──────────────┘                   │
└─────────────────────────────────────────────────────────────────────────────┘

---

3. 🚑 Master Disaster Recovery Operations

🔄 Reset Data (Re-seed E2E Test DB)

Rapidly restore a pristine state. You may need to re-seed the testdb for E2E validation. This avoids the overhead of tearing down the entire cluster:

node scripts/reset-database.mjs

🛠️ Perform Manual Cluster Recovery

If containers are restarted and the cluster fails to auto-recover via SET PERSIST:

node scripts/reboot-cluster.mjs

This script executes dba.rebootClusterFromCompleteOutage().

---

4. 🔌 Analyze Network Configurations

🐬 MySQL (via InnoDB Cluster)

Note

The local test cluster uses non-standard ports (3307-3309). This differs from the default 3306 shown in standard configuration examples. This avoids conflicts with local installations.

Property	Value
Primary (R/W)	localhost:3307
Read Replicas (R/O)	localhost:3308, localhost:3309
User	root
Password	root

🚦 MySQL Router

Property	Value
REST API URL	https://localhost:8443
REST API User	rest_api
REST API Password	router_api
RW Routing Port	6446 (→ PRIMARY)
RO Routing Port	6447 (Classic), 6449 (X-Protocol) (→ SECONDARY)

📊 Observability & Tools

Property	URL
Datadog	Internal to Docker Network (8126). Use pup to sync JSON dashboards like AI Efficiency and Token & Tool Metrics (including the MySQL-MCP Audit Log widget).
Prometheus	http://localhost:9090
Loki	http://localhost:3100
Promtail	(network-internal only)
Grafana	http://localhost:3001 (admin/admin)
Dozzle (Logs)	http://localhost:8080
Adminer (DB UI)	http://localhost:8081 (Server: mysql-node1 (uses internal Docker network port 3306), User: root, Pass: root)
ProxySQL Admin	localhost:6032
ProxySQL Data	localhost:6033
Redis	localhost:6379

---

5. ⚠️ Ensure WSL2 Stability (Critical)

This environment runs on native docker-ce inside WSL2 Ubuntu (no Docker Desktop). WSL2 has a known failure mode. The distro terminates without active Windows-side client sessions. This kills Docker and all containers.

🛡️ Maintain the Keepalive Mechanism

A Windows Scheduled Task (WSL-KeepAlive) runs at user logon. It executes scripts/wsl-keepalive.vbs. This script launches wsl.exe -d <Distro-Name> --exec sleep infinity with a hidden window. This holds the distro alive indefinitely.

🩺 Diagnose Crashes

If containers are cycling (green → red → green repeatedly):

1. Check if WSL is rebooting: wsl bash -c "journalctl --list-boots". Multiple short-lived boots indicates a WSL termination issue.
2. Check for the smoking gun: Run wsl bash -c "dmesg | grep InitTerminateInstanceInternal". This message means WSL sent systemctl poweroff to the distro.
3. Check the keepalive task: Get-ScheduledTask -TaskName 'WSL-KeepAlive' | Select State — must be Running.
4. Check Docker daemon: wsl bash -c "systemctl status docker" — must be active (running).
5. Check iptables backend: Certain host kernel environments require iptables-nft. If Docker fails to start, check /etc/docker/daemon.json and update-alternatives --display iptables.

📄 Review Key Config Files

File	Purpose
%USERPROFILE%\.wslconfig	WSL2 VM config: memory, swap, vmIdleTimeout=-1
/etc/docker/daemon.json	Docker storage driver, log rotation
/etc/systemd/system/wsl-keepalive.service	Backup in-distro keepalive (defense-in-depth)
scripts/wsl-keepalive.vbs	Hidden launcher for the Windows Scheduled Task

---

🌐 Explore Related Topics

• MySQL Router - Connect and query through MySQL Router
• ProxySQL - ProxySQL integration
• MySQL Shell - Run MySQL Shell scripts