Bug Report: could not get table primary key info

[rodolphe-code]

Overview of the Issue

Schema Engine fails to initialize with "Row count exceeded 10000" for large databases

When initializing an unmanaged vttablet against a MySQL database with approximately 8000 tables, the Schema Engine fails to start due to a hard-coded limit of 10000 rows in the schema loading process.

Reproduction Steps

Environment

• Vitess Version: 21.0.3
• MySQL Version: 8.0
• Database Size: ~150GB
• Number of Tables: ~8000

Steps to Reproduce

1. Set up an unmanaged vttablet pointing to an existing MySQL database
2. Configure vttablet with the following settings:

vttablet \
  $TOPOLOGY_FLAGS \
  --grpc_max_message_size 209715200 \
  --grpc_initial_window_size 209715200 \
  --grpc_initial_conn_window_size 209715200 \
  --queryserver-config-max-result-size 1000000 \
  --queryserver-config-pool-size 100 \
  --queryserver-config-transaction-cap 100 \
  --queryserver-config-stream-pool-size 500 \
  --queryserver-config-schema-reload-time 1m \
  --queryserver-config-idle-timeout 1h \
  --queryserver-config-query-timeout 300s \
  --log_dir $VTDATAROOT/tmp \
  --log_queries_to_file $VTDATAROOT/tmp/$tablet_logfile \
  --tablet-path $alias \
  --init_keyspace legacy \
  --init_shard 0 \
  --init_tablet_type replica \
  --enable_replication_reporter \
  --port 15200 \
  --grpc_port 16200 \
  --service_map 'grpc-queryservice,grpc-tabletmanager,grpc-updatestream' \
  --pid_file $VTDATAROOT/$tablet_dir/vttablet.pid \
  --heartbeat_on_demand_duration=5s \
  --db_host 127.0.0.1 \
  --db_port 3306 \
  --db_app_user vitess \
  --db_app_password "passpass" \
  --db_dba_user vitess \
  --db_dba_password "passpass" \
  --db_repl_user vitess \
  --db_repl_password "passpass" \
  --db_filtered_user vitess \
  --db_filtered_password "passpass" \
  --db_allprivs_user vitess \
  --db_allprivs_password "passpass" \
  --init_db_name_override my_old_database \
 > $VTDATAROOT/$tablet_dir/vttablet.out 2>&1 &

Binary Version

vttablet version Version: 21.0.3 (Git revision 94fdc736eae8928a8fdde44e9ec9c3bee1868d6f branch 'HEAD') built on Wed Feb 12 16:56:27 UTC 2025 by runner@fv-az1705-503 using go1.23.5 linux/amd64

vtgate version Version: 21.0.3 (Git revision 94fdc736eae8928a8fdde44e9ec9c3bee1868d6f branch 'HEAD') built on Wed Feb 12 16:56:27 UTC 2025 by runner@fv-az1705-503 using go1.23.5 linux/amd64

Operating System and Environment details

os-release : 

PRETTY_NAME="Ubuntu 22.04.5 LTS"
NAME="Ubuntu"
VERSION_ID="22.04"
VERSION="22.04.5 LTS (Jammy Jellyfish)"
VERSION_CODENAME=jammy
ID=ubuntu
ID_LIKE=debian
HOME_URL="https://www.ubuntu.com/"
SUPPORT_URL="https://help.ubuntu.com/"
BUG_REPORT_URL="https://bugs.launchpad.net/ubuntu/"
PRIVACY_POLICY_URL="https://www.ubuntu.com/legal/terms-and-policies/privacy-policy"
UBUNTU_CODENAME=jammy

uname -sr
Linux 5.15.0-136-generic

uname -m
x86_64

Log Fragments

[...]
I0411 11:33:22.822132   19345 sidecardb.go:466] Table schema was already up to date for the vreplication_log table in the _vt sidecar database
I0411 11:33:22.822167   19345 engine.go:176] syncSidecarDB done
I0411 11:33:22.822171   19345 engine.go:155] syncSidecarDB took 8 ms
I0411 11:33:22.822189   19345 engine.go:242] Schema Engine: opening
E0411 11:33:34.232377   19345 state_manager.go:291] Error transitioning to the desired state: PRIMARY, Serving, will keep retrying: Code: UNKNOWN
could not get table primary key info: Code: ABORTED
Row count exceeded 10000

I0411 11:33:34.232399   19345 state_manager.go:764] State: exiting lameduck
E0411 11:33:34.232403   19345 tm_state.go:360] Cannot start query service: Code: UNKNOWN
could not get table primary key info: Code: ABORTED
Row count exceeded 10000
[...]