Join GitHub today
GitHub is home to over 40 million developers working together to host and review code, manage projects, and build software together.Sign up
[YSQL] Optimize session initialization for multi-region deployment #3049
Doing it a second time is significantly faster.
Enabled tracing of RPCs to master leader, and we get about 583 RPCs (mostly a mix of Read/GetTableLocations) calls. As you can these calls arrive about 70ms apart. And so 583*70ms = 40sec -- explains why ysqlsh takes about 40sec to launch the very first time against a TServer. Subsequent ysqlsh connections to the same TServer are much faster.
Turned on some debug logging to print the protos for these calls to from yb-tserver to yb-master when a ysqlsh connects to a TServer for the first time:
A typical call to yb.master.MasterService.GetTableLocations looks like:
Sometimes same table_id calls come to yb-master with slightly different arguments:
By count the 2 table ids see most of the GetTableLocations calls.
Checked the top 2 offenders above and they are:
So we are not only call GetTableLocations many times, but we are calling it many times on the same tables. Most of these calls are to pg_index and pg_rewrite tables.
Now, coming to the 277 yb.tserver.TabletServerService.Read calls to yb-master, nearly 120 of them are those involving this table and its index.
The proto for 120 of these calls looks something like this:
The schema for the pg_rewrite table:
and sample rows:
And, the remaining big chunk (90 odd of the Read calls) are of this form to
For what it's worth, there were only 116 rows in that table: