[DocDB][Upgrade] Invalid argument: Invalid table definition: Error creating table mv_upgrade_db2_12_4_2.pg_temp_16427 on the master: num_tablets should be greater than 0. Client would need to wait for master leader get heartbeats from tserver.

[def-]

Jira Link: DB-4455

Description

Upgrade test fails when verifying objects, from 2.12.4.2-b1 to 2.17.1.0-b190: https://phabricator.dev.yugabyte.com/D20539

testupgrade-k8s-rf3-upgrade-2.12.4.2_1: Start
	(     0.473s) User Login : Success
	(     0.215s) Refresh YB Version : Success
	(    90.603s) Setup Provider : Success
	(     0.108s) Updating Health Check Interval to 60000 sec : Success
	(   421.649s) Create universe dfel-isd2464-134a9b7b27-20221212-094001 : Success
	(    38.000s) Start sample workloads : Success
	(   874.648s) Creating V2 backups in S3 : Success
	(   430.588s) Verify geo-partitioned objects : Success
	(   466.816s) Executing yb-admin upgrade_ysql : Success
	(     0.001s) Executing yb-admin upgrade_ysql : >>> Integration Test Failed <<< 
Verifying during upgrade failed:
Invalid argument: Invalid table definition: Error creating table mv_upgrade_db2_12_4_2.pg_temp_16427 on the master: num_tablets should be greater than 0. Client would need to wait for master leader get heartbeats from tserver.

	(    27.266s) Saved server log files and keys at /share/jenkins/workspace/itest-system-developer/logs/2.17.1.0_testupgrade-k8s-rf3-upgrade-2.12.4.2_1_20221212_141752 : Success
	(    95.903s) Destroy universe : Success
	(     0.233s) Check and stop workloads : Success
testupgrade-k8s-rf3-upgrade-2.12.4.2_1: End

This happens in relation to materialized views CC @fizaaluthra I'll notify if I also see it without the materialized view.
Master logs before:

I1212 14:14:05.488943 16294 ts_manager.cc:140] Registered new tablet server { permanent_uuid: "3626254c45874a2695bac1bba30a8eb0" instance_seqno: 1670854445470052 start_time_us: 1670854445470052 } with Master, full list: [{93369358a01f42949a726b3ef44c1e76, 0x0000000004a5b210 -> { permanent_uuid: 93369358a01f42949a726b3ef44c1e76 registration: common { private_rpc_addresses { host: "10.9.196.217" port: 9100 } http_addresses { host: "10.9.196.217" port: 9000 } cloud_info { placement_cloud: "aws" placement_region: "us-west-2" placement_zone: "us-west-2c" } placement_uuid: "741619b4-52b7-4138-afa9-7464bc9d251d" pg_port: 5433 } capabilities: 2189743739 capabilities: 1427296937 capabilities: 2980225056 placement_id: aws:us-west-2:us-west-2c }}, {3626254c45874a2695bac1bba30a8eb0, 0x0000000004b7f690 -> { permanent_uuid: 3626254c45874a2695bac1bba30a8eb0 registration: common { private_rpc_addresses { host: "10.9.105.50" port: 9100 } http_addresses { host: "10.9.105.50" port: 9000 } cloud_info { placement_cloud: "aws" placement_region: "us-west-2" placement_zone: "us-west-2a" } placement_uuid: "741619b4-52b7-4138-afa9-7464bc9d251d" pg_port: 5433 } capabilities: 2189743739 capabilities: 1427296937 capabilities: 2980225056 placement_id: aws:us-west-2:us-west-2a }}, {68ed1e71075b4a7d8a2e003fd10582d1, 0x0000000004faf210 -> { permanent_uuid: 68ed1e71075b4a7d8a2e003fd10582d1 registration: common { private_rpc_addresses { host: "10.9.134.11" port: 9100 } http_addresses { host: "10.9.134.11" port: 9000 } cloud_info { placement_cloud: "aws" placement_region: "us-west-2" placement_zone: "us-west-2b" } placement_uuid: "741619b4-52b7-4138-afa9-7464bc9d251d" pg_port: 5433 } capabilities: 2189743739 capabilities: 1427296937 capabilities: 2980225056 placement_id: aws:us-west-2:us-west-2b }}]
I1212 14:14:05.488988 16294 catalog_manager.cc:1052] T 00000000000000000000000000000000 P e6957f5f939141398cf04e9b1133468a: 3 tablet servers registered, creating the transaction status table
I1212 14:14:05.489073 16294 catalog_manager.cc:3752] Creating transaction status table transactions
I1212 14:14:05.489809 16294 catalog_manager.cc:2976] CreateTable from internal request:
name: "transactions"
schema {
  columns {
    name: "key"
    type {
      main: BINARY
    }
    is_key: true
    is_hash_key: true
    is_nullable: false
    is_static: false
    is_counter: false
    sorting_type: 0
    order: 0
    pg_type_oid: 0
  }
  table_properties {
    num_tablets: 0
  }
}
table_type: TRANSACTION_STATUS_TABLE_TYPE
namespace {
  name: "system"
}
I1212 14:14:05.489915 16294 catalog_manager.cc:3155] Setting default tablets to 0 with 0 primary servers
I1212 14:14:05.489935 16294 partition.cc:542] Creating partitions with num_tablets: 0
W1212 14:14:05.489989 16294 catalog_manager.cc:1063] T 00000000000000000000000000000000 P e6957f5f939141398cf04e9b1133468a: Failed creating transaction status table, waiting: Invalid argument (yb/common/partition.cc:547): num_tablets should be greater than 0. Client would need to wait for master leader get heartbeats from tserver.

Full logs: https://drive.google.com/file/d/1ew5lcKdM0RkRyprLMD9adu8sJ_jjI2b8/view?usp=share_link