Capturing some tserver startup logs before I forget...

[aegershman]

From startup on a tserver node on 2.1.2.0:

I0311 22:27:44.264277     6 tablet_server_main.cc:131] Using parsed rpc = q-m91513n3s0.q-g89927.bosh:9101
I0311 22:27:44.264338     6 tablet_server_main.cc:125] Reset YEDIS bind address to q-m91513n3s0.q-g89927.bosh:6379
I0311 22:27:44.264348     6 tablet_server_main.cc:125] Reset YCQL bind address to q-m91513n3s0.q-g89927.bosh:9042
...
I0311 22:27:44.279397     6 server_base.cc:487] Using private ip address q-m91513n3s0.q-g89927.bosh
I0311 22:27:44.281309     6 webserver.cc:148] Starting webserver on q-m91513n3s0.q-g89927.bosh:9000
...
I0311 22:27:44.281817    16 client-internal.cc:1693] New master addresses: q-m91512n3s0.q-g89926.bosh:7100
I0311 22:27:44.282894     6 webserver.cc:240] Webserver started. Bound to: http://10.156.89.42:9000/
I0311 22:27:44.283848     6 rpc_server.cc:169] RPC server started. Bound to: 10.156.89.42:9101
...
I0311 22:27:44.331427     6 rpc_server.cc:169] RPC server started. Bound to: 10.156.89.42:6379
I0311 22:27:44.333837     6 rpc_server.cc:169] RPC server started. Bound to: 10.156.89.42:9042
...
...
W0311 22:28:27.102784    47 outbound_call.cc:396] RPC callback for RPC call yb.tserver.TabletServerService.Read -> { remote: 10.156.89.41:7100 idx: 4 protocol: 0x00007ff62c451f00 -> tcp } , state=FINISHED_SUCCESS. took 0.046s
...
W0311 22:28:27.102784    47 outbound_call.cc:396] RPC callback for RPC call yb.tserver.TabletServerService.Read -> { remote: 10.156.89.41:7100 idx: 4 protocol: 0x00007ff62c451f00 -> tcp } , state=FINISHED_SUCCESS. took 0.046s
W0311 22:28:27.298724    43 cql_rpc.cc:279] CQL Call from 10.156.90.9:57088, stream id: 64 took 31090ms. Details:
W0311 22:28:27.298805    43 cql_rpc.cc:282] cql_details {
}
Trace: 
W0311 22:28:27.298827    43 cql_rpc.cc:283] 
I0311 22:28:27.309521    21 heartbeater.cc:287] P fc5a3bbceb3845af932a11c6b1dd065a: Connected to a leader master server at q-m91512n3s0.q-g89926.bosh:7100
I0311 22:28:27.309573    21 heartbeater.cc:339] P fc5a3bbceb3845af932a11c6b1dd065a: Registering TS with master...
I0311 22:28:27.309590    21 server_base.cc:487] Using private ip address q-m91513n3s0.q-g89927.bosh
I0311 22:28:27.309629    21 heartbeater.cc:348] P fc5a3bbceb3845af932a11c6b1dd065a: Sending a full tablet report to master...
...
I0311 22:28:27.041996    45 client-internal.cc:1693] New master addresses: q-m91512n3s0.q-g89926.bosh:7100
W0311 22:28:27.102784    47 outbound_call.cc:396] RPC callback for RPC call yb.tserver.TabletServerService.Read -> { remote: 10.156.89.41:7100 idx: 4 protocol: 0x00007ff62c451f00 -> tcp } , state=FINISHED_SUCCESS. took 0.046s
W0311 22:28:27.298724    43 cql_rpc.cc:279] CQL Call from 10.156.90.9:57088, stream id: 64 took 31090ms. Details:
W0311 22:28:27.298805    43 cql_rpc.cc:282] cql_details {
}
Trace: 
W0311 22:28:27.298827    43 cql_rpc.cc:283] 
I0311 22:28:27.309521    21 heartbeater.cc:287] P fc5a3bbceb3845af932a11c6b1dd065a: Connected to a leader master server at q-m91512n3s0.q-g89926.bosh:7100
I0311 22:28:27.309573    21 heartbeater.cc:339] P fc5a3bbceb3845af932a11c6b1dd065a: Registering TS with master...
I0311 22:28:27.309590    21 server_base.cc:487] Using private ip address q-m91513n3s0.q-g89927.bosh
I0311 22:28:27.309629    21 heartbeater.cc:348] P fc5a3bbceb3845af932a11c6b1dd065a: Sending a full tablet report to master...
W0311 22:28:27.333510    45 client-internal.cc:1134] GetTableSchemaRpc(table_identifier: table_name: "schema_keyspaces" namespace { name: "system" database_type: YQL_DATABASE_CQL }, num_attempts: 1) failed: Not found (yb/master/catalog_manager.cc:3527): The object does not exist: table_name: "schema_keyspaces" namespace { name: "system" database_type: YQL_DATABASE_CQL }
W0311 22:28:27.333575    45 client-internal.cc:1134] GetTableSchemaRpc(table_identifier: table_name: "schema_columnfamilies" namespace { name: "system" database_type: YQL_DATABASE_CQL }, num_attempts: 1) failed: Not found (yb/master/catalog_manager.cc:3527): The object does not exist: table_name: "schema_columnfamilies" namespace { name: "system" database_type: YQL_DATABASE_CQL }
W0311 22:28:27.333626    45 client-internal.cc:1134] GetTableSchemaRpc(table_identifier: table_name: "schema_columns" namespace { name: "system" database_type: YQL_DATABASE_CQL }, num_attempts: 1) failed: Not found (yb/master/catalog_manager.cc:3527): The object does not exist: table_name: "schema_columns" namespace { name: "system" database_type: YQL_DATABASE_CQL }
E0311 22:28:27.333631    43 process_context.cc:180] SQL Error: Object Not Found
SELECT * FROM system.schema_columnfamilies
              ^^^^^^^^^^^^^^^^^^^^^^^^^^^^
W0311 22:28:27.333663    47 client-internal.cc:1134] GetTableSchemaRpc(table_identifier: table_name: "schema_usertypes" namespace { name: "system" database_type: YQL_DATABASE_CQL }, num_attempts: 1) failed: Not found (yb/master/catalog_manager.cc:3527): The object does not exist: table_name: "schema_usertypes" namespace { name: "system" database_type: YQL_DATABASE_CQL }
W0311 22:28:27.333999    47 client-internal.cc:1134] GetTableSchemaRpc(table_identifier: table_name: "schema_functions" namespace { name: "system" database_type: YQL_DATABASE_CQL }, num_attempts: 1) failed: Not found (yb/master/catalog_manager.cc:3527): The object does not exist: table_name: "schema_functions" namespace { name: "system" database_type: YQL_DATABASE_CQL }
W0311 22:28:27.333827    45 client-internal.cc:1134] GetTableSchemaRpc(table_identifier: table_name: "schema_aggregates" namespace { name: "system" database_type: YQL_DATABASE_CQL }, num_attempts: 1) failed: Not found (yb/master/catalog_manager.cc:3527): The object does not exist: table_name: "schema_aggregates" namespace { name: "system" database_type: YQL_DATABASE_CQL }
E0311 22:28:27.334044    46 process_context.cc:180] SQL Error: Object Not Found
SELECT * FROM system.schema_keyspaces
              ^^^^^^^^^^^^^^^^^^^^^^^
E0311 22:28:27.334075    48 process_context.cc:180] SQL Error: Object Not Found
SELECT * FROM system.schema_usertypes
              ^^^^^^^^^^^^^^^^^^^^^^^
E0311 22:28:27.333741    49 process_context.cc:180] SQL Error: Object Not Found
SELECT * FROM system.schema_columns
              ^^^^^^^^^^^^^^^^^^^^^
E0311 22:28:27.334157    50 process_context.cc:180] SQL Error: Object Not Found
SELECT * FROM system.schema_functions
              ^^^^^^^^^^^^^^^^^^^^^^^
E0311 22:28:27.334235    51 process_context.cc:180] SQL Error: Object Not Found
SELECT * FROM system.schema_aggregates
              ^^^^^^^^^^^^^^^^^^^^^^^^
W0311 22:28:27.934576    47 client-internal.cc:1134] GetTableSchemaRpc(table_identifier: table_name: "schema_keyspaces" namespace { name: "system" database_type: YQL_DATABASE_CQL }, num_attempts: 1) failed: Not found (yb/master/catalog_manager.cc:3527): The object does not exist: table_name: "schema_keyspaces" namespace { name: "system" database_type: YQL_DATABASE_CQL }
E0311 22:28:27.934643    43 process_context.cc:180] SQL Error: Object Not Found
SELECT * FROM system.schema_keyspaces WHERE keyspace_name = 'ybdemo_keyspace'
              ^^^^^^^^^^^^^^^^^^^^^^^
W0311 22:28:27.937759    45 client-internal.cc:1134] GetTableSchemaRpc(table_identifier: table_name: "schema_usertypes" namespace { name: "system" database_type: YQL_DATABASE_CQL }, num_attempts: 1) failed: Not found (yb/master/catalog_manager.cc:3527): The object does not exist: table_name: "schema_usertypes" namespace { name: "system" database_type: YQL_DATABASE_CQL }
W0311 22:28:27.937830    53 client-internal.cc:1134] GetTableSchemaRpc(table_identifier: table_name: "schema_aggregates" namespace { name: "system" database_type: YQL_DATABASE_CQL }, num_attempts: 1) failed: Not found (yb/master/catalog_manager.cc:3527): The object does not exist: table_name: "schema_aggregates" namespace { name: "system" database_type: YQL_DATABASE_CQL }
W0311 22:28:27.937789    52 client-internal.cc:1134] GetTableSchemaRpc(table_identifier: table_name: "schema_columnfamilies" namespace { name: "system" database_type: YQL_DATABASE_CQL }, num_attempts: 1) failed: Not found (yb/master/catalog_manager.cc:3527): The object does not exist: table_name: "schema_columnfamilies" namespace { name: "system" database_type: YQL_DATABASE_CQL }
E0311 22:28:27.937885    50 process_context.cc:180] SQL Error: Object Not Found
SELECT * FROM system.schema_columnfamilies WHERE keyspace_name = 'ybdemo_keyspace'
              ^^^^^^^^^^^^^^^^^^^^^^^^^^^^
W0311 22:28:27.937937    47 client-internal.cc:1134] GetTableSchemaRpc(table_identifier: table_name: "schema_functions" namespace { name: "system" database_type: YQL_DATABASE_CQL }, num_attempts: 1) failed: Not found (yb/master/catalog_manager.cc:3527): The object does not exist: table_name: "schema_functions" namespace { name: "system" database_type: YQL_DATABASE_CQL }
W0311 22:28:27.937965    47 client-internal.cc:1134] GetTableSchemaRpc(table_identifier: table_name: "schema_columns" namespace { name: "system" database_type: YQL_DATABASE_CQL }, num_attempts: 1) failed: Not found (yb/master/catalog_manager.cc:3527): The object does not exist: table_name: "schema_columns" namespace { name: "system" database_type: YQL_DATABASE_CQL }
E0311 22:28:27.937992    46 process_context.cc:180] SQL Error: Object Not Found
SELECT * FROM system.schema_functions WHERE keyspace_name = 'ybdemo_keyspace'
              ^^^^^^^^^^^^^^^^^^^^^^^
E0311 22:28:27.937999    51 process_context.cc:180] SQL Error: Object Not Found
SELECT * FROM system.schema_usertypes WHERE keyspace_name = 'ybdemo_keyspace'
              ^^^^^^^^^^^^^^^^^^^^^^^
E0311 22:28:27.938097    49 process_context.cc:180] SQL Error: Object Not Found
SELECT * FROM system.schema_columns WHERE keyspace_name = 'ybdemo_keyspace'
              ^^^^^^^^^^^^^^^^^^^^^
E0311 22:28:27.938138    48 process_context.cc:180] SQL Error: Object Not Found
SELECT * FROM system.schema_aggregates WHERE keyspace_name = 'ybdemo_keyspace'
              ^^^^^^^^^^^^^^^^^^^^^^^^
...

see also:

• https://docs.yugabyte.com/latest/architecture/core-functions/high-availability/
• https://docs.yugabyte.com/latest/architecture/docdb/replication/