[YSQL] Make system catalog tables transactional

[robertpang]

Currently for implementation reasons system catalog tables are non-transactional. There has been a case where a client reads the tables in an inconsistent state while loading the sys catalog cache, leading to FATAL error. We should make the tables transactional to avoid such issues. This change can also make it easy to rollback sys catalog table changes in case of a failed DDL operation.

[mbautin]

Some initial things I'm noticing trying to implement this:

• Switching the main sys catalog table to be transactional works, at least on the surface. We still have to specify transaction_participant_context and transaction_coordinator_context in sys_catalog.cc.
• We need to flip postgres sys catalog tables to be transactional.
• We need to create the txn status table earlier in the CreateTable handler implementation in catalog_manager.cc so it is ready by the time we start sending DML to postgres sys catalog tables.
• To create the transaction status table, at least one tserver needs to be up (in the RF=1 case). Might need to wait for tserver to start before the master starts running initdb.
• Alternatively, we could allow the sys catalog tablet to also act as a transaction status tablet for transactions involving sys catalog tables.

[mbautin]

Also, our buffering operations needed to be disabled because it is currently not supported for transactional tables.

WIP patch: https://gist.githubusercontent.com/mbautin/db313ed7eae0c81a8dc12ff620c00706/raw
Log (fails on trying to create the second table, need to figure out if we're missing a commit): https://gist.githubusercontent.com/mbautin/69a884cee447402836a1a67905ce0967/raw

Also as another idea we should probably running initdb in non-transactional mode for speed.

[ndeodhar]

Fixed as part of #3108.