MX project -> Community integration

[ozgune]

We have been working on the Citus MX project in a separate repository. We intend to keep the MX repository as much in sync as possible with this repository.

We currently have the following components in MX that diverge from Community. As we integrate these components into Citus, we probably want to open up a separate github issue for each one and track the integration's progress there:

1. Use pg_dist_node instead of pg_worker_list.conf
2. 2PC recovery function
3. Distributed metadata syncing infrastructure
4. Nodes joining a cluster
5. create_clustered_table() : Do we need to have a separate function? This issue relates to our API revamp and we'll likely defer working on it until we finalize the API.

The first three components are easier to integrate.

[marcocitus]

Issues to be resolved in order of priority:

• node membership should use a catalog table instead of pg_worker_list.conf #786 node membership should use a catalog table instead of pg_worker_list.conf
• Add a UDF to copy the current metadata from master to worker #1011 Add a function for making a worker acquire metadata
• Add metadata snapshot infrastructure #790 Add metadata snapshot infrastructure
• Add worker transactions infrastructure with transaction recovery #792 Add worker transactions infrastructure with transaction recovery
• Command counter increment in the wrong place #714 Command counter increment in the wrong place
• Propagate metadata changes for hash-distributed tables with replication factor 1 #791 Propagate metadata changes for hash-distributed tables with replication factor 1
• Add worker sequence number generation logic #794 Add worker sequence number generation logic
• Error out when performing unsupported operations on workers #793 Error out when performing unsupported operations on workers
• Reference tables: MX integration #967 Reference Tables - MX Integration
• Sync pg_dist_node when nodes join/leave #797 Sync pg_dist_node when nodes join/leave
• Prevent co-location of MX and non-MX Tables #1009 Prevent co-location of mx and non-mx tables
• Differentiate worker and master job temporary folders  #796 Differentiate worker and master job temporary folders
• Add regression tests for queries from workers #795 Add regression tests for queries from workers

[marcocitus]

All the boxes are checked 🎆