Feature request: support citus read scaling

[xin-hedera]

What happened?

Citus supports serving read-only queries in the replica clusters formed by replica coordinators and replica worker nodes when:

• the replica coordinators is configured with citus.use_secondary_nodes = 'always'
• the read replica worker nodes are added to pg_dist_node using citus_add_secondary_node

The functionality is missing from the latest patroni release v3.1.0 as far as I know.

Being able to load balance read-only queries across all nodes is important since it provides better performance and higher resource utilization.

How can we reproduce it (as minimally and precisely as possible)?

Create a citus cluster using the demo docker-compose yaml file

• check pg_dist_node to verify there's no secondary node
• check citus.use_secondary_nodes on replica / standby coordinators to verify it's the default 'never'

What did you expect to happen?

patroni makes it optional to enable load balance read-only queries over all worker nodes

Patroni/PostgreSQL/DCS version

• Patroni version: 3.1.0
• PostgreSQL version: 14.8
• Citus extension version: 11.3
• DCS (and its version):

Patroni configuration file

N/A

patronictl show-config

N/A

Patroni log files

N/A

PostgreSQL log files

N/A

Have you tried to use GitHub issue search?

• Yes

Anything else we need to know?

No response

[CyberDem0n]

@xin-hedera please to check/try/test #2755