Enable multi-pool client connections

[mostafa]

At the moment GatewayD supports a pair of connection pools for managing available and busy connections. On one hand, these pools enables many incoming client, limited by the capacity of the pools, to be connected to a single database server. On the other hand, each client connection is mapped to a single server connection. This works, but is not ideal.

The idea here is to let GatewayD connect to multiple databases at the same time per configuration group, aka. tenant. Each configuration group should have configuration for multiple clients, pools, proxies and a single server. Note that clients, pools and proxies have named groups, yet there will be a single server listening for all of these databases. The distributionStrategy and the splitStrategy config parameters of the server object will decide how the connection are routed to their corresponding database connections (clients).

This should eventually enable a range of possibilities including routing, switching and relaying, to name the least.

This is a possible task list:

• Add a named config group (for clients, pools, proxies and servers) to config package to load the following config example
• Connect to multiple databases based on the named client config groups
• Create multiple pools per named config group and add corresponding clients to each pool
• Create multiple proxies per named config group and connect them with each pool
• Create a single server and attach all proxies to it
• Add tests

This is what the global config might look like after this change, yet other ideas should be explored.

clients:
  default:
    active-writes: # <- This is the "named config group"
      network: tcp
      address: localhost:5432
      ...
    standby-reads:
      network: tcp
      address: localhost:5433
      ...

pools:
  default:
    active-writes:
      size: 10
    standby-reads:
      size: 10

proxies:
  default:
    active-writes:
      healthCheckPeriod: 60s
    standby-reads:
      healthCheckPeriod: 60s

servers:
  default:
    distributionStrategy: ab-testing, canary, write-read-split (?), round-robin, murmur-hash, etc.
    splitStrategy:
      active-writes: 90 # percent
      standby-reads: 10 # percent (automatically deduced)
    ...

Related

• Add connection multiplexing to avoid 1:1 mapping of client to server connections #55

Resources

• Types of load balancing algorithms
• 10 Load Balancing Techniques: Mastering the Art of Distributed Computing
• Istio Traffic Management
• Linkerd Traffic Shifting

[mostafa]

Hey @likecodingloveproblems,

Would you like to work on this feature? This enables a lot of possibilities.

[likecodingloveproblems]

Actually I am interested :)

[mostafa]

Awesome! 🙌

[likecodingloveproblems]

I think this schema is correct not the above one:
@mostafa

clients:
    active-writes: # <- This is the "named config group"
      network: tcp
      address: localhost:5432
      ...
    standby-reads:
      network: tcp
      address: localhost:5433
      ...

pools:
    active-writes:
      size: 10
    standby-reads:
      size: 10

proxies:
    active-writes:
      healthCheckPeriod: 60s
    standby-reads:
      healthCheckPeriod: 60s

servers:
  default:
    distributionStrategy: ab-testing, canary, write-read-split (?), round-robin, murmur-hash, etc.
    splitStrategy:
      active-writes: 90 # percent
      standby-reads: 10 # percent (automatically deduced)
    ...

[mostafa]

@likecodingloveproblems
Now that I am rethinking it, it looks better the way you mentioned and also it'll be less nested. And then, the run command, and subsequently the server object, should be aware of non-matching set of config groups, especially in count and label, and then act accordingly based on the given parameters. You can find instances of [name], that refer to the config objects in the cmd/run.go file, which will eventually become objects stored in these global variables.

[likecodingloveproblems]

I try to draw the ERD.
Now I have two questions:

1. In the config file, we have a server object, we must relate proxies to servers as we want to have a one to many relation ship.
2. Now if a connection is closed or opened, we have OnOpen and OnClose functions on the Server, and get the server's Proxy and with a Map by O(1) the Connection is gotten, but now as the relation between Server and Proxy is becoming one to many, we must loop on all the proxies and then try to Find Connection's Proxy, Do you have any idea to have a better design?

[mostafa]

1. Correct.
2. We should figure out either 1) which proxy the connection wants to connect to or 2) which proxy the server wants to assign the connection to. Then we can store this information in a pool object inside the server, just like how available and busy connections work. Since the pool object is generic, the key can be the connection and the value can be the proxy it was assigned to.

[mostafa]

Hey @likecodingloveproblems,

I saw your awesome work you did in this PR. Is this still in progress?

[likecodingloveproblems]

Hi @mostafa
Thanks for your interest.
yes, I am working on it in weekends :)

[likecodingloveproblems]

I think split strategy is not related to server and must be a distribution strategy's parameter.
also for some strategies we don't need split strategy like round robin.

[mostafa]

Hey @likecodingloveproblems,

Any updates here?

[sinadarbouy]

@mostafa If I understand correctly, by removing nesting in clients, pools, and proxies, we can separate the server from the group of clients, pools, and proxies. This allows us to have two servers using the same proxy. Is that correct?

[mostafa]

@sinadarbouy No. The idea is to connect a single server instance to multiple proxies, each with different pools and different set of clients.

[sinadarbouy]

In this PR, we introduce a new configuration structure. We have separated the config groups for clients, pools, and proxies. This separation means we need to reference proxies in the server config groups to select proxies.

Additionally, we have implemented the default load balancer strategy, ROUND_ROBIN. To keep the scope of this PR manageable, the implementation for other strategies and loadBalancingRules will be addressed in separate PRs.

Here is the new configuration structure:

# GatewayD Global Configuration

loggers:
  default:
...

metrics:
  default:
...

clients:
  default:
...
  default-2:
...

pools:
  default:
...
  default-2:
...

proxies:
  default:
...
  default-2:
...

servers:
  default:
    network: tcp
    address: 0.0.0.0:15432
    proxies:
      - "default"
      - "default-2"
    loadBalancer:
      strategy: ROUND_ROBIN
      # Not yet implemented.
      # loadBalancingRules:
      #   - condition: "default"
      #     percentages:
      #       - proxy: "default"
      #         percentage: 70
      #       - proxy: "default-2"
      #         percentage: 30

api:
...

[sinadarbouy]

@mostafa
I think after the latest merge, all the strategies mentioned in the issue are now possible.
ab-testing, canary, write-read-split -> weightedroundrobin
murmur-hash -> consistenthash
round-robin and random.

Is there anything else you think is important to implement for this issue?

For other algorithms, I think it would be better to create separate issues based on need, with more details.

[mostafa]

Hey @sinadarbouy,

Thanks for your contributions! 🙏 I think this covers most of the cases, especially those mentioned in this PR. We should create separate tickets for expanding the functionality of this feature. I believe more features and functionality requires authenticated connections in the pool, as mentioned in the following ticket:

• Add connection multiplexing to avoid 1:1 mapping of client to server connections #55

I created a placeholder ticket to discuss the future enhancements we can build on top of this feature:

• Expand load balancing strategies #593

Thanks @likecodingloveproblems and @ccoVeille for helping make this possible.