esgateway pggateway: Changes to parsed nodes and resolvers to PG gateway HAProxy And Make resolvers configurable

[kalroy]

Signed-off-by: Kallol Roy karoy@progress.com

🔩 Description: What code changed, and why?

Automate when configured with multiple postgres nodes (master + followers) fails to resolve the added/removed follower from the cluster beyond the ones mentioned in the config.
The aim is to make sure that HAProxy should be able to resolve the nodes in a periodic manner based on the resolv.conf.

⛓️ Related Resources

This involves changes in:

1. Modifying HaProxy template to include the resolvers and server-template
2. Parsing the resolver.conf to find the default resolvers IP addresses
3. Update the config request to populate the HaProxy Config.

The resolvers can be added and passed by adding the following configurations:

[pg_gateway.v1.sys.resolvers]
# Multiple resolvers can be specified by
# adding the resolvers in the list.
nameservers = ["127.0.0.53:52"]
enable_system_nameservers = false

In case of ESgateway

[esgateway.v1.sys.ngx.main.resolvers]
nameservers = ["127.0.0.53:52"]
enable_system_nameservers = false

👍 Definition of Done

Setting the configurations should make following changes in the deployed environment:

1. Setting the esgateway section should make a change in the /hab/svc/automate-es-gateway/config/nginx.conf and the resolvers should be added. Example:

resolver 192.168.65.1

2. Setting the pg_gateway section should make a change in the /hab/svc/automate-pg-gateway/config/haproxy.conf and the resolvers should be added. Example:

resolvers pgdns
        nameserver dns0 192.168.65.1:52

👟 How to Build and Test the Change

1. Pull the latest code
2. Enter hab studio
3. Rebuild the components

rebuild components/automate-pg-gateway
rebuild components/automate-es-gateway
rebuild components/automate-cli
rebuild components/automate-deployment

4. Make the changes in the config.toml as above.
5. start_all_services
6. Have a look at /hab/svc/automate-es-gateway/config/nginx.conf and /hab/svc/automate-pg-gateway/config/haproxy.conf

✅ Checklist

• I have read the CONTRIBUTING document.
• Tests added/updated?
• Docs added/updated?
• All commits have been signed-off for the Developer Certification of Origin.

📷 Screenshots, if applicable

[netlify]

👷 Deploy preview for chef-automate processing.

🔨 Explore the source changes: 56aba38

🔍 Inspect the deploy logs: https://app.netlify.com/sites/chef-automate/deploys/5fe1e2f7b36229000702e330

[jaym]

You might be able to modify the config of some of the HA tests to get some integration coverage here

[msorens]

Thanks for adding the "how to test..." and "definition of done" sections. That gets the reader much closer to understanding what you are trying to achieve. However, for someone unfamiliar with the context (like me 😁 ), that seems to only show that the entered config lines end up in the correct location. But how does that connect to what your "description" says?

[kalroy]

I agree. I should have added more information there. The resolvers would make sure that the PG/ES node IPs can be resolved dynamically and any change in the cluster would not fail the application. In the current scenario when the node IPs are provided and if cluster node goes down, the gateway would keep on trying with the IP addresses. This encourages users to provide hostnames rather than the IPs.

[kalroy]

Found an issue while doing acceptance testing for this. The issue can be summarised as below:

While setting up external postgres. In case if the config does not have

[global.v1.external.postgresql.ssl]
   enable =false

Automate-dex expects the connection to be SSL enabled and expects the root_cert path.
This can be fixed from the code or we can update the doc to mention the behaviour.
But we can take it up in the upcoming release as generally for any external ES/PG setup, the connections would be SSL secured for obvious reasons.