Added support for server discovery, TLS and ACL login

[Ganeshrockz]

Changes proposed in this PR:

This PR introduces the following changes to consul ECS

• Adds consul server connection manager discover the consul server agents from the ECS control plane in the new architecture
• Adds support for ACL login in the Control plane module
• Adds support for protocol specific TLS (HTTPS & gRPC) configurations
• Adds the consulServers field to schema.json. The top level field looks like

{
   "consulServers": {
          "hosts": "<addresses>",
          "https": true, // <Defaults to true>, 
          "tls": true, // <Defaults to true>,
          "httpPort":  8501, //<Defaults to 8501 since https is enabled by default>
          "grpcPort": 8503, //<Defaults to 8503 since tls is enabled by default>
          "caCertFile": "",
          "httpsCACertFile": "",
          "skipServerWatch": false,
          "tlsServerName": ""
    }
}

• Configures the new client which can talk to the server directly.
• Stores the ACL token returned by the server connection manager in a separate file and writes it to a shared volume. This will be later used by the dataplane to configure itself.
• Server watch isn't implemented yet. Will be done in subsequent iterations.

How I've tested this PR:

Unit tests for now. I have decided to split the agentless implementation into multiple PRs and have the ganeshrockz/agentless branch as the target branch. Once all the necessary changes go into it, we need to rely on acceptance/integration tests before merging to main.

How I expect reviewers to test this PR:

Checklist:

• Tests added
• CHANGELOG entry added (Not needed)

[pglass]

The one comment that is a "must fix" is setting the partition in the login data.

The rest of the comments are minor/non-blocking (feel free to address/ignore how you feel is best).

Otherwise, approved so you can merge when ready. This is great work!

[pglass]

non-blocking / thinking aloud: What do you think about restructuring the config fields with nested objects for the http/grpc settings?

This would look like the following.

{
  "consulServers": {
    "hosts": "",
    "skipServerWatch": null,
    "defaults": {   ## Common/default TLS settings for both http and grpc
      "caCertFile": null,
      "tls": true,
      "tlsServerName": null
    },
    "http": {   ## HTTP-only settings (overrides `defaults` for http)
      "caCertFile": null,
      "port": null,
      "tls": null,
      "tlsServerName": null
    },
    "grpc": {  ## GRPC-specific settings (overrides `defaults` for gRPC)
      "caCertFile": null,
      "port": null,
      "tls": null,
      "tlsServerName": null
    }
  }
}

Although, looking at https://developer.hashicorp.com/consul/docs/k8s/helm#externalservers I think consul-k8s assumes the same TLS settings for https and grpc+tls to the Consul servers? So perhaps we follow their lead (I know I previously said we should have separate settings for those, but maybe it's uncommon to do so) 🤔

Maybe this is overcomplicating. What do you think?

[Ganeshrockz]

I think consul-k8s assumes the same TLS settings for https and grpc+tls to the Consul servers?

We support configuring individual TLS CA certs for HTTP and gRPC in the current version of Consul ECS. The mesh task's terraform module exposes two variables to control that. So I guess we should continue maintaining both the settings going forward as well. What do you think?

That said, I do like the way you restructured these inputs so that users can go with a default setting for both the protocols. I will merge this PR in (since it has already become huge) and raise a separate one to address this restructuring.

[pglass]

I guess we should continue maintaining both the settings going forward as well

Yeah, that makes sense. Though, the existing RPC cert was technically for Consul client agent's internal RPC (port 8300, not gRPC which is 8502/8503). Anyway, we can make breaking changes in a new version, so if you feel that it's easiest/simplest to start out with one set of settings for both grpc and https, I'm good with that.

I do like the shared block of tls settings, because it's probably pretty common that the TLS settings for http and grpc are the same. We also probably want to also support other tls settings, like client certificates, skip tls verification, so I like the nested objects to organize the many tls settings (especially if we have defaults and settings for each specific protocol).

[pglass]

I suspect the CI tests are failing with Consul 1.13 because of gRPC TLS settings that were weird in 1.13.

You could try to fix this, or we can drop tests for Consul 1.13. We typically maintain and test with N-2 versions of Consul. So with Consul 1.16 GA that would be 1.16, 1.15, and 1.14.

[Ganeshrockz]

We typically maintain and test with N-2 versions of Consul.

If that's the case, I can merge this PR first and later on drop the tests once Consul 1.16 becomes GA. Since this isn't getting merged to main I think it should be fine.