[WIP] Feat: Add clusterTLSConfigs to the Alertmanager CRD

[mviswanathsai]

Description

This PR adds mTLS config to AlertManager, allowing user to configure gossip between alert managers using mTLS for authentication.
Closes: #4241

Type of change

What type of changes does your code introduce to the Prometheus operator? Put an x in the box that apply.

• CHANGE (fix or feature that would cause existing functionality to not work as expected)
• [ x] FEATURE (non-breaking change which adds functionality)
• BUGFIX (non-breaking change which fixes an issue)
• ENHANCEMENT (non-breaking change which improves existing functionality)
• NONE (if none of the other choices apply. Example, tooling, build system, CI, docs, etc.)

Verification

Please check the Prometheus-Operator testing guidelines for recommendations about automated tests.

Changelog entry

Please put a one-line changelog entry below. This will be copied to the changelog file during the release process.

add mTLS configuration to alertmanager CRD

[mviswanathsai]

Questions:

1. What about concurrency and timeouts, that we have in AlertmanagerWebConfig? Would that apply here?
2. I am still not sure about the naming, feedback is much needed.
3. What about "de-duplication" logic in [WIP] add MTLS config for alert manager #4593? would we need to have different naming conventions for server and client tls creds? clarity is needed in this area.
4. Would like suggestions if any, for the code yet to be written. For example, I am writing the code in alertmanager/statefulset.go to include the volumes, mounts and args for the clusterTLSConfig (mTLSConfig), on the condition that the version of prometheus is >=0.27.0. This is yet to be pushed.

[mviswanathsai]

Requesting review: @simonpasquier @ArthurSens @nicolastakashi @slashpai.

Some concepts are still not clear to me, especially stuff that is not directly related to the code itself but the cluster tls config in general. I think the review will help a lot. Kindly bear with me, if I am not up to speed!

[mviswanathsai]

Background:
When configuring TLS credentials for both the server and the clients, there was an issue where the same names or parameters could be assigned to both, leading to duplication. To handle this, a "de-duplication" logic was implemented (Edit: in #4593). This logic ensured that secrets with the same name were not added more than once to the list of parameters.

Problem with De-duplication:
The "de-duplication" mechanism seems to assume that the secrets with the same name, regardless of being part of the client's tls creds or the server's are the same. I am unable to verify if this is right or not. So, to avoid any confusions, I adopted the following alternative approach.

Alternative Approach:
To avoid potential issues with de-duplication, a new approach was adopted. Unique suffixes are now assigned to the TLS credentials for the server and the clients. This ensures that each set of credentials is distinct, eliminating the risk of duplication without the need for additional logic. This logic can be seen in action here.

[simonpasquier]

Can you rebase on top of main to get #6612?

[mviswanathsai]

I reckon we also need to add tests, I will get to it once we are happy with the current changes.

[mviswanathsai]

Currently caught up with other things, will get back to this PR once things higher on my priority list are done!