CA check improvements for system-agent

[snasovich]

Placeholder
SURE-7665

[snasovich]

Blocked on design review currently.

[MbolotSuse]

Depends on #45628.

[Sahota1225]

#45973

[markusewalker]

Tested this with results over in Qase test run /RM/1631. I noted the following failure:

#	Scenario	Result
4	Setup Rancher with valid ca-certs and agent-tls-mode=system-store	❌

VALIDATION STEPS

1. Setup Rancher v2.9-head with a valid cert.pem and/or key.pem file.
   • Ensured agentTLSMode=system-store in the helm install command.
2. Ran command kubectl get settings agent-tls-mode -o yaml and noted that the value is strict:

$ kubectl get settings agent-tls-mode -o yaml
apiVersion: management.cattle.io/v3
customized: false
default: strict
kind: Setting
metadata:
  creationTimestamp: "2024-07-02T20:30:00Z"
  generation: 1
  name: agent-tls-mode
  resourceVersion: "936"
  uid: ba85e161-b9b9-4450-9265-d01d04ca59f9
source: ""
value: ""

There are two other scenarios that have the same steps as above that have not been tested yet due to this failure. Marking as blocked until this is resolved.

[MbolotSuse]

@markusewalker Can you provide more details? The default on 2.9.0 will be strict, but the actual value should be as in the helm command, system-store - your output doesn't provide sufficient detail to determine if that is the case.

[markusewalker]

@MbolotSuse sure, updated my comment with the information we discussed offline.

[markusewalker]

Re-ran the test cases outlined in Qase test run /RM/1631. Using v2.9.0-alpha7, I was successfully able to validate this issue. Summary of the scenarios tested provided below:

#	Scenario	Result
1	Validate Rancher is accessible with valid ca-certs	✅
2	Validate Rancher is inaccessible with invalid ca-certs	✅
3	Validate Rancher is unusable with invalid ca-certs and agentTLSMode=system-store	✅
4	Validate Rancher is accessible/usable with valid ca-certs and agentTLSMode=system-store	✅
5	Update agentTLSMode from strict to system-store	✅
6	Update agentTLSMode from system-store to strict	✅
7	Upgrade Rancher from v2.8-head -> v2.9-head	✅
8	Upgrade Rancher from v2.8-head -> v2.9-head w/agentTLSMode=strict	✅