Timeout Errors and Resource Loading Issues in HA Argo CD with Rancher Managed Clusters

[voidsp-rick]

Hi, I've set up Argo CD in HA mode in my infra cluster and connected three development clusters through Rancher with tokens added as secrets in Argo CD. The connections are established, but I'm encountering frequent timeouts when accessing Argo CD's public IP.

Issues Encountered:

1. Timeout Error:
   • Error Message: Unable to load data: error listing resource events: Get "external-dev-cluster-rancher-endpoint": dial tcp -argo-cluster-public-ip:443: i/o timeout
   • This occurs intermittently but frequently causes data load failures.

2. Resource Not Found:
   • Error Message: Resource not found in cluster: undefined/undefined:<application-resource>
   • Occurs sporadically and resolves after several refreshes.

Steps to Reproduce:

• Direct access to Kubernetes API via https://rancher-external-cluster-endpoint/version?timeout=32s works without delay.
• Argo CD experiences timeouts even though local cluster data is fetched without issues.

Expected Behavior:

• Argo CD should fetch data without encountering network timeouts or sporadic resource errors.

Actual Behavior:

• Frequent timeouts and intermittent resource not found errors in the Argo CD UI.

Environment Details:

• k3s version: v1.28.8+k3s1
• Argo CD version: v2.10.9+c071af8
• Rancher version: v2.8.3
• Network configuration: No rules or firewall settings that should impact traffic.

Looking forward to any suggestions or insights into what might be causing these issues and how to resolve them.

[luizbrito6]

@voidsp-rick Did you manage to solve this problem?