karmada search return empty list sometimes

[CharlesQQ]

What happened:
use the command kubectl get --raw /apis/search.karmada.io/v1alpha1/search/cache/apis/apps/v1/deployments | python -m json.tool to fetch all deployments from cache, but return empty list sometimes

log print following message:
W0602 10:32:24.626462 28 cache.go:96] SingleClusterInformerManager for cluster(xxx) is nil.

What you expected to happen:
return resource list of all member cluster

How to reproduce it (as minimally and precisely as possible):

Anything else we need to know?:

# kubectl get resourceregistries.search.karmada.io test -o yaml
apiVersion: search.karmada.io/v1alpha1
kind: ResourceRegistry
metadata:
  annotations:
  creationTimestamp: "2022-06-01T13:42:13Z"
  managedFields:
  - apiVersion: search.karmada.io/v1alpha1
    fieldsType: FieldsV1
    fieldsV1:
      f:metadata:
        f:annotations:
          .: {}
          f:kubectl.kubernetes.io/last-applied-configuration: {}
      f:spec:
        f:resourceSelectors: {}
        f:targetCluster:
          f:clusterNames: {}
    manager: kubectl
    operation: Update
    time: "2022-06-01T13:42:13Z"
  name: test
  resourceVersion: "246709202"
  uid: c82278f3-b2e2-40bc-a058-135484cb17bd
spec:
  resourceSelectors:
  - apiVersion: apps/v1
    kind: Deployment
  targetCluster:
    clusterNames:
    - xxx
status: {}

Environment:

• Karmada version: v1.2
• kubectl-karmada or karmadactl version (the result of kubectl-karmada version or karmadactl version):
• Others:

[XiShanYongYe-Chang]

Will it happen or will it happen occasionally?

Is there are some logs like this:

I0602 03:00:16.369366       1 controller.go:200] try to build informer manager for cluster member1
I0602 03:00:16.404053       1 controller.go:222] add informer for member1, apps/v1, Resource=deployments
I0602 03:00:16.404137       1 controller.go:225] start informer for member1
I0602 03:00:16.505041       1 controller.go:228] start informer for member1 done

We need to start an informer for member clusters.

[GitHubxsy]

The karmada-search component is a multi-instance component and works in load balancing mode. Some instances work abnormally？

[CharlesQQ]

Some instances work abnormally？

3 instances, I find that cluster in pull mode, log print following message:

E0603 10:23:42.547929      26 controller.go:160] Error cache memeber cluster member1, cluster member1 does not have a secret

W0603 10:44:39.627526      26 cache.go:96] SingleClusterInformerManager for cluster(member1) is nil.

[CharlesQQ]

karmada/pkg/search/controller.go

Lines 203 to 205 in 36368ce

	clusterDynamicClient, err := util.NewClusterDynamicClientSet(cluster, controlPlaneClient)
	if err != nil {
	return err

create clusterDynamicClient need secret, but in pull mode, karmada control plane can not find secret objects

[RainbowMango]

create clusterDynamicClient need secret, but in pull mode, karmada control plane can not find secret objects

Right. For now, karmada-agent won't report secret for Pull mode cluster.
I think it's time to do it now.

[CharlesQQ]

I think it's time to do it now.

May be I can try resolve it~
/assign

[RainbowMango]

Just share some ideas here:

1. We need to introduce a flag to karmada-agent to specify whether report secret when registering cluster.
2. Then implements the logic at registerWithControlPlaneAPIServer

[GitHubxsy]

The karmada search and aggregation API feature requires that the kube-apiserver of the member cluster be accessed from the karmada control plane. If you cannot directly access the network due to network restrictions, you are advised to use the reverse proxy mode and set the cluster.proxyURL field.

The aggregation API uses the ImpersonatorSecret (empty permission). The karmada search uses the cluster.secretRef (higher permission is required). Currently, the cluster.secretRef is missing.
If you do not use the karmada search feature, you can choose not to report the secret.

[XiShanYongYe-Chang]

Hi @CharlesQQ, dose this issue been fixed?

[RainbowMango]

ping @CharlesQQ

[CharlesQQ]

/close

[karmada-bot]

@CharlesQQ: Closing this issue.

In response to this:

/close

Instructions for interacting with me using PR comments are available here. If you have questions or suggestions related to my behavior, please file an issue against the kubernetes/test-infra repository.