Error when accessing member cluster by client-go using karmada-apiserver endpoint

[whitewindmills]

I can access resource of member cluster using my serviceaccount in CLI:

But got such an error using client-go:

What do you think about this question?:
I'm sure all the configuration is fine because I can access the host cluster resources via client-go.

Environment:

• Karmada version: v1.3.0
• Kubernetes version: v1.24.0
• Others:

[RainbowMango]

cc @XiShanYongYe-Chang @lonelyCZ

[XiShanYongYe-Chang]

Hi @b2k4j, How do you use client-go, can you help share your demo?

[whitewindmills]

@XiShanYongYe-Chang
Sure, my configuration of the karmada-apiserver is as following:

func CreateClientKarmadaConfig(host, cluster string, clientSecret *corev1.Secret) (*rest.Config, error) {
	ca := clientSecret.Data["ca.crt"]
	token := clientSecret.Data["token"]

	if err := os.MkdirAll(clientDir, fs.ModeDir); err != nil {
		klog.Errorf("failed to mkdir client dir `%s`: %v", clientDir, err)
		// keep going on
	}
	if err := os.WriteFile(caFile, ca, fs.ModePerm); err != nil {
		klog.Errorf("error writing CA file `%s`: %v", caFile, err)
		return nil, err
	}
	if err := os.WriteFile(tokenFile, token, fs.ModePerm); err != nil {
		klog.Errorf("error writing token file `%s`: %v", tokenFile, err)
		return nil, err
	}

	tlsClientConfig := rest.TLSClientConfig{}

	if _, err := certutil.NewPoolFromBytes(ca); err != nil {
		klog.Errorf("expected to load client CA config from %s, but got err: %v", caFile, err)
	} else {
		tlsClientConfig.CAFile = caFile
	}

	return &rest.Config{
		Host:            host,
		TLSClientConfig: tlsClientConfig,
		BearerToken:     string(token),
		BearerTokenFile: tokenFile,
                Proxy: func(req *http.Request) (*url.URL, error) {
		req.URL.Path = path.Join("/apis/cluster.karmada.io/v1alpha1/clusters", cluster, "proxy", req.URL.Path)
		return req.URL, nil
	        }
	}, nil
}

And the sample code is as follows:

clientset, err := sdkClient.NewKubeClientset()
if err != nil {
	setupLog.Error(err, "unable to start clientset")
	os.Exit(1)
}

podList, err := clientset.CoreV1().Pods("ccos-kube-apiserver").List(context.Background(), metav1.ListOptions{})
if err != nil {
	klog.Errorf("unable to list pods: %+v", err)
	os.Exit(1)
}

for _, n := range podList.Items {
	setupLog.Info("list nodes: %q", n.Name)
}

[XiShanYongYe-Chang]

Hi @b2k4j , I'm sorry, I don't know how to solve this problem. I think some parameters are not set correctly when constructing *rest.Config.

[lonelyCZ]

I also guess that it could be the parameter problem of *rest.Config, because we can use the Factory to access the resources of member clusters normally.

karmada/pkg/karmadactl/util/factory.go

Lines 58 to 59 in f68da6d

	// FactoryForMemberCluster returns a cmdutil.Factory for the member cluster
	func (f *factoryImpl) FactoryForMemberCluster(clusterName string) (cmdutil.Factory, error) {

[whitewindmills]

@XiShanYongYe-Chang
Fine, thanks for your reply.

[whitewindmills]

@lonelyCZ
Thanks, it's a nice tip!

[XiShanYongYe-Chang]

Our E2E also constructs requests to access:

karmada/test/e2e/aggregatedapi_test.go

Line 199 in 5811aea

code, err := helper.DoRequest(fmt.Sprintf(karmadaHost+clusterProxy+"api", member1), tomToken)

If there are new developments, we can talk more about it.

[whitewindmills]

@lonelyCZ @XiShanYongYe-Chang
Thanks for your help! I've fixed this issue.

[whitewindmills]

/close

[karmada-bot]

@b2k4j: Closing this issue.

In response to this:

/close

Instructions for interacting with me using PR comments are available here. If you have questions or suggestions related to my behavior, please file an issue against the kubernetes/test-infra repository.

[XiShanYongYe-Chang]

pkg/controllers/cluster/cluster_controller.go

Gold to here that. By the way, can you share your fix?

[whitewindmills]

I'm sorry I do not know the cause of this problem, and it's still a mystery. I resolve this problem by using *rest.Config.Host instead of *rest.Config.Proxy behaving like *rest.Config.Host = "https://karmada-apiserver.karmada-system.svc.cluster.local:5443/apis/cluster.karmada.io/v1alpha1/clusters/member1/proxy".

[XiShanYongYe-Chang]

Ok. thanks~