Join cluster error #1291

chenrc0529-ai · 2022-01-20T08:18:18Z

[root@k8s-master ~]# kubectl karmada join kubernetes-admin --kubeconfig=/etc/karmada/karmada-apiserver.config --cluster-kubeconfig=$HOME/.kube/config
W0120 16:17:30.264037 12485 cluster.go:106] failed to create cluster(kubernetes-admin). error: Cluster.cluster.karmada.io "kubernetes-admin" is invalid: [spec.secretRef.namespace: Required value, spec.secretRef.name: Required value, spec.impersonatorSecretRef.namespace: Required value, spec.impersonatorSecretRef.name: Required value]
W0120 16:17:30.264245 12485 cluster.go:50] failed to create cluster(kubernetes-admin). error: Cluster.cluster.karmada.io "kubernetes-admin" is invalid: [spec.secretRef.namespace: Required value, spec.secretRef.name: Required value, spec.impersonatorSecretRef.namespace: Required value, spec.impersonatorSecretRef.name: Required value]
Error: failed to create cluster(kubernetes-admin) object. error: Cluster.cluster.karmada.io "kubernetes-admin" is invalid: [spec.secretRef.namespace: Required value, spec.secretRef.name: Required value, spec.impersonatorSecretRef.namespace: Required value, spec.impersonatorSecretRef.name: Required value]

kubectl-karmada version：
make kubectl-karmada by the latest codes in github
[root@k8s-master ~]# kubectl-karmada version
kubectl karmada version: version.Info{GitVersion:"", GitCommit:"", GitTreeState:"clean", BuildDate:"2022-01-20T02:30:56Z", GoVersion:"go1.17.6", Compiler:"gc", Platform:"linux/amd64"}

RainbowMango · 2022-01-20T09:07:04Z

Can you tell us the versions of both Karmada and kubectl-karmada?

Karmada version:
kubectl-karmada or karmadactl version (the result of kubectl-karmada version or karmadactl version):

chenrc0529-ai · 2022-01-20T09:09:18Z

Can you tell us the versions of both Karmada and kubectl-karmada?
Karmada version:
kubectl-karmada or karmadactl version (the result of kubectl-karmada version or karmadactl version):

make kubectl-karmada by the latest codes

RainbowMango · 2022-01-20T09:12:34Z

What's the output of kubeclt karmada version?

Which version Karmada do you use?

chenrc0529-ai · 2022-01-20T09:17:40Z

What's the output of kubeclt karmada version?

Which version Karmada do you use?

[root@k8s-master ~]# kubectl karmada version
kubectl karmada version: version.Info{GitVersion:"", GitCommit:"", GitTreeState:"clean", BuildDate:"2022-01-20T02:30:56Z", GoVersion:"go1.17.6", Compiler:"gc", Platform:"linux/amd64"}

XiShanYongYe-Chang · 2022-01-20T09:22:28Z

Hi @chenrc0529-ai , have you ever found the reason why you failed?

chenrc0529-ai · 2022-01-20T09:36:33Z

Hi @chenrc0529-ai , have you ever found the reason why you failed?

Not yet

chenrc0529-ai · 2022-01-21T02:09:43Z

Can you tell us the versions of both Karmada and kubectl-karmada?
Karmada version:
kubectl-karmada or karmadactl version (the result of kubectl-karmada version or karmadactl version):

How can i solve this issue?

RainbowMango · 2022-01-21T02:37:41Z

Still don't know which version you are using.

RainbowMango · 2022-01-21T02:38:16Z

kubectl karmada version: version.Info{GitVersion:"", GitCommit:"", GitTreeState:"clean", BuildDate:"2022-01-20T02:30:56Z", GoVersion:"go1.17.6", Compiler:"gc", Platform:"linux/amd64"}

No version info.

chenrc0529-ai · 2022-01-21T02:59:15Z

kubectl-karmada

I pull the latest codes from github in Jan 13. Because i cannot download the image， so i modify the image source, then make kubectl-karmada in local.

chenrc0529-ai · 2022-01-21T03:20:03Z

kubectl karmada version: version.Info{GitVersion:"", GitCommit:"", GitTreeState:"clean", BuildDate:"2022-01-20T02:30:56Z", GoVersion:"go1.17.6", Compiler:"gc", Platform:"linux/amd64"}

No version info.
I pull the latest codes from github in Jan 13. Because i cannot download the image， so i modify the image source, then make kubectl-karmada in local.

SZT0728 · 2022-01-21T08:58:12Z

kubectl karmada version: version.Info{GitVersion:"", GitCommit:"", GitTreeState:"clean", BuildDate:"2022-01-20T02:30:56Z", GoVersion:"go1.17.6", Compiler:"gc", Platform:"linux/amd64"}

No version info.

i met the same question. i remember that it's can join cluster a few days ago. but now it show the errmsg:
W0121 16:58:03.484629 20954 cluster.go:106] failed to create cluster(kubernetes-admin). error: Internal error occurred: failed calling webhook "cluster.karmada.io": the server could not find the requested resource
W0121 16:58:03.484873 20954 cluster.go:50] failed to create cluster(kubernetes-admin). error: Internal error occurred: failed calling webhook "cluster.karmada.io": the server could not find the requested resource
Error: failed to create cluster(kubernetes-admin) object. error: Internal error occurred: failed calling webhook "cluster.karmada.io": the server could not find the requested resource

XiShanYongYe-Chang · 2022-01-21T09:04:57Z

Hi @SZT0728, the problem you have is different from the one described above. We have already removed cluster.karmada.io validate in webhook with PR #1152.

SZT0728 · 2022-01-21T09:08:53Z

Hi @SZT0728, the problem you have is different from the one described above. We have already removed cluster.karmada.io validate in webhook with PR #1152.

so,i build the kubectl-karmada from the master latest source??

XiShanYongYe-Chang · 2022-01-21T09:09:31Z

@SZT0728 , you can check validatingwebhookconfigurations on your environment:

kubectl get validatingwebhookconfigurations.admissionregistration.k8s.io validating-config -oyaml

Check whether it is the latest,

XiShanYongYe-Chang · 2022-01-21T09:14:35Z

Error from server (NotFound): validatingwebhookconfigurations.admissionregistration.k8s.io "validating-config" not found
how can i solve this?

Get it when you connect to karmada-apiserver, for example: kubectl --context karmada-apiserver get validatingwebhookconfigurations.admissionregistration.k8s.io validating-config -oyaml

SZT0728 · 2022-01-21T09:16:50Z

Error from server (NotFound): validatingwebhookconfigurations.admissionregistration.k8s.io "validating-config" not found
how can i solve this?

Get it when you connect to karmada-apiserver, for example: kubectl --context karmada-apiserver get validatingwebhookconfigurations.admissionregistration.k8s.io validating-config -oyaml

apiVersion: admissionregistration.k8s.io/v1
kind: ValidatingWebhookConfiguration
metadata:
creationTimestamp: "2022-01-21T08:43:13Z"
generation: 1
labels:
app: validating-config
managedFields:

apiVersion: admissionregistration.k8s.io/v1
fieldsType: FieldsV1
fieldsV1:
....
is it the latest???

SZT0728 · 2022-01-21T09:21:21Z

Error from server (NotFound): validatingwebhookconfigurations.admissionregistration.k8s.io "validating-config" not found
how can i solve this?

Get it when you connect to karmada-apiserver, for example: kubectl --context karmada-apiserver get validatingwebhookconfigurations.admissionregistration.k8s.io validating-config -oyaml

now i join the cluster, it show the error like this issue' describe
failed to create cluster(kubernetes). error: Cluster.cluster.karmada.io "kubernetes" is invalid: [spec.secretRef.namespace: Required value, spec.secretRef.name: Required value, spec.impersonatorSecretRef.namespace: Required value, spec.impersonatorSecretRef.name: Required value]
...

prodanlabs · 2022-01-21T09:21:39Z

hi @chenrc0529-ai is your problem solved?

prodanlabs · 2022-01-21T09:32:35Z

hi @SZT0728 Is ectd not persistent and restarted?

SZT0728 · 2022-01-21T09:38:37Z

hi @SZT0728 Is ectd not persistent and restarted?

no, it's normal

chenrc0529-ai · 2022-01-21T09:44:59Z

hi @chenrc0529-ai is your problem solved?

Not yet

prodanlabs · 2022-01-21T09:58:32Z

hi @chenrc0529-ai is your problem solved?

Not yet

Can you take a look at your apiservice?

root@dev-k8s-master01:~# kubectl --kubeconfig /etc/karmada/karmada-apiserver.config get apiservices | grep karmada
v1alpha1.cluster.karmada.io            karmada-system/karmada-aggregated-apiserver   True        9d
v1alpha1.config.karmada.io             Local                                         True        9d
v1alpha1.policy.karmada.io             Local                                         True        9d
v1alpha1.work.karmada.io               Local                                         True        9d
v1alpha2.work.karmada.io               Local                                         True        9d

chenrc0529-ai · 2022-01-24T03:29:06Z

kubectl --kubeconfig /etc/karmada/karmada-apiserver.config get apiservices | grep karmada

[root@k8s-master ~]# kubectl --kubeconfig /etc/karmada/karmada-apiserver.config get apiservices | grep karmada
v1alpha1.cluster.karmada.io karmada-system/karmada-aggregated-apiserver True 4d
v1alpha1.config.karmada.io Local True 4d
v1alpha1.policy.karmada.io Local True 4d
v1alpha1.work.karmada.io Local True 4d
v1alpha2.work.karmada.io Local True 4d

RainbowMango · 2022-01-25T03:22:05Z

Hi @chenrc0529-ai
The error log is similar to #1306. Can you help test it on your side?

chenrc0529-ai · 2022-01-25T09:06:04Z

kubectl karmada join kubernetes-admin --kubeconfig=/etc/karmada/karmada-apiserver.config --cluster-kubeconfig=$HOME/.kube/config

I modify the file pkg/util/secret.go, and rebuild the kubectl-karmada， the error still exist.

Hi @chenrc0529-ai The error log is similar to #1306. Can you help test it on your side?

I modify the file pkg/util/secret.go, and rebuild the kubectl-karmada， the error still exist.

RainbowMango · 2022-01-25T09:14:04Z

Well, @chenrc0529-ai Do you know how reproduce it ? I can help reproduce it on my side.

By the way, does the cluster kubernetes-admin ever joined before?( Is it the first time joining this cluster to Karmada?)

MingZhang-YBPS · 2022-02-11T13:48:25Z

I met similar issue, and the log in karmada-aggregated-apiserver says
W0211 12:58:51.501491 1 dispatcher.go:150] Failed calling webhook, failing closed cluster.karmada.io: failed calling webhook "cluster.karmada.io": the server could not find the requested resource

YuxiJin-tobeyjin · 2022-02-16T04:02:13Z

Same issue with the latest release, while using v0.10.1 there is no such problem

kubectl karmada version
kubectl karmada version: version.Info{GitVersion:"v1.0.1", GitCommit:"ce2267c5d04a5e97368285fd64fdbe5fa285856e", GitTreeState:"clean", BuildDate:"2022-01-21T09:37:08Z", GoVersion:"go1.16.13", Compiler:"gc", Platform:"linux/amd64"}

kubectl config rename-context "kubernetes-admin@cluster.local" "host" --kubeconfig=host-config
Context "kubernetes-admin@cluster.local" renamed to "host".

kubectl karmada join host --cluster-kubeconfig=host-config 
W0216 11:52:16.931388    8410 cluster.go:106] failed to create cluster(host). error: the server could not find the requested resource (post clusters.cluster.karmada.io)
W0216 11:52:16.931859    8410 cluster.go:50] failed to create cluster(host). error: the server could not find the requested resource (post clusters.cluster.karmada.io)
Error: failed to create cluster(host) object. error: the server could not find the requested resource (post clusters.cluster.karmada.io)

Maybe need to wait for a new release? Do we have a plan?

XiShanYongYe-Chang · 2022-02-16T07:16:51Z

Hi @chenrc0529-ai @SZT0728 @chenrc0529-ai @MingZhang-YBPS @YuxiJin-tobeyjin everybody. Can someone provide a full process for reproducing the problem?

I use the v1.0.1 version and make kubectl-karmada on my local site, but didn't reproduce the error.

There are a few questions that need to be confirmed:

Is the cluster added for the first time?
kubectl karmada version
whether the karmada version is upgraded from an earlier version(such as v0.10.1)?
check whether the issue it the same as Deploy karmada-aggregated-apiserver fllowing the upgrading doc but failed. #1214

YuxiJin-tobeyjin · 2022-02-16T07:42:58Z

Is the cluster added for the first time?
// yes

kubectl karmada version
// as follows

kubectl karmada version
kubectl karmada version: version.Info{GitVersion:"v1.0.1", GitCommit:"ce2267c5d04a5e97368285fd64fdbe5fa285856e", GitTreeState:"clean", BuildDate:"2022-01-21T09:37:08Z", GoVersion:"go1.16.13", Compiler:"gc", Platform:"linux/amd64"}

whether the karmada version is upgraded from an earlier version
// no, deploy from scratch, just deploy and undeploy sometimes (undeploy with "bash -x undeploy-karmada.sh ~/.kube/config kubernetes-admin@cluster.local" )

Reproduce:

1、deploy a local k8s cluster

2、bash remote-up-karmada.sh ~/.kube/config kubernetes-admin@cluster.local CLUSTER_IP_ONLY=true

Generating a 2048 bit RSA private key
......................+++
....................................................................................+++
writing new private key to '/root/.karmada/server-ca.key'
-----
Generating a 2048 bit RSA private key
.+++
....................+++
writing new private key to '/root/.karmada/front-proxy-ca.key'
-----
2022/02/16 15:11:12 [INFO] generate received request
2022/02/16 15:11:12 [INFO] received CSR
2022/02/16 15:11:12 [INFO] generating key: rsa-2048
2022/02/16 15:11:13 [INFO] encoded CSR
2022/02/16 15:11:13 [INFO] signed certificate with serial number 370804119998102905444109242487124500837254195160
2022/02/16 15:11:13 [INFO] generate received request
2022/02/16 15:11:13 [INFO] received CSR
2022/02/16 15:11:13 [INFO] generating key: rsa-2048
2022/02/16 15:11:13 [INFO] encoded CSR
2022/02/16 15:11:13 [INFO] signed certificate with serial number 532867452045251919217757510499376049896225951800
namespace/karmada-system created
serviceaccount/karmada-controller-manager created
serviceaccount/karmada-scheduler created
serviceaccount/karmada-webhook created
clusterrole.rbac.authorization.k8s.io/karmada-controller-manager unchanged
clusterrolebinding.rbac.authorization.k8s.io/karmada-controller-manager unchanged
secret/karmada-cert-secret created
secret/kubeconfig created
secret/webhook-cert created
statefulset.apps/etcd created
service/etcd-client created
service/etcd created
wait the etcd ready...
pod/etcd-0 condition met

Apply dynamic rendered apiserver service in /tmp/tmp.gHGDw6kEUh/karmada-apiserver.yaml.
deployment.apps/karmada-apiserver created
service/karmada-apiserver created
wait the karmada-apiserver ready...
pod/karmada-apiserver-786875ff54-9ksf8 condition met

Karmada API Server's IP is: 10.3.68.67, host cluster type is: remote
Cluster "karmada-apiserver" set.
User "karmada-apiserver" set.
Context "karmada-apiserver" created.
deployment.apps/karmada-kube-controller-manager created
deployment.apps/karmada-aggregated-apiserver created
service/karmada-aggregated-apiserver created
wait the karmada-aggregated-apiserver ready...
pod/karmada-aggregated-apiserver-86cb967c86-6wtrq condition met
namespace/karmada-system created
customresourcedefinition.apiextensions.k8s.io/clusteroverridepolicies.policy.karmada.io created
customresourcedefinition.apiextensions.k8s.io/clusterpropagationpolicies.policy.karmada.io created
customresourcedefinition.apiextensions.k8s.io/clusterresourcebindings.work.karmada.io created
customresourcedefinition.apiextensions.k8s.io/overridepolicies.policy.karmada.io created
customresourcedefinition.apiextensions.k8s.io/propagationpolicies.policy.karmada.io created
customresourcedefinition.apiextensions.k8s.io/resourcebindings.work.karmada.io created
customresourcedefinition.apiextensions.k8s.io/resourceinterpreterwebhookconfigurations.config.karmada.io created
customresourcedefinition.apiextensions.k8s.io/serviceexports.multicluster.x-k8s.io created
customresourcedefinition.apiextensions.k8s.io/serviceimports.multicluster.x-k8s.io created
customresourcedefinition.apiextensions.k8s.io/works.work.karmada.io created
mutatingwebhookconfiguration.admissionregistration.k8s.io/mutating-config created
validatingwebhookconfiguration.admissionregistration.k8s.io/validating-config created
apiservice.apiregistration.k8s.io/v1alpha1.cluster.karmada.io created
service/karmada-aggregated-apiserver created
wait the karmada-aggregated-apiserver Available...
apiservice.apiregistration.k8s.io/v1alpha1.cluster.karmada.io condition met
Switched to context "kubernetes-admin@cluster.local".
deployment.apps/karmada-controller-manager created
deployment.apps/karmada-scheduler created
deployment.apps/karmada-webhook created
service/karmada-webhook created
wait the karmada-controller-manager ready...
pod/karmada-controller-manager-b754b59d7-94wt6 condition met
wait the karmada-scheduler ready...
pod/karmada-scheduler-c48f85d94-zfslp condition met
wait the kube-controller-manager ready...
pod/karmada-kube-controller-manager-6758ff89b-bt8bc condition met
wait the karmada-webhook ready...
pod/karmada-webhook-85d564b96-ps5qs condition met

Karmada is installed.

Kubeconfig for karmada in file: /root/.kube/config, so you can run:
  export KUBECONFIG="/root/.kube/config"
Or use kubectl with --kubeconfig=/root/.kube/config
Please use 'kubectl config use-context karmada-apiserver' to switch the cluster of karmada control plane
And use 'kubectl config use-context kubernetes-admin@cluster.local' for debugging karmada installation

3、kubectl config rename-context "kubernetes-admin@cluster.local" "host" --kubeconfig=host-config

Context "kubernetes-admin@cluster.local" renamed to "host".

4、kubectl karmada join host --cluster-kubeconfig=host-config

W0216 15:13:55.964392   18057 cluster.go:106] failed to create cluster(host). error: the server could not find the requested resource (post clusters.cluster.karmada.io)
W0216 15:13:55.964561   18057 cluster.go:50] failed to create cluster(host). error: the server could not find the requested resource (post clusters.cluster.karmada.io)
Error: failed to create cluster(host) object. error: the server could not find the requested resource (post clusters.cluster.karmada.io)

Hope this helps.

XiShanYongYe-Chang · 2022-02-16T08:46:52Z

Hi @YuxiJin-tobeyjin, I follow your step, it works ok on my side.

Can you get clusters correctly with the command:

kubectl get clusters --v=6

YuxiJin-tobeyjin · 2022-02-16T10:43:25Z

kubectl config use-context  karmada-apiserver 
Switched to context "karmada-apiserver".

kubectl get clusters --v=6
I0216 18:42:18.105326   21340 loader.go:375] Config loaded from file:  /root/.kube/config
I0216 18:42:18.182650   21340 round_trippers.go:443] GET https://10.3.68.67:5443/apis/cluster.karmada.io/v1alpha1/clusters?limit=500 200 OK in 54 milliseconds
No resources found

XiShanYongYe-Chang · 2022-02-16T11:18:14Z

kubectl config use-context karmada-apiserver
Switched to context "karmada-apiserver".

Is this context not used when you join a cluster?

YuxiJin-tobeyjin · 2022-02-16T13:45:17Z

Yes, of course i did, the logs are as follows:

kubectl config use-context karmada-apiserver
Switched to context "karmada-apiserver".

kubectl karmada join host --cluster-kubeconfig=host-config 
W0216 14:18:22.478715    8192 cluster.go:106] failed to create cluster(host). error: Internal error occurred: failed calling webhook "cluster.karmada.io": the server could not find the requested resource
W0216 14:18:22.480569    8192 cluster.go:50] failed to create cluster(host). error: Internal error occurred: failed calling webhook "cluster.karmada.io": the server could not find the requested resource
Error: failed to create cluster(host) object. error: Internal error occurred: failed calling webhook "cluster.karmada.io": the server could not find the requested resource

Maybe the reason is validating-config, as mentioned above i've deployed and undeployed karmada many times, but it seems validating-config keeps unchanged, i will try it again tomorrow

kubectl get validatingwebhookconfigurations.admissionregistration.k8s.io validating-config
NAME                WEBHOOKS   AGE
validating-config   6          5d18h

XiShanYongYe-Chang · 2022-02-17T01:44:05Z

The error info is different from before.

Maybe check the logs for the 'Karmada-webhook' pod to see what's going on.

wenhuwang · 2022-02-22T05:48:15Z

Yes, of course i did, the logs are as follows:

kubectl config use-context karmada-apiserver
Switched to context "karmada-apiserver".

kubectl karmada join host --cluster-kubeconfig=host-config 
W0216 14:18:22.478715    8192 cluster.go:106] failed to create cluster(host). error: Internal error occurred: failed calling webhook "cluster.karmada.io": the server could not find the requested resource
W0216 14:18:22.480569    8192 cluster.go:50] failed to create cluster(host). error: Internal error occurred: failed calling webhook "cluster.karmada.io": the server could not find the requested resource
Error: failed to create cluster(host) object. error: Internal error occurred: failed calling webhook "cluster.karmada.io": the server could not find the requested resource

Maybe the reason is validating-config, as mentioned above i've deployed and undeployed karmada many times, but it seems validating-config keeps unchanged, i will try it again tomorrow

kubectl get validatingwebhookconfigurations.admissionregistration.k8s.io validating-config
NAME                WEBHOOKS   AGE
validating-config   6          5d18h

@YuxiJin-tobeyjin I encountered the same problem，it can be solved by doing the following:

delete the following configuration in validating-config validatingwebhookconfigurations resource：

- admissionReviewVersions:
  - v1
  clientConfig:
    caBundle: *****
    url: https://karmada-webhook.karmada-system.svc:443/validate-cluster
  failurePolicy: Fail
  matchPolicy: Equivalent
  name: cluster.karmada.io
  namespaceSelector: {}
  objectSelector: {}
  rules:
  - apiGroups:
    - cluster.karmada.io
    apiVersions:
    - '*'
    operations:
    - CREATE
    - UPDATE
    resources:
    - clusters
    scope: Cluster
  sideEffects: None
  timeoutSeconds: 3

Clean up the secret resource corresponding to the cluster

# kubectl -n karmada-cluster get secret
NAME                        TYPE     DATA   AGE
cluster1                    Opaque   2      20h
cluster1-impersonator       Opaque   1      20h

join cluster to karmada.

YuxiJin-tobeyjin · 2022-02-24T03:15:30Z

Thanks a lot @wenhuwang , problem solves.
@XiShanYongYe-Chang , so the cluster.karmada.io webhook-configuration needs to be deleted from the deploy files?

XiShanYongYe-Chang · 2022-02-24T03:32:26Z

We remove cluster.karmada.io webhook-configuration with PR #1152, If you use the master branch, it is needed. But with release v1.0.1, we haven't merged it.

Is there something wrong?

XiShanYongYe-Chang · 2022-03-02T06:59:56Z

In conclusion, two causes are found to cause the failure to join the cluster:

Exist cluster validation in validating-config validatingwebhookconfigurations: Join cluster error #1291 (comment)
The available of v1alpha1.cluster.karmada.io apiservice is False: fix: karmada join failed when secret exist #1306 (comment)

XiShanYongYe-Chang · 2022-03-04T06:34:25Z

Close first, we can reopen it if anyone occurs this problem again.

/close

karmada-bot · 2022-03-04T06:34:28Z

@XiShanYongYe-Chang: Closing this issue.

In response to this:

Close first, we can reopen it if anyone occurs this problem again.

/close

Instructions for interacting with me using PR comments are available here. If you have questions or suggestions related to my behavior, please file an issue against the kubernetes/test-infra repository.

chenrc0529-ai added the kind/question Indicates an issue that is a support question. label Jan 20, 2022

chenrc0529-ai closed this as completed Jan 20, 2022

chenrc0529-ai reopened this Jan 21, 2022

RainbowMango mentioned this issue Jan 25, 2022

fix: karmada join failed when secret exist #1306

Merged

zach593 mentioned this issue Feb 9, 2022

fixes gitVersion is missing from the latest image issue #1345

Merged

zach593 mentioned this issue Feb 14, 2022

fix default image tags for <karmadactl/kubectl-karmada> init #1358

Closed

karmada-bot closed this as completed Mar 4, 2022

Join cluster error #1291

Join cluster error #1291

Comments

chenrc0529-ai commented Jan 20, 2022 • edited

RainbowMango commented Jan 20, 2022

chenrc0529-ai commented Jan 20, 2022

RainbowMango commented Jan 20, 2022

chenrc0529-ai commented Jan 20, 2022

XiShanYongYe-Chang commented Jan 20, 2022

chenrc0529-ai commented Jan 20, 2022

chenrc0529-ai commented Jan 21, 2022

RainbowMango commented Jan 21, 2022

RainbowMango commented Jan 21, 2022

chenrc0529-ai commented Jan 21, 2022

chenrc0529-ai commented Jan 21, 2022

SZT0728 commented Jan 21, 2022

XiShanYongYe-Chang commented Jan 21, 2022

SZT0728 commented Jan 21, 2022

XiShanYongYe-Chang commented Jan 21, 2022

XiShanYongYe-Chang commented Jan 21, 2022

SZT0728 commented Jan 21, 2022

SZT0728 commented Jan 21, 2022

prodanlabs commented Jan 21, 2022

prodanlabs commented Jan 21, 2022

SZT0728 commented Jan 21, 2022

chenrc0529-ai commented Jan 21, 2022

prodanlabs commented Jan 21, 2022

chenrc0529-ai commented Jan 24, 2022

RainbowMango commented Jan 25, 2022

chenrc0529-ai commented Jan 25, 2022

RainbowMango commented Jan 25, 2022

MingZhang-YBPS commented Feb 11, 2022

YuxiJin-tobeyjin commented Feb 16, 2022

XiShanYongYe-Chang commented Feb 16, 2022

YuxiJin-tobeyjin commented Feb 16, 2022

XiShanYongYe-Chang commented Feb 16, 2022

YuxiJin-tobeyjin commented Feb 16, 2022

XiShanYongYe-Chang commented Feb 16, 2022 • edited

YuxiJin-tobeyjin commented Feb 16, 2022 • edited

XiShanYongYe-Chang commented Feb 17, 2022

wenhuwang commented Feb 22, 2022 • edited

YuxiJin-tobeyjin commented Feb 24, 2022

XiShanYongYe-Chang commented Feb 24, 2022

XiShanYongYe-Chang commented Mar 2, 2022

XiShanYongYe-Chang commented Mar 4, 2022

karmada-bot commented Mar 4, 2022

chenrc0529-ai commented Jan 20, 2022 •

edited

XiShanYongYe-Chang commented Feb 16, 2022 •

edited

YuxiJin-tobeyjin commented Feb 16, 2022 •

edited

wenhuwang commented Feb 22, 2022 •

edited