can not generate the iptables rules

[pengbinbin1]

I deploy edmesh-agent-edge on my edge server,and the log is like the image below,but there are no iptables rules generate about the edgemesh . the port 53 is not listening .

[pengbinbin1]

[Poorunga]

kubectl apply -f build/crds/istio/

[pengbinbin1]

kubectl apply -f build/crds/istio/
I have executed the command “kubectl apply -f build/crds/istio/”. I deployed the edgemesh-agent-cloud on the cloud nodoe and the logs are normal, but the iptables rules about EDGE-MESH are still not generated . On the edge node ,i deployed the edgemesh-agent-edge ,the log is the same like the image above, and I can not understand what is the connection between CRDs and iptables rules. When i exec the command "netstat -tunlp |grep 53" on the cloud node,there is an edgemesh-ag processing listen the 53 port, but when i exec the same command on the edge node,there is no processing listen the port 53,i need you help ,thanks

[Poorunga]

Is your OS centos8? You can refer to #30

[Poorunga]

/assign

[pengbinbin1]

Is your OS centos8? You can refer to #30

My OS is centos8, i have deployed the kubeedge 1.7 in another server , but the OS version is same. In the server where i deployed the kubeedge 1.7 , i can use command iptables to see the iptables rules about EDGE-MESH

[Poorunga]

The old version of edgemesh code is written in kubeedge and runs as a process. It will directly configure the ipatbles rules of your host, so you can see it directly on the host. The new version of edgemesh is extracted from kubeedge as a separate module and runs as a container. If you are using centos8, the iptables rules are only visible in the container. Why is there such a situation please read carefully #30 I mentioned above.
If you are using a centos8 system (what a coincidence, I am also using this system), and deploy edgemesh in a container, in fact ipatbles is already in effect (visible in the container), but you can't see it. But you can still see the effective ipatbles rules by entering the edgemesh-agent container:

$ docker exec -it <edgemesh-container-id>
// enter container env
# iptables -t nat -nvL

Then you can see the iptables rules created by edgemesh-agent.

(By the way, if your machine has previously deployed an old version of kubeedge and enabled the edgeMesh module, then it will have residual iptables rules on your machine: EDGE-MESH chain)

====== 中文翻译 ======
老版本的edgemesh代码写在kubeedge中，以进程运行，会直接配置你主机的ipatbles规则，所以直接在主机上可以看到。新版本的edgemesh作为单独的模块从kubeedge中抽出来了，以容器运行，如果你用的是centos8那么iptables规则仅在容器内可见。为什么会有这种情况请仔细阅读我上面提到的#30。
如果你是centos8系统（真巧啊，我也在用这个系统），并且以容器的方式部署edgemesh，其实ipatbles已经生效了（容器内可见），只不过你看不到罢了。但是你还是可以通过进入edgemesh-agent容器内看到生效的ipatbles规则：

$ docker exec -it <edgemesh-container-id>
// 进入容器环境后
# iptables -t nat -nvL

然后你就可以看到edgemesh-agent创建的iptables规则了。

（顺带一提，如果你的机器之前部署过老版本的kubeedge并且开启了edgeMesh模块，那么它可能会在你的机器上有残留的iptables规则：EDGE-MESH链）

[pengbinbin1]

The old version of edgemesh code is written in kubeedge and runs as a process. It will directly configure the ipatbles rules of your host, so you can see it directly on the host. The new version of edgemesh is extracted from kubeedge as a separate module and runs as a container. If you are using centos8, the iptables rules are only visible in the container. Why is there such a situation please read carefully #30 I mentioned above. If you are using a centos8 system (what a coincidence, I am also using this system), and deploy edgemesh in a container, in fact ipatbles is already in effect (visible in the container), but you can't see it. But you can still see the effective ipatbles rules by entering the edgemesh-agent container:

$ docker exec -it <edgemesh-container-id>
// enter container env
# iptables -t nat -nvL

Then you can see the iptables rules created by edgemesh-agent.

(By the way, if your machine has previously deployed an old version of kubeedge and enabled the edgeMesh module, then it will have residual iptables rules on your machine: EDGE-MESH chain)

====== 中文翻译 ====== 老版本的edgemesh代码写在kubeedge中，以进程运行，会直接配置你主机的ipatbles规则，所以直接在主机上可以看到。新版本的edgemesh作为单独的模块从kubeedge中抽出来了，以容器运行，如果你用的是centos8那么iptables规则仅在容器内可见。为什么会有这种情况请仔细阅读我上面提到的#30。 如果你是centos8系统（真巧啊，我也在用这个系统），并且以容器的方式部署edgemesh，其实ipatbles已经生效了（容器内可见），只不过你看不到罢了。但是你还是可以通过进入edgemesh-agent容器内看到生效的ipatbles规则：

$ docker exec -it <edgemesh-container-id>
// 进入容器环境后
# iptables -t nat -nvL

然后你就可以看到edgemesh-agent创建的iptables规则了。

（顺带一提，如果你的机器之前部署过老版本的kubeedge并且开启了edgeMesh模块，那么它会在你的机器上有残留的iptables规则：EDGE-MESH链）
Thank you very much! I can see the iptbles rules in the container. But i have another question,in my edge node,why the /etc/resolve.conf is empty,and no process listening the port 53? I have configed the DNS module .

[Poorunga]

Check out edgemesh-agent log:

$ docker logs <edgemesh-agent-contaier-id>
2021/11/01 06:55:40 INFO: Install client plugin, protocol: rest
2021/11/01 06:55:40 INFO: Installed service discovery plugin: edge
I1101 06:55:40.411916       1 server.go:303] edgemesh-agent running on EdgeMode
I1101 06:55:40.412164       1 server.go:64] Version: 1.8.0-48+740cfa8598ed78
I1101 06:55:40.412187       1 server.go:98] [1] New informers manager
I1101 06:55:40.412482       1 server.go:105] [2] Prepare agent to run
I1101 06:55:40.455543       1 server.go:111] [3] Register beehive modules
I1101 06:55:40.455707       1 module.go:34] Module edgedns registered successfully

You will see Module edgedns registered successfully

Check out port 53:

$ netstat -anp | grep 53 | grep edgemesh
udp        0      0 169.254.96.16:53        0.0.0.0:*                           23752/edgemesh-agen

Or

$ lsof -i:53 | grep edgemesh
edgemesh- 23752            root   26u  IPv4 4080641      0t0  UDP ke-edge1:domain

[pengbinbin1]

Check out edgemesh-agent log:

$ docker logs <edgemesh-agent-contaier-id>
2021/11/01 06:55:40 INFO: Install client plugin, protocol: rest
2021/11/01 06:55:40 INFO: Installed service discovery plugin: edge
I1101 06:55:40.411916       1 server.go:303] edgemesh-agent running on EdgeMode
I1101 06:55:40.412164       1 server.go:64] Version: 1.8.0-48+740cfa8598ed78
I1101 06:55:40.412187       1 server.go:98] [1] New informers manager
I1101 06:55:40.412482       1 server.go:105] [2] Prepare agent to run
I1101 06:55:40.455543       1 server.go:111] [3] Register beehive modules
I1101 06:55:40.455707       1 module.go:34] Module edgedns registered successfully

You will see Module edgedns registered successfully

Check out port 53:

$ netstat -anp | grep 53 | grep edgemesh
udp        0      0 169.254.96.16:53        0.0.0.0:*                           23752/edgemesh-agen

Or

$ lsof -i:53 | grep edgemesh
edgemesh- 23752            root   26u  IPv4 4080641      0t0  UDP ke-edge1:domain

the log out is like the image below, The edgemesh-agent-edge image is pulled from dockerhub,the version is 1.8.0 and I find that the log about version is 1.7.2

[Poorunga]

Is the image ID correct?

kubeedge/edgemesh-agent                 v1.8.0              0229a8131577        2 months ago        66.2MB

0229a8131577 is image ID

[pengbinbin1]

Is the image ID correct?

kubeedge/edgemesh-agent                 v1.8.0              0229a8131577        2 months ago        66.2MB

0229a8131577 is image ID

YES,the image ID is 0229a8131577

[Poorunga]

the version is 1.8.0 and I find that the log about version is 1.7.2

I don't think this will cause any problems.

Maybe these errors are the cause of the problem: Failed to watch xxx...

Is edge list-watch normal? try curl 127.0.0.1:10550/api/v1/services

[pengbinbin1]

the version is 1.8.0 and I find that the log about version is 1.7.2

I don't think this will cause any problems.

Maybe these errors are the cause of the problem: Failed to watch xxx...

Is edge list-watch normal? try curl 127.0.0.1:10550/api/v1/services

[Poorunga]

Will the list-watch request be slow? Will it be longer than 5s?

By the way, is your cloudcore container started or process started?

[pengbinbin1]

First,the list-watch is quickely,it is not longer than 1s. And i checkout the cloudcore log ,the log is like the image below. The cloudcore image is v1.8.0 from dockerhub, image ID is a1417a4b5c05,and why the iptables excutable can not found?

[Poorunga]

Someone encountered the same problem as you, and we have solved it, please refer to: #93

The main thing is to modify the serviceaccount of cloudcore

[pengbinbin1]

Someone encountered the same problem as you, and we have solved it, please refer to: #93

The main thing is to modify the serviceaccount of cloudcore
OK，thank you very much, i will try it tommrow. And you mean if the cloudcore can start normaly, then the edgemesh-agent-edge can run normaly?

[Poorunga]

you mean if the cloudcore can start normaly, then the edgemesh-agent-edge can run normaly?

Yep, the edge list-watch relies on cloudcore's dynamicController. You only need to reconfigure the clusterrole of cloudcore, and then restart cloudcore.

kubectl edit clusterrole cloudcore
----
- apiGroups:
  - networking.istio.io
  resources:
  - '*'
  verbs:
  - get
  - list
  - watch

[pengbinbin1]

you mean if the cloudcore can start normaly, then the edgemesh-agent-edge can run normaly?

Yep, the edge list-watch relies on cloudcore's dynamicController. You only need to reconfigure the clusterrole of cloudcore, and then restart cloudcore.

kubectl edit clusterrole cloudcore
----
- apiGroups:
  - networking.istio.io
  resources:
  - '*'
  verbs:
  - get
  - list
  - watch

Thank you very much,now the 53 port is listening by edgemesh-ag on my edge node,but when i curl myservice uese servicename, the dns module in edgemesh-agent logs can not found my service,and i found the cloudcore has some error logs,the log is like the blow image

[pengbinbin1]

[pengbinbin1]

I have found some error logs from edgecore "failed to list obj from cloud"

[Poorunga]

Have you edited cloudcore clusterrole, and restart it?

[pengbinbin1]

Yes, I have edited cloudcore clusterrole and restart the cloucore.

[Poorunga]

Is the image you use kubeedge/edgemesh-agent:latest?

[pengbinbin1]

Yes, I have edited cloudcore clusterrole and restart the cloucore.

Is the image you use kubeedge/edgemesh-agent:latest?

I will try the latest one

[Poorunga]

If you want to use latest, you need delete edgemesh serviceaccount, clusterrole, clusterrolebind, configmap, daemonset, deployment, secret, then re-create them.

$ kubectl get cm -n kubeedge | grep edgemesh
edgemesh-agent-cfg    1      3h32m
edgemesh-server-cfg   1      3h32m

$ kubectl get secret -n kubeedge | grep edgemeshaddr
edgemeshaddrsecret            Opaque                                4      15d

and so on...

[pengbinbin1]

And i have to cheange the version to least? I used v1.8.0 agemesh-agent now,the error log is like the image

[Poorunga]

This is not an error, your edgemesh-agent is running normally

[pengbinbin1]

This is not an error, your edgemesh-agent is running normally

But when i curl my service on the edge node use the service name ,it logs can not resolve the servicename.And the edgemesh-agent logs "mysvc1" not found

[Poorunga]

Try to directly access the cluster IP of the service, example:

curl 10.98.12.1:8080

[pengbinbin1]

Try to directly access the cluster IP of the service, example:

curl 10.98.12.1:8080

Thanks for your patience and support, now my edgemesh is running normally,and i can curl my service use servicename,thank you very much

[Poorunga]

Congratulations~👍