ktctl connect 使用后dns解析等并没有走到集群内

[xinyangbest]

Describe the bug
2019/08/23 15:35:11 Client address 10.0.0.12
2019/08/23 15:35:11 Deploying proxy deployment kt-connect-daemon-tztcp in namespace default
2019/08/23 15:35:12 Pods not ready......
2019/08/23 15:35:14 Pod status is Pending
2019/08/23 15:35:16 Pod status is Running
2019/08/23 15:35:16 Success deploy proxy deployment kt-connect-daemon-tztcp in namespace default
Forwarding from 127.0.0.1:2222 -> 22
2019/08/23 15:35:18 port-forward start at pid: 3019
Daemon Start At 2777

集群内pod 访问可以
root@php-test-7cddb54698-j6tn9:~# curl php-test.default.svc.cluster.local:18306 | grep head
% Total % Received % Xferd Average Speed Time Time Time Current
Dload Upload Total Spent Left Speed
100 4036 100 4036 0 0 622k 0 --:--:-- --:--:-- --:--:-- 656k

article, aside, dialog, figcaption, figure, footer, header, hgroup, main, nav, section {

机器上执行
[root@VM_0_12_centos ~]# curl php-test.default.svc.cluster.local:18306

curl: (6) Could not resolve host: php-test.default.svc.cluster.local; Unknown error

Environment (please complete the following information):

• OS: [e.g. Linux VM_0_12_centos 3.10.0-862.9.1.el7.x86_64 镜像获取的时候特别慢 #1 SMP Mon Jul 16 16:29:36 UTC 2018 x86_64 x86_64 x86_64 GNU/Linux]
• Kubernetes [v1.12.4-tke.4]
• KT Version [e.g. 0.0.4]

Additional context
使用后dns解析等并没有走到集群内

[yunlzheng]

@xinyangbest 麻烦提供了一下加上-d的日志输出，里面能看到所有代理的子网

[xinyangbest]

没有-d这个参数
ktctl connect -d
Incorrect Usage: flag provided but not defined: -d

NAME:
ktctl connect - connection to kubernetes cluster

USAGE:
ktctl connect [command options] [arguments...]

OPTIONS:
--port value Local SSH Proxy port (default: 2222)
--disableDNS Disable Cluster DNS
--cidr value Custom CIDR eq '172.2.0.0/16'

2019/09/11 16:03:03 flag provided but not defined: -d

[yunlzheng]

ktctl -d connect

[xinyangbest]

[root@VM_0_12_centos ~]# ktctl -d connect
2019/09/12 10:43:36 Client address 10.0.0.12
2019/09/12 10:43:36 Deploying proxy deployment kt-connect-daemon-jpzmb in namespace default
2019/09/12 10:43:36 Pods not ready......
2019/09/12 10:43:38 Pod status is Pending
2019/09/12 10:43:40 Pod status is Running
2019/09/12 10:43:40 Success deploy proxy deployment kt-connect-daemon-jpzmb in namespace default
2019/09/12 10:43:40 Child, os.Args = [ktctl -d connect]
2019/09/12 10:43:40 Child, cmd.Args = [kubectl --kubeconfig=/root/.kube/config -n default port-forward deployments/kt-connect-daemon-jpzmb 2222:22]
Forwarding from 127.0.0.1:2222 -> 22
2019/09/12 10:43:42 port-forward start at pid: 1250
2019/09/12 10:43:47 Child, os.Args = [ktctl -d connect]
2019/09/12 10:43:47 Child, cmd.Args = [sshuttle --dns --to-ns 172.16.0.201 -e ssh -oStrictHostKeyChecking=no -oUserKnownHostsFile=/dev/null -i /tmp/kt_id_rsa -r root@127.0.0.1:2222 -x 127.0.0.1 172.16.0.0/24 172.16.0.0/16]
Daemon Start At 977

[root@VM_0_12_centos ~]# nslookup php-test.default.svc.cluster.local
Server: 183.60.83.19
Address: 183.60.83.19#53

** server can't find php-test.default.svc.cluster.local: NXDOMAIN

[root@VM_0_12_centos ~]# nslookup php-test.default.svc.cluster.local 172.16.255.250
Server: 172.16.255.250
Address: 172.16.255.250#53

Name: php-test.default.svc.cluster.local
Address: 172.16.255.251

[chenliang]

我也遇到这个问题了， 最后发现是 sshuttle 未安装，我看你的启动日志没有
2:25PM DBG vpn(sshuttle) start at pid: 76425
可能应该是sshuttle未安装

[xinyangbest]

之前因为网络原因 安装pip install sshuttle 失败了 后来忘记了 尴尬

[ysjjovo]

我在wsl2上也遇到这个问题，访问podIp,clusterIp都正常，DNS不行。日志如下：

➜   ktctl -d connect
11:36AM INF KtConnect 0.2.5 start at 9216 (linux)
11:36AM DBG Private Key generated
11:36AM DBG Public key generated
11:36AM INF Successful create config map kt-connect-shadow-pkefb
11:36AM DBG Client address 172.22.90.43
11:36AM INF Deploying shadow pod kt-connect-shadow-pkefb in namespace default
11:36AM INF Waiting for pod kt-connect-shadow-pkefb ...
11:36AM INF Pod kt-connect-shadow-pkefb is ready
11:36AM INF Node has empty PodCIDR, try to get CIDR with pod sample
11:36AM DBG Pod CIDR is [10.1.0.0/24 192.168.65.4/32 192.168.65.4/32 192.168.65.4/32 192.168.65.4/32 192.168.65.4/32]
11:36AM DBG Service CIDR is [10.99.223.161/32 10.108.235.247/32 10.101.24.220/32 10.106.82.151/32 10.111.60.7/32 10.96.0.0/24 10.98.0.0/16]
11:36AM DBG Request port forward pod:22 -> local:2222 via https://kubernetes.docker.internal:6443
11:36AM DBG Waiting for port forward (dial tcp :2222: connect: connection refused), retry: 1
Forwarding from 127.0.0.1:2222 -> 22
Forwarding from [::1]:2222 -> 22
11:36AM INF Port forward connection established
Handling connection for 2222
11:36AM DBG Child, os.Args = [ktctl -d connect]
11:36AM DBG Child, name = vpn(sshuttle), cmd.Args = [sshuttle --dns --to-ns 10.1.0.179 --verbose --ssh-cmd ssh -oStrictHostKeyChecking=no -oUserKnownHostsFile=/dev/null -i /root/.ktctl/pk/kt-connect-shadow-pkefb_id_rsa --remote root@127.0.0.1:2222 --exclude 127.0.0.1 10.1.0.0/24 192.168.65.4/32 192.168.65.4/32 192.168.65.4/32 192.168.65.4/32 192.168.65.4/32 10.99.223.161/32 10.108.235.247/32 10.101.24.220/32 10.106.82.151/32 10.111.60.7/32 10.96.0.0/24 10.98.0.0/16]
Starting sshuttle proxy (version 1.1.0).
c : Starting firewall manager with command: ['/usr/bin/python3', '/usr/local/bin/sshuttle', '-v', '--method', 'auto', '--firewall']
fw: Starting firewall with Python version 3.6.9
fw: ready method name nat.
c : IPv6 enabled: Using default IPv6 listen address ::1
c : Method: nat
c : IPv4: on
c : IPv6: on
c : UDP : off (not available with nat method)
c : DNS : on
c : User: off (available)
c : Subnets to forward through remote host (type, IP, cidr mask width, startPort, endPort):
c :   (<AddressFamily.AF_INET: 2>, '10.1.0.0', 24, 0, 0)
c :   (<AddressFamily.AF_INET: 2>, '192.168.65.4', 32, 0, 0)
c :   (<AddressFamily.AF_INET: 2>, '192.168.65.4', 32, 0, 0)
c :   (<AddressFamily.AF_INET: 2>, '192.168.65.4', 32, 0, 0)
c :   (<AddressFamily.AF_INET: 2>, '192.168.65.4', 32, 0, 0)
c :   (<AddressFamily.AF_INET: 2>, '192.168.65.4', 32, 0, 0)
c :   (<AddressFamily.AF_INET: 2>, '10.99.223.161', 32, 0, 0)
c :   (<AddressFamily.AF_INET: 2>, '10.108.235.247', 32, 0, 0)
c :   (<AddressFamily.AF_INET: 2>, '10.101.24.220', 32, 0, 0)
c :   (<AddressFamily.AF_INET: 2>, '10.106.82.151', 32, 0, 0)
c :   (<AddressFamily.AF_INET: 2>, '10.111.60.7', 32, 0, 0)
c :   (<AddressFamily.AF_INET: 2>, '10.96.0.0', 24, 0, 0)
c :   (<AddressFamily.AF_INET: 2>, '10.98.0.0', 16, 0, 0)
c : Subnets to exclude from forwarding:
c :   (<AddressFamily.AF_INET: 2>, '127.0.0.1', 32, 0, 0)
c :   (<AddressFamily.AF_INET: 2>, '127.0.0.1', 32, 0, 0)
c :   (<AddressFamily.AF_INET6: 10>, '::1', 128, 0, 0)
c : DNS requests normally directed at these servers will be redirected to remote:
c :   (<AddressFamily.AF_INET: 2>, '172.22.80.1')
c : TCP redirector listening on ('::1', 12300, 0, 0).
c : TCP redirector listening on ('127.0.0.1', 12300).
c : DNS listening on ('::1', 12299, 0, 0).
c : DNS listening on ('127.0.0.1', 12299).
11:36AM INF Start vpn(sshuttle) at pid: 9227
11:36AM INF All looks good, now you can access to resources in the kubernetes cluster
c : Starting client with Python version 3.6.9
c : Connecting to server...
Handling connection for 2222
Warning: Permanently added '[127.0.0.1]:2222' (ECDSA) to the list of known hosts.
 s: Running server on remote host with /usr/bin/python3 (version 3.8.10)
 s: latency control setting = True
 s: auto-nets:False
c : Connected to server.
fw: setting up.
fw: ip6tables -w -t nat -N sshuttle-12300
fw: ip6tables -w -t nat -F sshuttle-12300
fw: ip6tables -w -t nat -I OUTPUT 1 -j sshuttle-12300
fw: ip6tables -w -t nat -I PREROUTING 1 -j sshuttle-12300
fw: ip6tables -w -t nat -A sshuttle-12300 -j RETURN -m addrtype --dst-type LOCAL
fw: ip6tables -w -t nat -A sshuttle-12300 -j RETURN --dest ::1/128 -p tcp
fw: iptables -w -t nat -N sshuttle-12300
fw: iptables -w -t nat -F sshuttle-12300
fw: iptables -w -t nat -I OUTPUT 1 -j sshuttle-12300
fw: iptables -w -t nat -I PREROUTING 1 -j sshuttle-12300
fw: iptables -w -t nat -A sshuttle-12300 -j REDIRECT --dest 172.22.80.1 -p udp --dport 53 --to-ports 12299
fw: iptables -w -t nat -A sshuttle-12300 -j RETURN -m addrtype --dst-type LOCAL
fw: iptables -w -t nat -A sshuttle-12300 -j RETURN --dest 127.0.0.1/32 -p tcp
fw: iptables -w -t nat -A sshuttle-12300 -j RETURN --dest 127.0.0.1/32 -p tcp
fw: iptables -w -t nat -A sshuttle-12300 -j REDIRECT --dest 192.168.65.4/32 -p tcp --to-ports 12300
fw: iptables -w -t nat -A sshuttle-12300 -j REDIRECT --dest 192.168.65.4/32 -p tcp --to-ports 12300
fw: iptables -w -t nat -A sshuttle-12300 -j REDIRECT --dest 192.168.65.4/32 -p tcp --to-ports 12300
fw: iptables -w -t nat -A sshuttle-12300 -j REDIRECT --dest 192.168.65.4/32 -p tcp --to-ports 12300
fw: iptables -w -t nat -A sshuttle-12300 -j REDIRECT --dest 192.168.65.4/32 -p tcp --to-ports 12300
fw: iptables -w -t nat -A sshuttle-12300 -j REDIRECT --dest 10.99.223.161/32 -p tcp --to-ports 12300
fw: iptables -w -t nat -A sshuttle-12300 -j REDIRECT --dest 10.108.235.247/32 -p tcp --to-ports 12300
fw: iptables -w -t nat -A sshuttle-12300 -j REDIRECT --dest 10.101.24.220/32 -p tcp --to-ports 12300
fw: iptables -w -t nat -A sshuttle-12300 -j REDIRECT --dest 10.106.82.151/32 -p tcp --to-ports 12300
fw: iptables -w -t nat -A sshuttle-12300 -j REDIRECT --dest 10.111.60.7/32 -p tcp --to-ports 12300
fw: iptables -w -t nat -A sshuttle-12300 -j REDIRECT --dest 10.1.0.0/24 -p tcp --to-ports 12300
fw: iptables -w -t nat -A sshuttle-12300 -j REDIRECT --dest 10.96.0.0/24 -p tcp --to-ports 12300
fw: iptables -w -t nat -A sshuttle-12300 -j REDIRECT --dest 10.98.0.0/16 -p tcp --to-ports 12300
sd_bus_open_system: No such file or directory
fw: Received non-zero return code 1 when flushing DNS resolver cache.
11:37AM DBG Heartbeat port forward 2222 ticked at 2022-02-12 11:37:55
Handling connection for 2222
11:38AM DBG Heartbeat port forward 2222 ticked at 2022-02-12 11:38:53
Handling connection for 2222
Handling connection for 2222
11:39AM DBG Heartbeat port forward 2222 ticked at 2022-02-12 11:39:51

其中有类似这样的错。可能和wsl2有关

sd_bus_open_system: No such file or directory
fw: Received non-zero return code 1 when flushing DNS resolver cache.