Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

anget node working but access fail on anget node #10109

Closed
1369521908 opened this issue May 16, 2024 · 3 comments
Closed

anget node working but access fail on anget node #10109

1369521908 opened this issue May 16, 2024 · 3 comments

Comments

@1369521908
Copy link

1369521908 commented May 16, 2024 •
edited by brandond

Environmental Info:
K3s Version:
k3s version v1.30.0+k3s1 (1454953)
go version go1.22.2

Node(s) CPU architecture, OS, and Version:
Linux server-01 5.4.0-181-generic #201-Ubuntu SMP Thu Mar 28 15:39:01 UTC 2024 x86_64 x86_64 x86_64 GNU/Linux

Cluster Configuration:
1 servers, 1 agents

Describe the bug:
anget node working but access fail on anget node

Steps To Reproduce:

  • Installed K3s:
    use autok3s to install cluster

Expected behavior:
able to access logs on all pod

Actual behavior:
cant access on some pod

Additional context / logs:
deployment

apiVersion: v1
kind: Service
metadata:
  name: service-nginx-clusterip
spec:
  type: ClusterIP
  selector:
    app: nginx
  ports:
    - port: 4000
      targetPort: 80

---

apiVersion: apps/v1
kind: Deployment
metadata:
  name: nginx-deployment
spec:
  selector:
    matchLabels:
      app: nginx
  template:
    metadata:
      labels:
        app: nginx
    spec:
      containers:
        - name: nginx-container
          image: nginx
  replicas: 2

ingress

apiVersion: networking.k8s.io/v1
kind: Ingress
metadata:
  name: nginx
  annotations:
    # We are defining this annotation to prevent nginx
    # from redirecting requests to `https` for now
    nginx.ingress.kubernetes.io/ssl-redirect: "false"
spec:
  rules:
    - http:
        paths:
          - path: /
            pathType: Prefix
            backend:
              service:
                name: service-nginx-clusterip
                port:
                  number: 4000

root@server-01:/vagrant_data/test/hellok8s# kubectl get nodes -o wide 
NAME        STATUS   ROLES                       AGE   VERSION        INTERNAL-IP   EXTERNAL-IP      OS-IMAGE             KERNEL-VERSION      CONTAINER-RUNTIME
node-01     Ready    <none>                      34m   v1.30.0+k3s1   10.0.2.15     192.168.56.111   Ubuntu 20.04.6 LTS   5.4.0-176-generic   containerd://1.7.15-k3s1
server-01   Ready    control-plane,etcd,master   36m   v1.30.0+k3s1   10.0.2.15     192.168.56.101   Ubuntu 20.04.6 LTS   5.4.0-181-generic   containerd://1.7.15-k3s1
root@server-01:/vagrant_data/test/hellok8s# kubectl apply -f nginx.yaml 
service/service-nginx-clusterip created
deployment.apps/nginx-deployment created
root@server-01:/vagrant_data/test/hellok8s# kubectl apply -f ingress.yaml 
ingress.networking.k8s.io/nginx created
root@server-01:/vagrant_data/test/hellok8s# kubectl get pods -o wide
NAME                                READY   STATUS    RESTARTS   AGE   IP           NODE        NOMINATED NODE   READINESS GATES
nginx-deployment-55df85b668-5jrpm   1/1     Running   0          15s   10.42.1.6    node-01     <none>           <none>
nginx-deployment-55df85b668-p7xbm   1/1     Running   0          15s   10.42.0.15   server-01   <none>           <none>
root@server-01:/vagrant_data/test/hellok8s# kubectl logs nginx-deployment-55df85b668-p7xbm
/docker-entrypoint.sh: /docker-entrypoint.d/ is not empty, will attempt to perform configuration
/docker-entrypoint.sh: Looking for shell scripts in /docker-entrypoint.d/
/docker-entrypoint.sh: Launching /docker-entrypoint.d/10-listen-on-ipv6-by-default.sh
10-listen-on-ipv6-by-default.sh: info: Getting the checksum of /etc/nginx/conf.d/default.conf
10-listen-on-ipv6-by-default.sh: info: Enabled listen on IPv6 in /etc/nginx/conf.d/default.conf
/docker-entrypoint.sh: Sourcing /docker-entrypoint.d/15-local-resolvers.envsh
/docker-entrypoint.sh: Launching /docker-entrypoint.d/20-envsubst-on-templates.sh
/docker-entrypoint.sh: Launching /docker-entrypoint.d/30-tune-worker-processes.sh
/docker-entrypoint.sh: Configuration complete; ready for start up
2024/05/16 03:26:31 [notice] 1#1: using the "epoll" event method
2024/05/16 03:26:31 [notice] 1#1: nginx/1.25.5
2024/05/16 03:26:31 [notice] 1#1: built by gcc 12.2.0 (Debian 12.2.0-14) 
2024/05/16 03:26:31 [notice] 1#1: OS: Linux 5.4.0-181-generic
2024/05/16 03:26:31 [notice] 1#1: getrlimit(RLIMIT_NOFILE): 1048576:1048576
2024/05/16 03:26:31 [notice] 1#1: start worker processes
2024/05/16 03:26:31 [notice] 1#1: start worker process 29
2024/05/16 03:26:31 [notice] 1#1: start worker process 30
2024/05/16 03:26:31 [notice] 1#1: start worker process 31
2024/05/16 03:26:31 [notice] 1#1: start worker process 32
root@server-01:/vagrant_data/test/hellok8s# kubectl logs nginx-deployment-55df85b668-5jrpm
Error from server (NotFound): the server could not find the requested resource ( pods/log nginx-deployment-55df85b668-5jrpm)
root@server-01:/vagrant_data/test/hellok8s#
@1369521908
Copy link
Author

1369521908 commented May 16, 2024 •
edited by brandond

install logs

time="2024-05-16T02:49:40Z" level=info msg="[native] begin to create cluster cluster-k8s..."
time="2024-05-16T02:49:40Z" level=info msg="[native] executing init k3s cluster logic..."
time="2024-05-16T02:49:40Z" level=info msg="[native] initialize control node..."
time="2024-05-16T02:49:40Z" level=info msg="the 1/5 time tring to ssh to 192.168.56.101:22 with user root"
-----BEGIN CERTIFICATE-----
MIIDOzCCAiOgAwIBAgIRAK/LCM1Id/aAftjgyVYE710wDQYJKoZIhvcNAQELBQAw
FDESMBAGA1UEAxMJaGFyYm9yLWNhMB4XDTI0MDUxMzA4MTUxMFoXDTI1MDUxMzA4
MTUxMFowHTEbMBkGA1UEAxMSY29yZS5oYXJib3IuZG9tYWluMIIBIjANBgkqhkiG
9w0BAQEFAAOCAQ8AMIIBCgKCAQEA1uRpwIPrF5dfyN4h3+dQo5nIM90Z/b1Q4OPQ
XmfSxnRLgtfFtzMNbAbqQmOBjC7Mj06WoJm/ujKoeY7VqDmMoBMnAdFRFXUWz1yP
AIewlEKgpyJmrJ50FIS045s/lSKvSGMzoZImOQ4JNwCXjj4mgf/009ZRnk+OX+Du
HJtSYXVi7gXHwUTJ3Pm1N3etRuAK/MzLxzv/IAX+vl0DQfc8bY8lu5iEFZgjTgwp
PtievzGzEd5joQho3fMu4pSBYl1WRAwA3473bMu6zHXejFI1+lzG6mfKWU7U1qOH
6MEkCA5wvzhoAFmjPIpas+TbdVUrFSJRqNMt4Bzb9iKVXqBo4QIDAQABo38wfTAO
BgNVHQ8BAf8EBAMCBaAwHQYDVR0lBBYwFAYIKwYBBQUHAwEGCCsGAQUFBwMCMAwG
A1UdEwEB/wQCMAAwHwYDVR0jBBgwFoAUc/sJ87gZrXRUyKdYvlVCnT1gFJEwHQYD
VR0RBBYwFIISY29yZS5oYXJib3IuZG9tYWluMA0GCSqGSIb3DQEBCwUAA4IBAQBO
Vmv1nzhggPjYYOUS+AixtotJP52I/Jg1NODpwCwwEE2ppsEkzxX//0q0JIpZUclw
ve5DWJwcAz0UyzNrO9w43pqZMZfVyCnoFsInFhAzGY3+0oVqPQvpMKclJ8PGBmJI
m0WSWxEwhbglVC4yc7b4FBm0+d8LsooEL/3T0nZO9MXUSEzCSZrAfJ/+0r8zAumq
mI2Q1Cn1435zFf2AU29SO746BGgTBY3CuYvuJ0TwDMJQGC0pBdpmKSZpkOqeiDVu
3VgnqAsYVLbztN7xlXJz4fx6zeY5kUN6loxrwkWMFPvlG7mflZZ0f4AutHzIfr62
85vnW+jrx2EzZq/oYALr
-----END CERTIFICATE-----
-----BEGIN RSA PRIVATE KEY-----
MIIEpAIBAAKCAQEA1uRpwIPrF5dfyN4h3+dQo5nIM90Z/b1Q4OPQXmfSxnRLgtfF
tzMNbAbqQmOBjC7Mj06WoJm/ujKoeY7VqDmMoBMnAdFRFXUWz1yPAIewlEKgpyJm
rJ50FIS045s/lSKvSGMzoZImOQ4JNwCXjj4mgf/009ZRnk+OX+DuHJtSYXVi7gXH
wUTJ3Pm1N3etRuAK/MzLxzv/IAX+vl0DQfc8bY8lu5iEFZgjTgwpPtievzGzEd5j
oQho3fMu4pSBYl1WRAwA3473bMu6zHXejFI1+lzG6mfKWU7U1qOH6MEkCA5wvzho
AFmjPIpas+TbdVUrFSJRqNMt4Bzb9iKVXqBo4QIDAQABAoIBAEbZIcCV/aFWh/7q
tZJjaiO2JNpskBAnJY4vrdgiNMy3NNobl5UR4N/3264trl1wZILu7xxd9GdIImGJ
ZQnEh4eTKO4he7EuXGnbY05SbjZDTUItRlqtFypYombUHX9m4xFjm3cufymaZhFb
QZxlAz8jCy34YApowlD8EKUf1GZKubUjcfkbXwmfm48Fxd2MZ189Dz7STNKP8mBF
Sa9M7Z3ZFK1vPR2LBYxq2wUU0cyTQFYhOWwseHyFb7CfuUD4EyTdzp0dy9Yjp4gj
q/X0EaR61qeiIM6hGO0PYTr5DoKwW/mUVueykqiLYtMs4o8GkcGeEMuc+/5vkdc2
BNOMhd0CgYEA5cZm4+KkcoFt15LET5YmGmaCyZzT2tYADPMh2cUWHPq5OSyvNVx0
VE/UFF9vuDyhLVwBdhEltSmn12qqNl0EYg5VazqlFPlg+e6ewymjeay1HXWIX9YD
Q9ci0bhEtaQ6G8Z3e7mLD6XNE3YPBwhgK/31qGvDJHCKP2UbUrKZNp8CgYEA72sq
FEkx85v327dtrWNbCcSxUUCmoeQ9W7+Ba9p4rwByWN3eFLfJntNfMtA49vxlpTi6
wHOzTVZpXK5K7Uh2ayogXqiR3cKCLkkP9uLdqJHkztTYHMrz/by8UMo483hH15jE
5ez1MmLnXrGnLOtiuTVvt0g/8Wo4+mS9/exPsH8CgYASsbWlTMc8WbNqGpLGNt0a
wMRbStshrs+KPi2+3f4SlYOEV1JZ8Q+13FjQ34LjZmtmmVm1FWgP+TeimxcGHUNC
BxCEDN1pe3CcZQGdt0GJoLeLkghqY6sYVeSLU4ajkJz5rYgDFPFvcHni33QRimjo
q1zVOw9BY7abucojTEyS2QKBgQCSJFP3NgarsRTGpVzpbbBwcgPamSAdyfg6PNd/
QkhQanFiFWwuMVF9BYEivz9GZCKXi91TBEz6dDi0Ph+2X3kJSRmhLjx7pPNZ1XdN
OIYdE5EbP4j7alctFzvMBORmmZEg5TurLc0uCHk6sC5fmP8WFqwusId3kWNeVIg0
qz6peQKBgQCWeFUBifBdVtiwQqLz3odVYj2Aw0wYwRYI8v4T39C2bqCXtSyAL3IT
Z4qDZPqhWXc8KGbZf4C4ozBdDZaphtO4idXaxQwvM2/3VNw1/HTC6oSYtl5K+XEB
26Y6Qj+GPxtYfqSfDsD/uEM+qVK3U5KKS3KbLjGirrJsSQQlwL01pQ==
-----END RSA PRIVATE KEY-----
-----BEGIN CERTIFICATE-----
MIIDOzCCAiOgAwIBAgIRAK/LCM1Id/aAftjgyVYE710wDQYJKoZIhvcNAQELBQAw
FDESMBAGA1UEAxMJaGFyYm9yLWNhMB4XDTI0MDUxMzA4MTUxMFoXDTI1MDUxMzA4
MTUxMFowHTEbMBkGA1UEAxMSY29yZS5oYXJib3IuZG9tYWluMIIBIjANBgkqhkiG
9w0BAQEFAAOCAQ8AMIIBCgKCAQEA1uRpwIPrF5dfyN4h3+dQo5nIM90Z/b1Q4OPQ
XmfSxnRLgtfFtzMNbAbqQmOBjC7Mj06WoJm/ujKoeY7VqDmMoBMnAdFRFXUWz1yP
AIewlEKgpyJmrJ50FIS045s/lSKvSGMzoZImOQ4JNwCXjj4mgf/009ZRnk+OX+Du
HJtSYXVi7gXHwUTJ3Pm1N3etRuAK/MzLxzv/IAX+vl0DQfc8bY8lu5iEFZgjTgwp
PtievzGzEd5joQho3fMu4pSBYl1WRAwA3473bMu6zHXejFI1+lzG6mfKWU7U1qOH
6MEkCA5wvzhoAFmjPIpas+TbdVUrFSJRqNMt4Bzb9iKVXqBo4QIDAQABo38wfTAO
BgNVHQ8BAf8EBAMCBaAwHQYDVR0lBBYwFAYIKwYBBQUHAwEGCCsGAQUFBwMCMAwG
A1UdEwEB/wQCMAAwHwYDVR0jBBgwFoAUc/sJ87gZrXRUyKdYvlVCnT1gFJEwHQYD
VR0RBBYwFIISY29yZS5oYXJib3IuZG9tYWluMA0GCSqGSIb3DQEBCwUAA4IBAQBO
Vmv1nzhggPjYYOUS+AixtotJP52I/Jg1NODpwCwwEE2ppsEkzxX//0q0JIpZUclw
ve5DWJwcAz0UyzNrO9w43pqZMZfVyCnoFsInFhAzGY3+0oVqPQvpMKclJ8PGBmJI
m0WSWxEwhbglVC4yc7b4FBm0+d8LsooEL/3T0nZO9MXUSEzCSZrAfJ/+0r8zAumq
mI2Q1Cn1435zFf2AU29SO746BGgTBY3CuYvuJ0TwDMJQGC0pBdpmKSZpkOqeiDVu
3VgnqAsYVLbztN7xlXJz4fx6zeY5kUN6loxrwkWMFPvlG7mflZZ0f4AutHzIfr62
85vnW+jrx2EzZq/oYALr
-----END CERTIFICATE-----
auths: null
configs:
core.harbor.domain:
auth:
auth: ""
identitytoken: ""
password: Harbor12345
username: admin
tls:
ca_file: /etc/rancher/k3s/core.harbor.domain/ca
cert_file: /etc/rancher/k3s/core.harbor.domain/cert
insecure_skip_verify: true
key_file: /etc/rancher/k3s/core.harbor.domain/key
mirrors:
docker.io:
endpoint:
- https://docker.m.daocloud.io
rewrite: null
harbor.io:
endpoint:
- https://core.harbor.domain
rewrite: null
time="2024-05-16T02:49:40Z" level=info msg="[cluster] k3s master command: curl -sLS https://rancher-mirror.rancher.cn/k3s/k3s-install.sh | INSTALL_K3S_EXEC='server --advertise-address=192.168.56.101 --cluster-cidr=10.42.0.0/16 --cluster-init --node-external-ip=192.168.56.101 --tls-san=192.168.56.101 ' INSTALL_K3S_MIRROR='cn' INSTALL_K3S_VERSION='v1.30.0+k3s1' K3S_TOKEN='bc790121fb2f8c758e6cdaf1b0367b89' sh -"
time="2024-05-16T02:49:40Z" level=info msg="the 1/5 time tring to ssh to 192.168.56.101:22 with user root"
[INFO] Using v1.30.0+k3s1 as release
[INFO] Downloading hash rancher-mirror.rancher.cn/k3s/v1.30.0-k3s1/sha256sum-amd64.txt
[INFO] Downloading binary rancher-mirror.rancher.cn/k3s/v1.30.0-k3s1/k3s
[INFO] Verifying binary download
[INFO] Installing k3s to /usr/local/bin/k3s
[INFO] Skipping installation of SELinux RPM
[INFO] Creating /usr/local/bin/kubectl symlink to k3s
[INFO] Creating /usr/local/bin/crictl symlink to k3s
[INFO] Skipping /usr/local/bin/ctr symlink to k3s, command exists in PATH at /usr/bin/ctr
[INFO] Creating killall script /usr/local/bin/k3s-killall.sh
[INFO] Creating uninstall script /usr/local/bin/k3s-uninstall.sh
[INFO] env: Creating environment file /etc/systemd/system/k3s.service.env
[INFO] systemd: Creating service file /etc/systemd/system/k3s.service
[INFO] systemd: Enabling k3s unit
Created symlink /etc/systemd/system/multi-user.target.wants/k3s.service → /etc/systemd/system/k3s.service.
[INFO] systemd: Starting k3s
time="2024-05-16T02:49:55Z" level=info msg="[native] successfully initialize the first control node"
time="2024-05-16T02:49:55Z" level=info msg="the 1/5 time tring to ssh to 192.168.56.101:22 with user root"
apiVersion: v1
clusters:
- cluster:
certificate-authority-data: LS0tLS1CRUdJTiBDRVJUSUZJQ0FURS0tLS0tCk1JSUJkekNDQVIyZ0F3SUJBZ0lCQURBS0JnZ3Foa2pPUFFRREFqQWpNU0V3SHdZRFZRUUREQmhyTTNNdGMyVnkKZG1WeUxXTmhRREUzTVRVNE1qYzNPVEF3SGhjTk1qUXdOVEUyTURJME9UVXdXaGNOTXpRd05URTBNREkwT1RVdwpXakFqTVNFd0h3WURWUVFEREJock0zTXRjMlZ5ZG1WeUxXTmhRREUzTVRVNE1qYzNPVEF3V1RBVEJnY3Foa2pPClBRSUJCZ2dxaGtqT1BRTUJCd05DQUFUTk55dTRaelRpQ3pBRHJsbTdNSVBoc25Dc0NJdVVNQ0x0ZHBVK2laK1IKL1dZTTVPWVFzdHJEUmhnYU5EYWFiZW84VFoxZkJ0NHZlY0pmU1VaS3FiUWZvMEl3UURBT0JnTlZIUThCQWY4RQpCQU1DQXFRd0R3WURWUjBUQVFIL0JBVXdBd0VCL3pBZEJnTlZIUTRFRmdRVXVXbExadFRudTBMcER3VnVZdEl2CnIxWWRUN2t3Q2dZSUtvWkl6ajBFQXdJRFNBQXdSUUloQUp2TEk2bGgxS2VhVHJpeXRpNndoKzNJUlJGQVBQWUsKU05hbHBsZVp4R1o1QWlCdlFUWm8yRU1adkhaU0MwT0hnNzZEUlkvM2hYdEtnc0g2VndSK21MUkI0Zz09Ci0tLS0tRU5EIENFUlRJRklDQVRFLS0tLS0K
server: https://127.0.0.1:6443
name: default
contexts:
- context:
cluster: default
user: default
name: default
current-context: default
kind: Config
preferences: {}
users:
- name: default
user:
client-certificate-data: LS0tLS1CRUdJTiBDRVJUSUZJQ0FURS0tLS0tCk1JSUJrVENDQVRlZ0F3SUJBZ0lJYmlDSzRhVmRuTWd3Q2dZSUtvWkl6ajBFQXdJd0l6RWhNQjhHQTFVRUF3d1kKYXpOekxXTnNhV1Z1ZEMxallVQXhOekUxT0RJM056a3dNQjRYRFRJME1EVXhOakF5TkRrMU1Gb1hEVEkxTURVeApOakF5TkRrMU1Gb3dNREVYTUJVR0ExVUVDaE1PYzNsemRHVnRPbTFoYzNSbGNuTXhGVEFUQmdOVkJBTVRESE41CmMzUmxiVHBoWkcxcGJqQlpNQk1HQnlxR1NNNDlBZ0VHQ0NxR1NNNDlBd0VIQTBJQUJPaHhnSHpZZ0JTSzJLcTAKZVFMR2FsSTZRQno2M2FkRHNacmZBVTRMZVgwMjBmNklyY2JSQVBFWlNtVitpamdvbXZ4NmI5a3Z6Qm95NGMwWQowQkN6bHRhalNEQkdNQTRHQTFVZER3RUIvd1FFQXdJRm9EQVRCZ05WSFNVRUREQUtCZ2dyQmdFRkJRY0RBakFmCkJnTlZIU01FR0RBV2dCUVh3VzJYMzhrK0drbFdhNTJyd1JpZDFScE5rekFLQmdncWhrak9QUVFEQWdOSUFEQkYKQWlFQWljSEtKNE5KQUNTR1BIQThGbDZ4Q0l3aHNSeTFLNDAvSUdrL1drYUF4WlVDSUJPN3V0c0ZYQ3RrQ0l0dwpsQlkrTGxYODZ5c1RibGg0UFlFYVI0dEVYMjFICi0tLS0tRU5EIENFUlRJRklDQVRFLS0tLS0KLS0tLS1CRUdJTiBDRVJUSUZJQ0FURS0tLS0tCk1JSUJkekNDQVIyZ0F3SUJBZ0lCQURBS0JnZ3Foa2pPUFFRREFqQWpNU0V3SHdZRFZRUUREQmhyTTNNdFkyeHAKWlc1MExXTmhRREUzTVRVNE1qYzNPVEF3SGhjTk1qUXdOVEUyTURJME9UVXdXaGNOTXpRd05URTBNREkwT1RVdwpXakFqTVNFd0h3WURWUVFEREJock0zTXRZMnhwWlc1MExXTmhRREUzTVRVNE1qYzNPVEF3V1RBVEJnY3Foa2pPClBRSUJCZ2dxaGtqT1BRTUJCd05DQUFSQTJhNHVUU0VMRHhldkM0anI5RzVSSWU1UjAwVWdYcTJXL1pNTXpCU1IKdUdLTWVXOHVYUnNHYmxkTU9OMk51VnNkZnErd1R0eTdlaFQzU0lacDlGUmVvMEl3UURBT0JnTlZIUThCQWY4RQpCQU1DQXFRd0R3WURWUjBUQVFIL0JBVXdBd0VCL3pBZEJnTlZIUTRFRmdRVUY4RnRsOS9KUGhwSlZtdWRxOEVZCm5kVWFUWk13Q2dZSUtvWkl6ajBFQXdJRFNBQXdSUUlnSWFFZHI1cFQ1WGZweThMSEFUdFp5SUkyNERxdlZYSFkKbDQvZ2hIUlJPREFDSVFEZzNxTDBUb2dlekFpdG5qYWpaZEdPSXRZN0dxdmtwSEczSzBSbDVMWVpZZz09Ci0tLS0tRU5EIENFUlRJRklDQVRFLS0tLS0K
client-key-data: LS0tLS1CRUdJTiBFQyBQUklWQVRFIEtFWS0tLS0tCk1IY0NBUUVFSUpFWFpobmRwVC84NkdML2ZsSEwwUkp1cEdCTFVIVnFFaFFqZS9aYzRPcjJvQW9HQ0NxR1NNNDkKQXdFSG9VUURRZ0FFNkhHQWZOaUFGSXJZcXJSNUFzWnFVanBBSFByZHAwT3htdDhCVGd0NWZUYlIvb2l0eHRFQQo4UmxLWlg2S09DaWEvSHB2MlMvTUdqTGh6UmpRRUxPVzFnPT0KLS0tLS1FTkQgRUMgUFJJVkFURSBLRVktLS0tLQo=
time="2024-05-16T02:49:55Z" level=info msg="[native] deploying additional manifests"
time="2024-05-16T02:49:55Z" level=info msg="[native] kube-explorer for cluster cluster-k8s will listen on 127.0.0.1:41255..."
time="2024-05-16T02:49:55Z" level=info msg="[native] successfully deployed additional manifests"
time="2024-05-16T02:49:55Z" level=info msg="[native] successfully executed init k3s cluster logic"
time="2024-05-16T02:49:55Z" level=info msg="=========================== Prompt Info ==========================="
time="2024-05-16T02:49:55Z" level=info msg="Use 'autok3s kubectl config use-context cluster-k8s'"
time="2024-05-16T02:49:55Z" level=info msg="Use 'autok3s kubectl get pods -A' get POD status`"
time="2024-05-16T02:51:53Z" level=info msg="[native] begin to join nodes for cluster-k8s..."
time="2024-05-16T02:51:53Z" level=info msg="[native] executing join k3s node logic"
time="2024-05-16T02:51:53Z" level=info msg="[native] joining k3s worker-1..."
time="2024-05-16T02:51:53Z" level=info msg="the 1/5 time tring to ssh to 192.168.56.111:22 with user root"
-----BEGIN CERTIFICATE-----
MIIDOzCCAiOgAwIBAgIRAK/LCM1Id/aAftjgyVYE710wDQYJKoZIhvcNAQELBQAw
FDESMBAGA1UEAxMJaGFyYm9yLWNhMB4XDTI0MDUxMzA4MTUxMFoXDTI1MDUxMzA4
MTUxMFowHTEbMBkGA1UEAxMSY29yZS5oYXJib3IuZG9tYWluMIIBIjANBgkqhkiG
9w0BAQEFAAOCAQ8AMIIBCgKCAQEA1uRpwIPrF5dfyN4h3+dQo5nIM90Z/b1Q4OPQ
XmfSxnRLgtfFtzMNbAbqQmOBjC7Mj06WoJm/ujKoeY7VqDmMoBMnAdFRFXUWz1yP
AIewlEKgpyJmrJ50FIS045s/lSKvSGMzoZImOQ4JNwCXjj4mgf/009ZRnk+OX+Du
HJtSYXVi7gXHwUTJ3Pm1N3etRuAK/MzLxzv/IAX+vl0DQfc8bY8lu5iEFZgjTgwp
PtievzGzEd5joQho3fMu4pSBYl1WRAwA3473bMu6zHXejFI1+lzG6mfKWU7U1qOH
6MEkCA5wvzhoAFmjPIpas+TbdVUrFSJRqNMt4Bzb9iKVXqBo4QIDAQABo38wfTAO
BgNVHQ8BAf8EBAMCBaAwHQYDVR0lBBYwFAYIKwYBBQUHAwEGCCsGAQUFBwMCMAwG
A1UdEwEB/wQCMAAwHwYDVR0jBBgwFoAUc/sJ87gZrXRUyKdYvlVCnT1gFJEwHQYD
VR0RBBYwFIISY29yZS5oYXJib3IuZG9tYWluMA0GCSqGSIb3DQEBCwUAA4IBAQBO
Vmv1nzhggPjYYOUS+AixtotJP52I/Jg1NODpwCwwEE2ppsEkzxX//0q0JIpZUclw
ve5DWJwcAz0UyzNrO9w43pqZMZfVyCnoFsInFhAzGY3+0oVqPQvpMKclJ8PGBmJI
m0WSWxEwhbglVC4yc7b4FBm0+d8LsooEL/3T0nZO9MXUSEzCSZrAfJ/+0r8zAumq
mI2Q1Cn1435zFf2AU29SO746BGgTBY3CuYvuJ0TwDMJQGC0pBdpmKSZpkOqeiDVu
3VgnqAsYVLbztN7xlXJz4fx6zeY5kUN6loxrwkWMFPvlG7mflZZ0f4AutHzIfr62
85vnW+jrx2EzZq/oYALr
-----END CERTIFICATE-----
-----BEGIN RSA PRIVATE KEY-----
MIIEpAIBAAKCAQEA1uRpwIPrF5dfyN4h3+dQo5nIM90Z/b1Q4OPQXmfSxnRLgtfF
tzMNbAbqQmOBjC7Mj06WoJm/ujKoeY7VqDmMoBMnAdFRFXUWz1yPAIewlEKgpyJm
rJ50FIS045s/lSKvSGMzoZImOQ4JNwCXjj4mgf/009ZRnk+OX+DuHJtSYXVi7gXH
wUTJ3Pm1N3etRuAK/MzLxzv/IAX+vl0DQfc8bY8lu5iEFZgjTgwpPtievzGzEd5j
oQho3fMu4pSBYl1WRAwA3473bMu6zHXejFI1+lzG6mfKWU7U1qOH6MEkCA5wvzho
AFmjPIpas+TbdVUrFSJRqNMt4Bzb9iKVXqBo4QIDAQABAoIBAEbZIcCV/aFWh/7q
tZJjaiO2JNpskBAnJY4vrdgiNMy3NNobl5UR4N/3264trl1wZILu7xxd9GdIImGJ
ZQnEh4eTKO4he7EuXGnbY05SbjZDTUItRlqtFypYombUHX9m4xFjm3cufymaZhFb
QZxlAz8jCy34YApowlD8EKUf1GZKubUjcfkbXwmfm48Fxd2MZ189Dz7STNKP8mBF
Sa9M7Z3ZFK1vPR2LBYxq2wUU0cyTQFYhOWwseHyFb7CfuUD4EyTdzp0dy9Yjp4gj
q/X0EaR61qeiIM6hGO0PYTr5DoKwW/mUVueykqiLYtMs4o8GkcGeEMuc+/5vkdc2
BNOMhd0CgYEA5cZm4+KkcoFt15LET5YmGmaCyZzT2tYADPMh2cUWHPq5OSyvNVx0
VE/UFF9vuDyhLVwBdhEltSmn12qqNl0EYg5VazqlFPlg+e6ewymjeay1HXWIX9YD
Q9ci0bhEtaQ6G8Z3e7mLD6XNE3YPBwhgK/31qGvDJHCKP2UbUrKZNp8CgYEA72sq
FEkx85v327dtrWNbCcSxUUCmoeQ9W7+Ba9p4rwByWN3eFLfJntNfMtA49vxlpTi6
wHOzTVZpXK5K7Uh2ayogXqiR3cKCLkkP9uLdqJHkztTYHMrz/by8UMo483hH15jE
5ez1MmLnXrGnLOtiuTVvt0g/8Wo4+mS9/exPsH8CgYASsbWlTMc8WbNqGpLGNt0a
wMRbStshrs+KPi2+3f4SlYOEV1JZ8Q+13FjQ34LjZmtmmVm1FWgP+TeimxcGHUNC
BxCEDN1pe3CcZQGdt0GJoLeLkghqY6sYVeSLU4ajkJz5rYgDFPFvcHni33QRimjo
q1zVOw9BY7abucojTEyS2QKBgQCSJFP3NgarsRTGpVzpbbBwcgPamSAdyfg6PNd/
QkhQanFiFWwuMVF9BYEivz9GZCKXi91TBEz6dDi0Ph+2X3kJSRmhLjx7pPNZ1XdN
OIYdE5EbP4j7alctFzvMBORmmZEg5TurLc0uCHk6sC5fmP8WFqwusId3kWNeVIg0
qz6peQKBgQCWeFUBifBdVtiwQqLz3odVYj2Aw0wYwRYI8v4T39C2bqCXtSyAL3IT
Z4qDZPqhWXc8KGbZf4C4ozBdDZaphtO4idXaxQwvM2/3VNw1/HTC6oSYtl5K+XEB
26Y6Qj+GPxtYfqSfDsD/uEM+qVK3U5KKS3KbLjGirrJsSQQlwL01pQ==
-----END RSA PRIVATE KEY-----
-----BEGIN CERTIFICATE-----
MIIDOzCCAiOgAwIBAgIRAK/LCM1Id/aAftjgyVYE710wDQYJKoZIhvcNAQELBQAw
FDESMBAGA1UEAxMJaGFyYm9yLWNhMB4XDTI0MDUxMzA4MTUxMFoXDTI1MDUxMzA4
MTUxMFowHTEbMBkGA1UEAxMSY29yZS5oYXJib3IuZG9tYWluMIIBIjANBgkqhkiG
9w0BAQEFAAOCAQ8AMIIBCgKCAQEA1uRpwIPrF5dfyN4h3+dQo5nIM90Z/b1Q4OPQ
XmfSxnRLgtfFtzMNbAbqQmOBjC7Mj06WoJm/ujKoeY7VqDmMoBMnAdFRFXUWz1yP
AIewlEKgpyJmrJ50FIS045s/lSKvSGMzoZImOQ4JNwCXjj4mgf/009ZRnk+OX+Du
HJtSYXVi7gXHwUTJ3Pm1N3etRuAK/MzLxzv/IAX+vl0DQfc8bY8lu5iEFZgjTgwp
PtievzGzEd5joQho3fMu4pSBYl1WRAwA3473bMu6zHXejFI1+lzG6mfKWU7U1qOH
6MEkCA5wvzhoAFmjPIpas+TbdVUrFSJRqNMt4Bzb9iKVXqBo4QIDAQABo38wfTAO
BgNVHQ8BAf8EBAMCBaAwHQYDVR0lBBYwFAYIKwYBBQUHAwEGCCsGAQUFBwMCMAwG
A1UdEwEB/wQCMAAwHwYDVR0jBBgwFoAUc/sJ87gZrXRUyKdYvlVCnT1gFJEwHQYD
VR0RBBYwFIISY29yZS5oYXJib3IuZG9tYWluMA0GCSqGSIb3DQEBCwUAA4IBAQBO
Vmv1nzhggPjYYOUS+AixtotJP52I/Jg1NODpwCwwEE2ppsEkzxX//0q0JIpZUclw
ve5DWJwcAz0UyzNrO9w43pqZMZfVyCnoFsInFhAzGY3+0oVqPQvpMKclJ8PGBmJI
m0WSWxEwhbglVC4yc7b4FBm0+d8LsooEL/3T0nZO9MXUSEzCSZrAfJ/+0r8zAumq
mI2Q1Cn1435zFf2AU29SO746BGgTBY3CuYvuJ0TwDMJQGC0pBdpmKSZpkOqeiDVu
3VgnqAsYVLbztN7xlXJz4fx6zeY5kUN6loxrwkWMFPvlG7mflZZ0f4AutHzIfr62
85vnW+jrx2EzZq/oYALr
-----END CERTIFICATE-----
auths: null
configs:
core.harbor.domain:
auth:
auth: ""
identitytoken: ""
password: Harbor12345
username: admin
tls:
ca_file: /etc/rancher/k3s/core.harbor.domain/ca
cert_file: /etc/rancher/k3s/core.harbor.domain/cert
insecure_skip_verify: true
key_file: /etc/rancher/k3s/core.harbor.domain/key
mirrors:
docker.io:
endpoint:
- https://docker.m.daocloud.io
rewrite: null
harbor.io:
endpoint:
- https://core.harbor.domain
rewrite: null
time="2024-05-16T02:51:53Z" level=info msg="[cluster] k3s worker command: curl -sLS https://rancher-mirror.rancher.cn/k3s/k3s-install.sh | INSTALL_K3S_EXEC='--node-external-ip=192.168.56.111 ' INSTALL_K3S_MIRROR='cn' INSTALL_K3S_VERSION='v1.30.0+k3s1' K3S_TOKEN='bc790121fb2f8c758e6cdaf1b0367b89' K3S_URL='https://192.168.56.101:6443' sh -"
time="2024-05-16T02:51:53Z" level=info msg="the 1/5 time tring to ssh to 192.168.56.111:22 with user root"
[INFO] Using v1.30.0+k3s1 as release
[INFO] Downloading hash rancher-mirror.rancher.cn/k3s/v1.30.0-k3s1/sha256sum-amd64.txt
[INFO] Downloading binary rancher-mirror.rancher.cn/k3s/v1.30.0-k3s1/k3s
[INFO] Verifying binary download
[INFO] Installing k3s to /usr/local/bin/k3s
[INFO] Skipping installation of SELinux RPM
[INFO] Creating /usr/local/bin/kubectl symlink to k3s
[INFO] Creating /usr/local/bin/crictl symlink to k3s
[INFO] Creating /usr/local/bin/ctr symlink to k3s
[INFO] Creating killall script /usr/local/bin/k3s-killall.sh
[INFO] Creating uninstall script /usr/local/bin/k3s-agent-uninstall.sh
[INFO] env: Creating environment file /etc/systemd/system/k3s-agent.service.env
[INFO] systemd: Creating service file /etc/systemd/system/k3s-agent.service
[INFO] systemd: Enabling k3s-agent unit
Created symlink /etc/systemd/system/multi-user.target.wants/k3s-agent.service → /etc/systemd/system/k3s-agent.service.
[INFO] systemd: Starting k3s-agent
time="2024-05-16T02:52:09Z" level=info msg="[native] successfully joined k3s worker-1"
time="2024-05-16T02:52:09Z" level=info msg="[native] successfully executed join k3s node logic"
time="2024-05-16T02:52:09Z" level=info msg="[native] successfully executed join logic"

@brandond
Copy link
Contributor

brandond commented May 16, 2024 •
edited

I'm not really sure what I'm looking at here, most of this output appears to be from autok3s.

However, I do see that both of your nodes have the same internal IP address:

NAME        STATUS   ROLES                       AGE   VERSION        INTERNAL-IP   EXTERNAL-IP      OS-IMAGE             KERNEL-VERSION      CONTAINER-RUNTIME
node-01     Ready    <none>                      34m   v1.30.0+k3s1   10.0.2.15     192.168.56.111   Ubuntu 20.04.6 LTS   5.4.0-176-generic   containerd://1.7.15-k3s1
server-01   Ready    control-plane,etcd,master   36m   v1.30.0+k3s1   10.0.2.15     192.168.56.101   Ubuntu 20.04.6 LTS   5.4.0-181-generic   containerd://1.7.15-k3s1

That is not going to work. Your nodes need unique IPs.

Also note that the external-ip field is meant to be used for a public IP that is NATed to the private IP, it is not meant to be another interface with a private IP directly attached to the node.

@1369521908
Copy link
Author

I'm not really sure what I'm looking at here, most of this output appears to be from autok3s.

However, I do see that both of your nodes have the same internal IP address:

NAME        STATUS   ROLES                       AGE   VERSION        INTERNAL-IP   EXTERNAL-IP      OS-IMAGE             KERNEL-VERSION      CONTAINER-RUNTIME
node-01     Ready    <none>                      34m   v1.30.0+k3s1   10.0.2.15     192.168.56.111   Ubuntu 20.04.6 LTS   5.4.0-176-generic   containerd://1.7.15-k3s1
server-01   Ready    control-plane,etcd,master   36m   v1.30.0+k3s1   10.0.2.15     192.168.56.101   Ubuntu 20.04.6 LTS   5.4.0-181-generic   containerd://1.7.15-k3s1

That is not going to work. Your nodes need unique IPs.

Also note that the external-ip field is meant to be used for a public IP that is NATed to the private IP, it is not meant to be another interface with a private IP directly attached to the node.

ok i redo the network and try it again, thanks!

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Assignees
No one assigned
Labels
None yet
Projects
K3s Development
Status: Done Issue
Milestone
No milestone
Development

No branches or pull requests
2 participants
@brandond @1369521908