oneke_troubleshoot

Troubleshooting

Broken OneGate access

For detailed info about OneGate please refer to the OneGate Usage and OneGate Configuration documents.

Because OneKE is a OneFlow service it requires OneFlow and OneGate OpenNebula components to be operational.

If the OneKE service is stuck in the DEPLOYING state and only VMs from the VNF role are visible, it is likely there is some networking or configuration issue regarding the OneGate component. You can try to confirm if OneGate is reachable from VNF nodes by logging in to a VNF node via SSH and executing the following command:

    $ ssh root@10.2.11.86 onegate vm show
    Warning: Permanently added '10.2.11.86' (ED25519) to the list of known hosts.
    VM 227
    NAME                : vnf_0_(service_105)

If the OneGate endpoint is not reachable from VNF nodes, you'll see an error/timeout message.

If the OneKE service is stuck in the DEPLOYING state and all VMs from all roles are visible, and you've also confirmed that VMs from the VNF role can access the OneGate component, there still may be a networking issue on the leader VNF node itself. You can try to confirm if OneGate is reachable from Kubernetes nodes via SSH by executing the following command:

    $ ssh -J root@10.2.11.86 root@172.20.0.101 onegate vm show
    Warning: Permanently added '10.2.11.86' (ED25519) to the list of known hosts.
    Warning: Permanently added '172.20.0.101' (ED25519) to the list of known hosts.
    VM 228
    NAME                : master_0_(service_105)

If you see error/timeout message on a Kubernetes node, but not on a VNF node, you should investigate networking config and logs on the leader VNF VM, specifically the /var/log/messages file.

Broken access to the public Internet

If you're constantly getting the ImagePullBackOff error in Kubernetes, please log in to a worker node and check:

• Check if the default gateway points to the private VIP address:

        $ ssh -J root@10.2.11.86 root@172.20.0.102 ip route show default
        Warning: Permanently added '10.2.11.86' (ED25519) to the list of known hosts.
        Warning: Permanently added '172.20.0.102' (ED25519) to the list of known hosts.
        default via 172.20.0.86 dev eth0

• Check if the DNS config points to the nameserver defined in the private VNET:

        $ ssh -J root@10.2.11.86 root@172.20.0.102 cat /etc/resolv.conf
        Warning: Permanently added '10.2.11.86' (ED25519) to the list of known hosts.
        Warning: Permanently added '172.20.0.102' (ED25519) to the list of known hosts.
        nameserver 1.1.1.1

If in all the above cases everything looks correct, then you should investigate networking config and logs on the leader VNF VM, specifically the /var/log/messages file.