Kubernetes cluster stuck in Stopping state

[saffronjam]

ISSUE TYPE

• Bug Report

COMPONENT NAME

API

CLOUDSTACK VERSION

4.17.0 (cloud have occurred on 4.16.1)

CONFIGURATION

Single host running management server and database

OS / ENVIRONMENT

Ubuntu 20.04

SUMMARY

When attempting to delete a Kubernetes cluster, it gets stuck in Stopping state.

And every time a kubernetes-related API call is made, such as "deleteKubernetesCluster", error 530 appears

Investigating the cloudstack-management.service logs, there seem to be an unhandled null pointer exception when issuing some string operation (perhaps a toString() on a null reference). Looking into the source code, it appears to be the case.

STEPS TO REPRODUCE

Removing clusters was possible, but this time it appears as some reference was deleted in database, which nullified a field in a row in the table kubernetes_supported_version, or related table. Therefore, it would be hard to write steps to reproduce.

EXPECTED RESULTS

Friendlier error message, recoverable state. How can such a field become null?

ACTUAL RESULTS

Kubernetes clusters stuck in stopping state, refuses to be deleted, prompting multiple error code 530 messages
(every delete operation is ignored since a cluster cannot be deleted while in the stopping state)

[Pearl1594]

@saffronjam is the Kubernetes binaries iso for the corresponding Kubernetes version available? Based on the NPE it appears that the iso for the Kubernetes version in question isn't available.

[saffronjam]

Thanks for the quick reply.

I think that was the case.

I resolved it by marking the entries in the database as removed.

You can close this issue :)