reset command improvements

[jasmingacic]

Fixes part of the issue #928.
Reset command will remove service stubs regardless of the role installed.
When running reset command without k0s binaries already being unpacked in the data directory reset command is not able to detect which role has already been installed. So reset will now try to remove both service stubs k0scontroller and k0sworker.

Signed-off-by: Jasmin Gacic jgacic@mirantis.com

[kke]

The behavior is a bit sketchy.

First it gets the role:

	role := install.GetRoleByStagedKubelet(c.K0sVars.BinDir)

This will return "worker" if no installation was detected.

Then it proceeds to uninstall both controller and worker:

	err := install.UninstallService("controller")
	if err != nil {  ...	}
	err = install.UninstallService("worker")
	if err != nil {  ...	}

I assume one of them will always report the logger.Infof("failed to uninstall k0s service: %v", err) in the err check block, as both of them will always run.

Finally it will proceed to do cleanup based on the install.GetRoleByStagedKubelet from above, regardless of which Uninstall succeeded. In case of controller+worker both cleanups will run in series.

I think the best would be to fix the GetRoleByStagedKubelet to only return "worker" if it somehow can validate that a worker config exists and use install.InstalledService to figure out if a service was installed.

[jasmingacic]

@kke GetRoleByStagedKubelet relies on existence of kubelet or api binary on the host in case that neither exist it will return empty role. So in case that the install failed due to existence of service file and no binaries have been before extracted reset won't be able to detect the role and therefore to remove service stubs which are named k0scontroller and k0sworker.
So this is a crude solution for only one part of the issue we are seeing.

[kke]

@kke GetRoleByStagedKubelet relies on existence of kubelet or api binary on the host in case that neither exist it will return empty role. So in case that the install failed due to existence of service file and no binaries have been before extracted reset won't be able to detect the role and therefore to remove service stubs which are named k0scontroller and k0sworker.
So this is a crude solution for only one part of the issue we are seeing.

k0s/pkg/install/cleanup.go

Lines 67 to 79 in de25ffe

	// This function attempts to find out the host role, by staged binaries
	func GetRoleByStagedKubelet(binPath string) string {
	apiBinary := fmt.Sprintf("%s/%s", binPath, "kube-apiserver")
	kubeletBinary := fmt.Sprintf("%s/%s", binPath, "kubelet")
	if util.FileExists(apiBinary) && util.FileExists(kubeletBinary) {
	return "controller+worker"
	} else if util.FileExists(apiBinary) {
	return "controller"
	} else {
	return "worker"
	}
	}

seems to me, it will never return an empty role, but always fall back to "worker".

[jasmingacic]

Yeah returning "worker" when binaries are not present caused reset to behave weird.

[jnummelin]

Added some specific comments.

There's bunch of funcs that are exported but lacks proper comments.

In general this is pretty good, I like the added structure and robustness for the reset/cleanup functionality.

[jnummelin]

Minor nit about the output:

/ # ./k0s reset
*  containerd steps
INFO[2021-06-11 07:15:58] successfully removed k0s containers!         
INFO[2021-06-11 07:15:58] no config file given, using defaults         
*  remove directories step
WARN[2021-06-11 07:15:58] failed to unmount /var/lib/k0s               
*  CNI leftovers cleanup step
k0s cleanup operations done. To ensure a full reset, a node reboot is recommended.
Error: errors received during clean-up: [failed to delete /var/lib/k0s. err: unlinkat /var/lib/k0s: device or resource busy]
2021-06-11 07:15:58.173736 I | errors received during clean-up: [failed to delete /var/lib/k0s. err: unlinkat /var/lib/k0s: device or resource busy]

The output is now coming from both fmt.Println and through logrus which makes it bit "messy" IMO. I'd rather have all output through logrus.

[kke]

The output is now coming from both fmt.Println and through logrus which makes it bit "messy" IMO. I'd rather have all output through logrus.

There's also direct output from the commands started via os.Exec as the stdout and stderr are piped directly.