New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

KubernetesAssistant : improve detection of ready containers in pod #1051

Closed
dcdh opened this Issue May 10, 2018 · 2 comments

Comments

Projects
None yet
2 participants
@dcdh
Contributor

dcdh commented May 10, 2018

Using the KubernetesAssistant (or OpenshiftAssistant) we can programmatically wait for a pod to be ready by using the method: awaitApplicationReadinessOrFail

This method use another one called isRunning(Pod pod) to detect if the Pod is running.

Currently the implementation is:

private boolean isRunning(Pod pod) {
    return "running".equalsIgnoreCase(pod.getStatus().getPhase());
}

However, after multiple manual tests, the check made by the isRunning method is inadequate.

Inadequate in term of the main objective of the method awaitApplicationReadinessOrFail because we want to know if the pod is ready meaning that each containers in the pod are ready.

Next, I've noticed that when the phase return running it does not mean that all containers in pod are ready (ie the readiness probe is ok for each of them).

So, having a pod phase defined to running means that containers have been started with readiness probe currently not check and the pod is in "Not ready" state displayed on the web console. Furthermore after all readiness probes are ok the phase will also return running.

So I propose this implementation instead which will check that all containers in the pod are ready:

private boolean isRunning(Pod pod) {
	return pod.getStatus()
			.getContainerStatuses()
			.stream()
			.allMatch(ContainerStatus::getReady);
}

And I guess that it should be a good point to rename the method isRunning to isReady or areAllContainersReady or something else.

Eratum

I confirm this code works :)

private boolean isRunning(Pod pod) {
    Readiness.isPodReady(pod);
}

So this is working:

public void awaitApplicationReadinessOrFail() {
        await().atMost(5, TimeUnit.MINUTES).until(() -> {
                List<Pod> list = client.pods().inNamespace(namespace).list().getItems();
                return list.stream()
                    .filter(pod -> pod.getMetadata().getName().startsWith(applicationName) && !pod.getMetadata().getName().endsWith("-deploy"))
                    .filter(Readiness::isPodReady)
                    .collect(Collectors.toList()).size() >= 1;
            }
        );
}

Workaround:

Expose a route and use awaitUrl method to wait for the application to be ready.

Regards,
Damien

@dcdh dcdh changed the title from KubernetesAssistant : improve detection of running pod to KubernetesAssistant : improve detection of ready containers in pod May 10, 2018

dcdh added a commit to dcdh/arquillian-cube that referenced this issue May 15, 2018

@dcdh

This comment has been minimized.

Contributor

dcdh commented May 15, 2018

I have created a pull request #1053

I rewrote the method to use the Kubernetes and Openshift api.

@lordofthejars

This comment has been minimized.

Member

lordofthejars commented May 15, 2018

Thank you very much for your help.

@lordofthejars lordofthejars added this to the 1.53.4 milestone May 15, 2018

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment