This repository has been archived by the owner on Feb 29, 2024. It is now read-only.
Commit
This commit does not belong to any branch on this repository, and may belong to a fork outside of the repository.
Raise an error if a service or container is failed
Sometimes container or service does not start, and this doesn't make the CI fail. Until now, the failed containers are listed in the /var/log/extras/ tree, but it's not checked on a regular basis. This patch intends to make a hard failure in case either a service or a container doesn't start as expected. Co-Authored-By: Cédric Jeanneret <cjeanner@redhat.com> Related-Bug: #1816523 Change-Id: I001e2f27d2b562bb0be87c8eaadcf3622e530498
- Loading branch information
1 parent
37b2a26
commit 7498342
Showing
9 changed files
with
82 additions
and
0 deletions.
There are no files selected for viewing
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Original file line number | Diff line number | Diff line change |
---|---|---|
@@ -0,0 +1,5 @@ | ||
--- | ||
features: | ||
- New "validate_services" boolean flag in order to check the service state | ||
- New "validate-services" role in charge of container and service state | ||
validation |
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Original file line number | Diff line number | Diff line change |
---|---|---|
@@ -0,0 +1,44 @@ | ||
--- | ||
- name: Do we have docker | ||
stat: | ||
path: /usr/bin/docker | ||
register: docker_stat | ||
|
||
- name: Does docker run | ||
systemd: | ||
name: docker | ||
register: docker_svc | ||
when: docker_stat.stat.exists | ||
|
||
- name: Do we have podman | ||
stat: | ||
path: /usr/bin/podman | ||
register: podman_stat | ||
|
||
# Check both docker and podman, even if we don't have any container. | ||
# This will help catching failed container in mixed env, for example while | ||
# migrating from docker to podman or the reverse. | ||
- name: Get failed containers for podman | ||
when: podman_stat.stat.exists | ||
shell: > | ||
{% raw %} | ||
podman ps -a --filter 'status=exited' --format '{{.Names}} {{.Status}}' | ||
{% endraw %} | ||
register: failed_podman | ||
|
||
- name: Get failed containers from docker | ||
when: docker_svc.status['SubState'] == 'running' | ||
shell: > | ||
{% raw %} | ||
docker ps -a --filter 'status=exited' --format '{{.Names}} {{.Status}}' | ||
{% endraw %} | ||
register: failed_docker | ||
|
||
- name: Fail if we detect failed container | ||
fail: | ||
msg: | | ||
Failed container detected. Please check the following locations | ||
/var/log/extras/failed_containers.log | ||
/var/log/extras/ | ||
when: item is not match(".* Exited \(0\) .* ago") | ||
loop: "{{ failed_podman.stdout_lines + failed_docker.stdout_lines}}" |
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Original file line number | Diff line number | Diff line change |
---|---|---|
@@ -0,0 +1,3 @@ | ||
--- | ||
- include_tasks: containers.yaml | ||
- include_tasks: systemd.yaml |
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Original file line number | Diff line number | Diff line change |
---|---|---|
@@ -0,0 +1,10 @@ | ||
--- | ||
- name: Get failed services from Systemd | ||
shell: > | ||
systemctl list-units --failed --plain --no-legend --no-pager | ||
register: systemd_state | ||
|
||
- name: Fails if we find failed systemd units | ||
assert: | ||
that: | ||
- systemd_state.stdout_lines|length == 0 |