tests: fix "chroot with overlay root" flake - #6970
Merged
Merged
Conversation
I have seen this fail twice on onf of my PRs. The problems is chown -R is currently not safe against concurrent removal of files and will report ENOENT errors which make it exit 1 and fail the test. However since it actually still chown's all files even when it fails we can ignore the failure here. In the test context in particular thew problem is we have the full / as lower dir so any host file removals in /var/lib/containers can make this fail. Because the blob info cache uses a global path of var/lib/containers/cache/blob-info-cache-v1.sqlite regardless of the per test root/runroot and sqlite transactions will create and remove the blob-info-cache-v1.sqlite-journal many times we have a good chance to hit the race. So we see failures like this: chown: changing ownership of '/tmp/buildah_tests.vrbqgx/chroot/merged/var/lib/containers/cache/blob-info-cache-v1.sqlite-journal The linked issues also shows a standalone coreutils reproducer to prove this is a problem there. Fixes: podman-container-tools#6967 Signed-off-by: Paul Holzinger <pholzing@redhat.com>
Contributor
|
LGTM |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
3 participants
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
What type of PR is this?
What this PR does / why we need it:
I have seen this fail twice on onf of my PRs. The problems is chown -R is currently not safe against concurrent removal of files and will report ENOENT errors which make it exit 1 and fail the test. However since it actually still chown's all files even when it fails we can ignore the failure here.
In the test context in particular thew problem is we have the full / as lower dir so any host file removals in /var/lib/containers can make this fail. Because the blob info cache uses a global path of var/lib/containers/cache/blob-info-cache-v1.sqlite regardless of the per test root/runroot and sqlite transactions will create and remove the blob-info-cache-v1.sqlite-journal many times we have a good chance to hit the race.
So we see failures like this:
chown: changing ownership of '/tmp/buildah_tests.vrbqgx/chroot/merged/var/lib/containers/cache/blob-info-cache-v1.sqlite-journal
The linked issues also shows a standalone coreutils reproducer to prove this is a problem there.
How to verify it
Which issue(s) this PR fixes:
Fixes: #6967
Special notes for your reviewer:
I plan to submit a bug report against coreutils and hopefully fix this properly there but this will take more time and it would take a long time until a new coreutils would land in our CI images anyway so this here seems the simplest to fix this for now and repduce some flakes.
Does this PR introduce a user-facing change?