tools: add logmonitor on stderr by default #123
Merged
Conversation
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
If a test fails there is little visiblity right now what exactly went wrong because the osbuild --json will default to a NullMontior. This commit switches to a LogMontior on stderr so that if the test fails we get the full log of the operation.
mvo5
added a commit
to mvo5/manifest-db
that referenced
this pull request
May 2, 2024
We get regular failures in the manifest-db run. With the extra visibility from osbuild#123 it seems the errors we get (sometimes) are related to an xfs mount ``` mount/- (org.osbuild.xfs): mounting /dev/98302bc3-1aa3-4eab-96e9-9f1c6eb539ce/rootlv -> /var/lib/osbuild/store/tmp/buildroot-tmp-3y2wyhos/mounts/ mount/- (org.osbuild.xfs): already unmounted: /var/lib/osbuild/store/tmp/buildroot-tmp-3y2wyhos/mounts/ ``` which looks similar the UUID clashes we observed in osbuild/osbuild#1641 Upon inspecting the manifests it seems we have quite a few UUIDs in our manifests. Those *might* explain the issues that we sometimes see because the UUIDs is global to the host so running multiple tests in parallel can lead to clashes when xfs/btrfs assume that the mount is already done when in fact it was a different loopdevice that just happend to clash with the UUID. This commit creates a new UUID for each file where a duplicated UUID is found (but keeps that UUID inside the same file stable). We could make this smaller by limiting this to only xfs/btfs which (AFAIK) are the only systems affected by UUID clashes but my script is not smart enough for this.
mvo5
added a commit
to mvo5/manifest-db
that referenced
this pull request
May 2, 2024
We get regular failures in the manifest-db run. With the extra visibility from osbuild#123 it seems the errors we get (sometimes) are related to an xfs mount ``` mount/- (org.osbuild.xfs): mounting /dev/98302bc3-1aa3-4eab-96e9-9f1c6eb539ce/rootlv -> /var/lib/osbuild/store/tmp/buildroot-tmp-3y2wyhos/mounts/ mount/- (org.osbuild.xfs): already unmounted: /var/lib/osbuild/store/tmp/buildroot-tmp-3y2wyhos/mounts/ ``` which looks similar the UUID clashes we observed in osbuild/osbuild#1641 Upon inspecting the manifests it seems we have quite a few UUIDs in our manifests. Those *might* explain the issues that we sometimes see because the UUIDs is global to the host so running multiple tests in parallel can lead to clashes when xfs/btrfs assume that the mount is already done when in fact it was a different loopdevice that just happend to clash with the UUID. This commit creates a new UUID for each file where a duplicated UUID is found (but keeps that UUID inside the same file stable). We could make this smaller by limiting this to only xfs/btfs which (AFAIK) are the only systems affected by UUID clashes but my script is not smart enough for this.
mvo5
added a commit
to mvo5/manifest-db
that referenced
this pull request
May 2, 2024
We get regular failures in the manifest-db run. With the extra visibility from osbuild#123 it seems the errors we get (sometimes) are related to an xfs mount ``` mount/- (org.osbuild.xfs): mounting /dev/98302bc3-1aa3-4eab-96e9-9f1c6eb539ce/rootlv -> /var/lib/osbuild/store/tmp/buildroot-tmp-3y2wyhos/mounts/ mount/- (org.osbuild.xfs): already unmounted: /var/lib/osbuild/store/tmp/buildroot-tmp-3y2wyhos/mounts/ ``` which looks similar the UUID clashes we observed in osbuild/osbuild#1641 Upon inspecting the manifests it seems we have quite a few UUIDs in our manifests. Those *might* explain the issues that we sometimes see because the UUIDs is global to the host so running multiple tests in parallel can lead to clashes when xfs/btrfs assume that the mount is already done when in fact it was a different loopdevice that just happend to clash with the UUID. This commit creates a new UUID for each file where a duplicated UUID is found (but keeps that UUID inside the same file stable). We could make this smaller by limiting this to only xfs/btfs which (AFAIK) are the only systems affected by UUID clashes but my script is not smart enough for this.
achilleas-k
approved these changes
May 3, 2024
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
If a test fails there is little visiblity right now what exactly went wrong because the osbuild --json will default to a NullMontior.
This commit switches to a LogMontior on stderr so that if the test fails we get the full log of the operation.
This was useful to debug a failure in osbuild/osbuild#1573