manifest-db: make all UUIDs unique #124
Closed
Conversation
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
We get regular failures in the manifest-db run. With the extra visibility from osbuild#123 it seems the errors we get (sometimes) are related to an xfs mount ``` mount/- (org.osbuild.xfs): mounting /dev/98302bc3-1aa3-4eab-96e9-9f1c6eb539ce/rootlv -> /var/lib/osbuild/store/tmp/buildroot-tmp-3y2wyhos/mounts/ mount/- (org.osbuild.xfs): already unmounted: /var/lib/osbuild/store/tmp/buildroot-tmp-3y2wyhos/mounts/ ``` which looks similar the UUID clashes we observed in osbuild/osbuild#1641 Upon inspecting the manifests it seems we have quite a few UUIDs in our manifests. Those *might* explain the issues that we sometimes see because the UUIDs is global to the host so running multiple tests in parallel can lead to clashes when xfs/btrfs assume that the mount is already done when in fact it was a different loopdevice that just happend to clash with the UUID. This commit creates a new UUID for each file where a duplicated UUID is found (but keeps that UUID inside the same file stable). We could make this smaller by limiting this to only xfs/btfs which (AFAIK) are the only systems affected by UUID clashes but my script is not smart enough for this.
mvo5
added a commit
to mvo5/osbuild-composer
that referenced
this pull request
May 2, 2024
To ensure that manifests get random(ish) and stable UUIDs we set the seed arg based on the filename and the environment (`OSBUILD_GEN_MANIFEST_RNG_SEED`). This should fix the issue discovered in osbuild/manifest-db#124 that duplicated UUIDs for xfs/btrfs can trigger random(ish) and hard to diagnose errors.
mvo5
added a commit
to mvo5/images
that referenced
this pull request
May 6, 2024
To ensure that manifests get random(ish) and stable UUIDs we set the rng seed arg based on the filename. This should fix the issue discovered in osbuild/manifest-db#124 that duplicated UUIDs for xfs/btrfs can trigger random(ish) and hard to diagnose errors. This is the same as osbuild/osbuild-composer#4124 hopefully for the right place this time.
mvo5
added a commit
to mvo5/images
that referenced
this pull request
May 6, 2024
To ensure that manifests get random(ish) and stable UUIDs we set the rng seed arg based on the filename. This should fix the issue discovered in osbuild/manifest-db#124 that duplicated UUIDs for xfs/btrfs can trigger random(ish) and hard to diagnose errors. This is the same as osbuild/osbuild-composer#4124 hopefully for the right place this time.
mvo5
added a commit
to mvo5/images
that referenced
this pull request
May 8, 2024
To ensure that manifests get random(ish) and stable UUIDs we set the rng seed arg based on the filename. This should fix the issue discovered in osbuild/manifest-db#124 that duplicated UUIDs for xfs/btrfs can trigger random(ish) and hard to diagnose errors. This is the same as osbuild/osbuild-composer#4124 hopefully for the right place this time.
mvo5
added a commit
to mvo5/images
that referenced
this pull request
May 14, 2024
To ensure that manifests get random(ish) and stable UUIDs we set the rng seed arg based on the filename. This should fix the issue discovered in osbuild/manifest-db#124 that duplicated UUIDs for xfs/btrfs can trigger random(ish) and hard to diagnose errors. This is the same as osbuild/osbuild-composer#4124 hopefully for the right place this time.
mvo5
added a commit
to mvo5/images
that referenced
this pull request
May 14, 2024
To ensure that manifests get random(ish) and stable UUIDs we set the rng seed arg based on the filename. This should fix the issue discovered in osbuild/manifest-db#124 that duplicated UUIDs for xfs/btrfs can trigger random(ish) and hard to diagnose errors. This is the same as osbuild/osbuild-composer#4124 hopefully for the right place this time.
github-merge-queue bot
pushed a commit
to osbuild/images
that referenced
this pull request
May 14, 2024
To ensure that manifests get random(ish) and stable UUIDs we set the rng seed arg based on the filename. This should fix the issue discovered in osbuild/manifest-db#124 that duplicated UUIDs for xfs/btrfs can trigger random(ish) and hard to diagnose errors. This is the same as osbuild/osbuild-composer#4124 hopefully for the right place this time.
|
Closing for now as I'm not sure if the next manifest-db generation wouldn't just override it again and also it seems the producer side of this should get fixed (i.e. images and osbuild-composer(?)) |
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
We get regular failures in the manifest-db run. With the extra
visibility from #123
it seems the errors we get (sometimes) are related to an xfs mount
which looks similar the UUID clashes we observed in
osbuild/osbuild#1641
Upon inspecting the manifests it seems we have quite a few UUIDs in
our manifests. Those might explain the issues that we sometimes
see because the UUIDs is global to the host so running multiple tests
in parallel can lead to clashes when xfs/btrfs assume that the
mount is already done when in fact it was a different loopdevice
that just happend to clash with the UUID.
This commit creates a new UUID for each file where a duplicated
UUID is found (but keeps that UUID inside the same file stable).
We could make this smaller by limiting this to only xfs/btfs which
(AFAIK) are the only systems affected by UUID clashes but my
script is not smart enough for this.
Fwiw, the script found the following duplicated uuids:
[edit: draft because this would also need a regression test to ensure we do not add clashing uuids again]