Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

scylla_raid_setup: may need to run wipefs before reformatting disk when previous filesystem exists #13737

Closed
syuu1228 opened this issue May 1, 2023 · 3 comments
Milestone

Comments

@syuu1228
Copy link
Contributor

syuu1228 commented May 1, 2023

I found reports on Fedora and Arch user communities which says cannot find /dev/disk/by-uuid entry after formatted disk.
It seems similar with our issue on scylla_raid_setup (#11359 and scylladb/scylla-enterprise#2835 and others).

references:
https://forums.fedoraforum.org/showthread.php?288890-kernel-3-7-7-201-missing-dev-disk-by-uuid-entries-no-dev-disk-by-label
https://bbs.archlinux.org/viewtopic.php?id=229064
https://bugzilla.redhat.com/show_bug.cgi?id=902512

The discussion on the thread says, when we reformat a volume with another filesystem, kernel and libblkid may
skip to populate /dev/disk/by-* since it detected two filesystem signatures, because mkfs.xxx did not cleared previous filesystem signature.
To avoid this, we need to run wipefs before running mkfs.

It is probably rare case, and probably fixed on recent versions of distribution, but I suppose we should run wipefs for safety.

syuu1228 added a commit to syuu1228/scylla that referenced this issue May 1, 2023
The discussion on the thread says, when we reformat a volume with another
filesystem, kernel and libblkid may skip to populate /dev/disk/by-* since it
detected two filesystem signatures, because mkfs.xxx did not cleared previous
filesystem signature.
To avoid this, we need to run wipefs before running mkfs.

Also dropped -f option from mkfs.xfs, it will check wipefs is working as we
expected.

Closes scylladb#13737
syuu1228 added a commit to syuu1228/scylla that referenced this issue May 1, 2023
The discussion on the thread says, when we reformat a volume with another
filesystem, kernel and libblkid may skip to populate /dev/disk/by-* since it
detected two filesystem signatures, because mkfs.xxx did not cleared previous
filesystem signature.
To avoid this, we need to run wipefs before running mkfs.

Also dropped -f option from mkfs.xfs, it will check wipefs is working as we
expected.

Closes scylladb#13737

Signed-off-by: Takuya ASADA <syuu@scylladb.com>
syuu1228 added a commit to syuu1228/scylla that referenced this issue May 8, 2023
The discussion on the thread says, when we reformat a volume with another
filesystem, kernel and libblkid may skip to populate /dev/disk/by-* since it
detected two filesystem signatures, because mkfs.xxx did not cleared previous
filesystem signature.
To avoid this, we need to run wipefs before running mkfs.

Note that this runs wipefs twice, for target disks and also for RAID device.
wipefs for RAID device is needed since wipefs on disks doesn't clear filesystem signatures on /dev/mdX (we may see previous filesystem signature on /dev/mdX when we construct RAID volume multiple time on same disks).

Also dropped -f option from mkfs.xfs, it will check wipefs is working as we
expected.

Closes scylladb#13737

Signed-off-by: Takuya ASADA <syuu@scylladb.com>
syuu1228 added a commit to syuu1228/scylla that referenced this issue May 8, 2023
The discussion on the thread says, when we reformat a volume with another
filesystem, kernel and libblkid may skip to populate /dev/disk/by-* since it
detected two filesystem signatures, because mkfs.xxx did not cleared previous
filesystem signature.
To avoid this, we need to run wipefs before running mkfs.

Note that this runs wipefs twice, for target disks and also for RAID device.
wipefs for RAID device is needed since wipefs on disks doesn't clear filesystem signatures on /dev/mdX (we may see previous filesystem signature on /dev/mdX when we construct RAID volume multiple time on same disks).

Also dropped -f option from mkfs.xfs, it will check wipefs is working as we
expected.

Fixes scylladb#13737

Signed-off-by: Takuya ASADA <syuu@scylladb.com>
@DoronArazii DoronArazii added this to the 5.3 milestone Jun 20, 2023
@DoronArazii DoronArazii added backport/5.2 Issues that should be backported to 5.2 branch once they'll be fixed Requires-Backport-to-5.1 labels Jul 3, 2023
@DoronArazii
Copy link

@scylladb/scylla-maint please consider backport to all 5.x + 2021.1 & 2022.1

@DoronArazii DoronArazii modified the milestones: 5.3, 5.4 Jul 3, 2023
denesb pushed a commit that referenced this issue Jul 11, 2023
The discussion on the thread says, when we reformat a volume with another
filesystem, kernel and libblkid may skip to populate /dev/disk/by-* since it
detected two filesystem signatures, because mkfs.xxx did not cleared previous
filesystem signature.
To avoid this, we need to run wipefs before running mkfs.

Note that this runs wipefs twice, for target disks and also for RAID device.
wipefs for RAID device is needed since wipefs on disks doesn't clear filesystem signatures on /dev/mdX (we may see previous filesystem signature on /dev/mdX when we construct RAID volume multiple time on same disks).

Also dropped -f option from mkfs.xfs, it will check wipefs is working as we
expected.

Fixes #13737

Signed-off-by: Takuya ASADA <syuu@scylladb.com>

Closes #13738

(cherry picked from commit fdceda2)
denesb pushed a commit that referenced this issue Jul 11, 2023
The discussion on the thread says, when we reformat a volume with another
filesystem, kernel and libblkid may skip to populate /dev/disk/by-* since it
detected two filesystem signatures, because mkfs.xxx did not cleared previous
filesystem signature.
To avoid this, we need to run wipefs before running mkfs.

Note that this runs wipefs twice, for target disks and also for RAID device.
wipefs for RAID device is needed since wipefs on disks doesn't clear filesystem signatures on /dev/mdX (we may see previous filesystem signature on /dev/mdX when we construct RAID volume multiple time on same disks).

Also dropped -f option from mkfs.xfs, it will check wipefs is working as we
expected.

Fixes #13737

Signed-off-by: Takuya ASADA <syuu@scylladb.com>

Closes #13738

(cherry picked from commit fdceda2)
@denesb
Copy link
Contributor

denesb commented Jul 11, 2023

Backported to 5.2, 5.1, 2022.1. Already part of 5.3.
@syuu1228 Backport to 2021.1 is not clean, I will need a backport PR for that.

@denesb denesb removed backport/5.2 Issues that should be backported to 5.2 branch once they'll be fixed Requires-Backport-to-5.1 labels Jul 11, 2023
@DoronArazii DoronArazii modified the milestones: 5.4, 5.3 Jul 11, 2023
denesb pushed a commit that referenced this issue Jul 11, 2023
The discussion on the thread says, when we reformat a volume with another
filesystem, kernel and libblkid may skip to populate /dev/disk/by-* since it
detected two filesystem signatures, because mkfs.xxx did not cleared previous
filesystem signature.
To avoid this, we need to run wipefs before running mkfs.

Note that this runs wipefs twice, for target disks and also for RAID device.
wipefs for RAID device is needed since wipefs on disks doesn't clear filesystem signatures on /dev/mdX (we may see previous filesystem signature on /dev/mdX when we construct RAID volume multiple time on same disks).

Also dropped -f option from mkfs.xfs, it will check wipefs is working as we
expected.

Fixes #13737

Signed-off-by: Takuya ASADA <syuu@scylladb.com>

Closes #13738

(cherry picked from commit fdceda2)
@DoronArazii
Copy link

Backported.
Removing the label.

@DoronArazii DoronArazii modified the milestones: 5.3, 5.4 Jul 25, 2023
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging a pull request may close this issue.

4 participants