New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
SLE15 on ppcl: mkfs.xfs fails if sunit > 0 but swidth = 0 #1998
Comments
Hello @abbbi, Do you know if this system was upgraded from older (SLES12, SLES11) or clean installed ? Thanks V. |
problem is that swidth is set to 0: -d sunit=64 -d swidth=0 data stripe width (0) must be a multiple of the data stripe unit (64) working around by setting -d swidth=64 in diskrestore.sh System was freshly installed from scratch. |
what about /var/lib/rear/layout/xfs, can you provide it ? V. |
|
Thanks, V. |
Another interesting thing is that it also trys to mount with mount option -o sunit=64, so one has to remove this from the mount command line in diskrestore.sh aswell. I endet up removing the -d sunit=64 -d swidth=0 options from the mkfs.xfs command and the mount option and was able to recover the system then. |
Honestly I don't remember how all this XFS magic work, I'll need to take a closer look, later today ... V. |
Anyhow, if you could upload log created by V. |
Unfortunately it was a customer system which im not remote anymore. I tried to reproduce on a SLES15 system in house on PPC, and it seems the sunit=8 setting on the other system is already an difference. My XFS filesystem (created by the standard sles15 installation) looks like:
Probably thats some SAP related filesystem options. |
These options are probably auto detected by mkfs during filesystem creation. As the system uses Multipath SAN disks only, i think the xfs tools might suspect other options than on a regular filesystem: honestly its the first time i see these options used too :) |
Never mind. Can you tell me what version of XFS is used on SLE15 ? ( I'm unfortunately still missing such VM in my collection.) V. |
its
but as i can see the filesystem sh already has some logic caring about stuff like this: https://github.com/rear/rear/blob/master/usr/share/rear/lib/filesystems-functions.sh#L205 |
Yes, that is my humble work ;-) V. |
FYI: My disklayout.conf:
My sda2.xfs
|
@abbbi I ask because what the YaST installer creates can be different Or was that particular XFS perhaps specifically set up |
Cant say for sure at the moment. It was running SAP HANA so i think chances are high its the SAP4HANA Release. All i have left at the moment ist the os.conf from REAR from the rear backup log:
|
ReaR XFS code must work regardless of Linux distribution. V. |
Im not sure you will hit this problem with a regular SLE15 installation. I have various SLES15 systems around that do not have an XFS formatted with -d sunit=8 -d swidth=0. I even cannot format an volume using these options to be able to get an xfs_info output that causes the problem: sles15ppcl:~ # mkfs.xfs -d sunit=8 -d swidth=0 ./myfile im not quite sure how it was even possible to format it in that way. One thing i could imagine is that the volume was created too small and then has been resized with xfs_growfs so that the width information output by xfs_info doesnt match anymore. Or that the multipath configuration does something strange resulting in an xfs_info output not making sense. |
:-) I'm exactly at the same spot right now... Can't create filesystem where
I'm afraid that this is not possible either, because if you specify sunit > 0 you must specify swidth as well and swidth must be multiple of sunit (and can't be 0) V. |
Not sure about the upgrade, looking at the xfsprogs source this check has been around since 2002. One recent commit from xfsprogs:
not sure we hit an issue in xfs here. I have asked at #xfs in freenode, lets see if i get response. |
response from #xfs
|
here is how to reproduce this situation. The filesystem is only mountable by older kernels such as on sles15 which do not check swidth/sunit parameters during mount. This should help to make REAR at least handle such situations gracefully :)
|
Hello @abbbi, From what I've understood from your communication with David, First I was thinking that ReaR XFS code incorrectly parsed stored output from xfs_info but I was wrong. ReaR handled situation correctly and tried to recreate filesystem as it originally was. This behavior attracted your attention and you've find out that something is not right so you can start further investigation on "why XFS changed its behavior yet another time :-)). At the same time ReaR allowed reasonable fallback (by editing I've assigned labels Discuss / RFC because this topic can be widely and endlessly discussed :-) and other people might have other opinions ... In either way, if you have ideas how current code can be improved don't hesitate to share them. V. |
I still don't understand if this is a transitioning problem or a permanent problem. Or is the problem that after the |
Hello @schlomo, V. |
In theory V. |
@gozora By the way (cf. "by the way" below): We already had some other issues where "rear recover" revealed For an example see my By setting up a disaster recovery procedure with ReaR and by verifying that 'rear recover' actually works to recreate the original system on replacement hardware one gets "by the way" some kind of confirmation that the basic setup of the original system is o.k. because when the basic setup of the original system is broken it is likely that then it is not possible to recreate such a broken original system on replacement hardware. Simply put: Run 'rear mkbackup' on your original system and 'rear recover' on your replacement hardware to verify that your original system is still o.k. In general when someone likes to use ReaR for disaster recovery That SDB article is referenced in our SLE-HA manuals, e.g. cf. Users who do not do that are not supported, But of course we try to help them here at ReaR is by design prepared to let an experienced admin even deal with such Experienced users and system admins can adapt or extend the ReaR scripts to make it work for their particular cases and - if the worst comes to the worst - even temporary quick and dirty workarounds are relatively easily possible. Bottom line: |
@abbbi Is this not a perfect case for a short FAQ? We would like to share it on our ReaR site - see https://github.com/rear/rear.github.com/blob/master/documentation/faq.md (perhaps prepare a PR to make our life easier ;-) |
( finally I won my fight against markdown ;-) |
@gozora for me its some weird case and as we have seen not a problem in REAR. But if you look closely, there seems to be something messed up in the REAR part too. The xfs_info output prints sunit=8 but rear tries to format with: -d sunit=64 not sure if that is an error crafted by the fact that swidth=0, but worth a look anyway, not that we |
@abbbi it is a feature ;-) V. |
@gozora I have one and I like to do a pull request as soon as time permits Hint: |
…ser_issue_1998 Let the user optionally specify mkfs.xfs options if needed to recreate XFS filesystems with different options than before e.g. in MIGRATION_MODE because of different replacement hardware or because the original system was already broken as in #1998 Via the new MKFS_XFS_OPTIONS config variable one can specify mkfs.xfs options for all mkfs.xfs calls and/or via device specific config variables like MKFS_XFS_OPTIONS_SDA2 one can specify mkfs.xfs options only for the mkfs.xfs call for that dvice (e.g. for /dev/sda2 via MKFS_XFS_OPTIONS_SDA2). Device specific config variables take precedence over MKFS_XFS_OPTIONS which takes precedence over the not changed default behaviour which recreates XFS filesystems with same options as on the original system.
With #2005 merged |
Additional note, because as today this problem striked again at another customer, again on PowerPC with SLES15. The introduced MKFS_XFS_OPTIONS property helps during creation of the filesystems, but one has to make sure that the mount options which are passed to the volume are the right ones too. It might happen that the mount options include the sunit=64 option too, resulting in mount issues:
So the disklayout.conf has to be edited before recovery too, to get things working again. |
Relax-and-Recover (ReaR) Issue Template
Fill in the following items before submitting a new issue
(quick response is not guaranteed with free support):
2.4
SLES15 on PPC
Recover works until system tries to create filesystem via MKFS, errors out with:
The text was updated successfully, but these errors were encountered: