Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Update manager.inc to allow mount of labeled filesystems #543

Closed
wants to merge 1 commit into from
Closed

Update manager.inc to allow mount of labeled filesystems #543

wants to merge 1 commit into from

Conversation

burnbasket
Copy link

  1. Reordered checks by specicifity
    is_devicefile will catch anything in /dev and its subdirectories, so I think it should be done last. (Sidenote: Maybe you wanted ^/dev/[^\/]+$ in functions.inc?)
  2. Label will never match the full /dev/disk/by-label path, so I propose to match only the string after the last slash. (This might be beneficial for the uuid part, too, but I didn't make this change here.)
  3. The final else statement only handled the "label" case (and didn't in case the label wasn't on its own ;-) making it unnecessary. The proposed change should handle isolated labels as well as labels with path, because labels can't contain slashes afaik.
  4. I didn't use else-if constructs, because the label case has a sub condition with a regex match and I didn't know how to do that as an && in the first condition. To keep things consistent used !found in all conditions except obviously the first.

I've never coded in PHP before, so please forgive me if it looks clumsy.

  • References issue
  • Includes tests for new functionality or reproducer for bug

1) Reordered checks by specicifity
is_devicefile will catch anything in /dev and its subdirectories, so I think it should be done last. (Sidenote: Maybe you wanted ^\/dev\/[^\/]+$ in functions.inc?)
2) Label will never match the full /dev/disk/by-label path, so I propose to match only the string after the last slash. (This might be beneficial for the uuid part, too, but I didn't make this change here.)
3) The final else statement only handled the "label" case (and didn't in case the label wasn't on its own ;-) making it unnecessary. The proposed change should handle isolated labels as well as labels with path, because labels can't contain slashes afaik.
4) I didn't use else-if constructs, because the label case has a sub condition with a regex match and I didn't know how to do that as an && in the first condition. To keep things consistent used !found in all conditions except obviously the first.

I've never coded in PHP before, so please forgive me if it looks clumsy.
@claassistantio
Copy link

claassistantio commented Dec 23, 2019

CLA assistant check
All committers have signed the CLA.

@votdev
Copy link
Member

votdev commented Dec 23, 2019

Can you add some example device names in the code to make it more clear what happens here.

Please also add a description why this PR is necessary, currently i do not understand the intention.

Please add the signed-off-by line (Signed-off-by: Frank Mustermann <frank.mustermann@xxx.yyy>).

@burnbasket
Copy link
Author

Mount button threw exception with this device:
/dev/disk/by-label/openmediavaultData
It ended up being handled in the is_devicefile clause, because line 313 in functions.inc matches everything in /dev and its subdirectories.
preg_match('/^/dev/.+$/i', $var)

@burnbasket
Copy link
Author

Both is_fs_uuid and is_devicefile_by in functions.inc are more specific.
I used is_devicefile_by because it was already there.

@votdev
Copy link
Member

votdev commented Dec 23, 2019

How can I reproduce this? I never saw that behavior and the code is really old there.

@burnbasket
Copy link
Author

The only thing I did was create a volume with label, in this case
mkfs.btrfs --label openmediavaultData --metadata raid10 --data raid10 /dev/disk/by-id/ata-TOSHIBA_DT01ACA300_ /dev/disk/by-id/ata-SAMSUNG_HD204UI_ /dev/disk/by-id/ata-SAMSUNG_HD204UI_ /dev/disk/by-id/ata-SAMSUNG_HD103SI_

@burnbasket
Copy link
Author

But the fs really doesn't matter.

@burnbasket
Copy link
Author

burnbasket commented Dec 23, 2019

function is_devicefile($var) { if (FALSE === is_string($var)) return FALSE; return preg_match('/^\/dev\/.+$/i', $var) ? TRUE : FALSE; }
will always match any string starting with /dev/

@burnbasket
Copy link
Author

burnbasket commented Dec 23, 2019

This could be changed by using this instead:/^\/dev\/[^\/]+$/

@votdev
Copy link
Member

votdev commented Dec 23, 2019

Mount button threw exception with this device:
/dev/disk/by-label/openmediavaultData
It ended up being handled in the is_devicefile clause, because line 313 in functions.inc matches everything in /dev and its subdirectories.
preg_match('/^/dev/.+$/i', $var)

That it the intention of this function to check if it is a device path, and that also includes /dev/disk/by... paths. Changing this will break everything in OMV, so this would be a really bad idea.

@burnbasket
Copy link
Author

OK, but in this case, the label case can never be reached.

@votdev
Copy link
Member

votdev commented Dec 23, 2019

Please post the output of blkid and the dir tree of the device, e.g. /dev/... I can not merge this PR before I fully can reproduce this, otherwise it might break other installations.

@burnbasket
Copy link
Author

I can see that.
blkid /dev/disk/by-label/openmediavaultData -o full /dev/disk/by-label/openmediavaultData: LABEL="openmediavaultData" UUID="6f22c380-dc25-424c-bb08-5495048b58dd" UUID_SUB="b8cd5870-c357-4f76-bb87-13a50625786e" TYPE="btrfs"

@burnbasket
Copy link
Author

label is a special case of dev imo

@votdev
Copy link
Member

votdev commented Dec 23, 2019

I need to check the code, but it works for me in the past and there were no bug reports related this till now. I assume it has something to do with BTRFS. I think a device by path link should be resolved to a canonical path when using realpath, so a previous code path should be executed in this case. But as said, I need to check that.
Changing code here is really dangerous, so we need to take care not to break everything.

@burnbasket
Copy link
Author

functions.inc condition
function is_devicefile_by($var) { if (FALSE === is_string($var)) return FALSE; return preg_match('/^\/dev\/disk\/by-\S+\/.+$/i', $var) ? TRUE : FALSE; }
is more specific than
function is_devicefile($var) { if (FALSE === is_string($var)) return FALSE; return preg_match('/^\/dev\/.+$/i', $var) ? TRUE : FALSE; }

@burnbasket
Copy link
Author

I really get that.

@votdev votdev added the 5.x label Dec 23, 2019
@burnbasket
Copy link
Author

A workaround would be to use one of the constituent drives instead of the label, but that didn't jive with me.

@burnbasket
Copy link
Author

lsblk NAME MAJ:MIN RM SIZE RO TYPE MOUNTPOINT sda 8:0 0 167.7G 0 disk ├─sda1 8:1 0 512M 0 part /boot/efi ├─sda2 8:2 0 159.5G 0 part / └─sda3 8:3 0 7.7G 0 part [SWAP] sdb 8:16 1 1.8T 0 disk sdc 8:32 1 1.8T 0 disk sdd 8:48 1 931.5G 0 disk sde 8:64 1 2.7T 0 disk

@burnbasket
Copy link
Author

tree /dev/disk /dev/disk ├── by-id │   ├── ata-INTEL_SSDSC2CT180A3____ -> ../../sda │   ├── ata-INTEL_SSDSC2CT180A3____-part1 -> ../../sda1 │   ├── ata-INTEL_SSDSC2CT180A3____-part2 -> ../../sda2 │   ├── ata-INTEL_SSDSC2CT180A3____-part3 -> ../../sda3 │   ├── ata-SAMSUNG_HD103SI____ -> ../../sdd │   ├── ata-SAMSUNG_HD204UI____ -> ../../sdc │   ├── ata-SAMSUNG_HD204UI____ -> ../../sdb │   ├── ata-TOSHIBA_DT01ACA300____ -> ../../sde │   ├── wwn-0x5000039fe6ed496c -> ../../sde │   ├── wwn-0x5001517bb29361b4 -> ../../sda │   ├── wwn-0x5001517bb29361b4-part1 -> ../../sda1 │   ├── wwn-0x5001517bb29361b4-part2 -> ../../sda2 │   ├── wwn-0x5001517bb29361b4-part3 -> ../../sda3 │   ├── wwn-0x50024e90013839d6 -> ../../sdd │   ├── wwn-0x50024e90040f7928 -> ../../sdc │   └── wwn-0x50024e90040f7935 -> ../../sdb ├── by-label │   └── openmediavaultData -> ../../sde ├── by-partuuid │   ├── 28529ae3-3cac-4860-85c0-c744453f83c6 -> ../../sda1 │   ├── 5a55c57b-283b-4cf0-95b2-11228f578b03 -> ../../sda2 │   └── 630ee4a7-95fb-46c1-b82a-212613e5bba5 -> ../../sda3 ├── by-path │   ├── pci-0000:00:13.0-ata-1 -> ../../sda │   ├── pci-0000:00:13.0-ata-1-part1 -> ../../sda1 │   ├── pci-0000:00:13.0-ata-1-part2 -> ../../sda2 │   ├── pci-0000:00:13.0-ata-1-part3 -> ../../sda3 │   ├── pci-0000:01:00.0-ata-1 -> ../../sdb │   ├── pci-0000:01:00.0-ata-3 -> ../../sdc │   └── pci-0000:01:00.0-ata-4 -> ../../sde └── by-uuid ├── 25cf0f95-165a-4551-a6c5-38fa3da9a4a2 -> ../../sda3 ├── 470cd4dc-d89a-4b0e-a56f-d5e7b5b2abb4 -> ../../sda2 ├── 607C-4827 -> ../../sda1 └── 6f22c380-dc25-424c-bb08-5495048b58dd -> ../../sde

@burnbasket
Copy link
Author

burnbasket commented Dec 23, 2019

On the topic of canonical file paths, why not make the (unchanging) device ids or wwns or the (editable) labels the main anchor instead of the almost random /dev/sd* device order?

I realize you're on the same page as the kernel there, and you can't argue with the kernel, so these boots are made for walking.

Devices keep changin' when they ought to be samin' and some of these days these boots are walking all over us.

:-)

Happy Holidays!

@votdev
Copy link
Member

votdev commented Dec 23, 2019

On the topic of canonical file paths, why not make the (unchanging) device ids or wwns or the (editable) labels the main anchor instead of the almost random /dev/sd* device order?

I realize you're on the same page as the kernel there, and you can't argue with the kernel, so these boots are made for walking.

Devices keep changin' when they ought to be samin' and some of these days these boots are walking all over us.

:-)

Happy Holidays!

Because of that OMV prefers predictable device names for storing in the database and for configuration files, but canonical device names are the best to work at runtime because they are simple.

@burnbasket
Copy link
Author

I did some quickie tests with unmodded an modified filesystem/manager.inc:
TL;DR: For the tested scanarios, original and modded version behaved precisely the same.

Scenarios:
Ext2 filesystem on partition without label
Ext2 filesystem on partition with label
Ext2 filesystem on unpartitioned device without label
Ext2 filesystem on unpartitioned device with label

All these scenarios where performed with the original and with the modified version of filesystem/manager.inc, with poweroff and new boot between changes of this file.

For each scenario, the disk on which the test was performed was initialised with 8MB of zeros, then partitioned or not as required by the scenario and a new filesystem created on the cli. Where required, the filesystem was then labeled.

More details in the next post.

@burnbasket
Copy link
Author

burnbasket commented Dec 23, 2019

Full version of the tests attached.
omvTest20191224T004230.txt

@votdev
Copy link
Member

votdev commented Dec 23, 2019

You still have not described the intention of the PR. Could you please do that? I'm now more irritated when i look into your attached file. I can see a Salt stack trace? Are you trying to fix that? If yes, why do you think this can be fixed in the PHP backend code that is not related to the Python code used by Salt?

You have raised more question than you've answered.

@votdev
Copy link
Member

votdev commented Dec 24, 2019

tree /dev/disk /dev/disk ├── by-id │ ├── ata-INTEL_SSDSC2CT180A3____ -> ../../sda │ ├── ata-INTEL_SSDSC2CT180A3____-part1 -> ../../sda1 │ ├── ata-INTEL_SSDSC2CT180A3____-part2 -> ../../sda2 │ ├── ata-INTEL_SSDSC2CT180A3____-part3 -> ../../sda3 │ ├── ata-SAMSUNG_HD103SI____ -> ../../sdd │ ├── ata-SAMSUNG_HD204UI____ -> ../../sdc │ ├── ata-SAMSUNG_HD204UI____ -> ../../sdb │ ├── ata-TOSHIBA_DT01ACA300____ -> ../../sde │ ├── wwn-0x5000039fe6ed496c -> ../../sde │ ├── wwn-0x5001517bb29361b4 -> ../../sda │ ├── wwn-0x5001517bb29361b4-part1 -> ../../sda1 │ ├── wwn-0x5001517bb29361b4-part2 -> ../../sda2 │ ├── wwn-0x5001517bb29361b4-part3 -> ../../sda3 │ ├── wwn-0x50024e90013839d6 -> ../../sdd │ ├── wwn-0x50024e90040f7928 -> ../../sdc │ └── wwn-0x50024e90040f7935 -> ../../sdb ├── by-label │ └── openmediavaultData -> ../../sde ├── by-partuuid │ ├── 28529ae3-3cac-4860-85c0-c744453f83c6 -> ../../sda1 │ ├── 5a55c57b-283b-4cf0-95b2-11228f578b03 -> ../../sda2 │ └── 630ee4a7-95fb-46c1-b82a-212613e5bba5 -> ../../sda3 ├── by-path │ ├── pci-0000:00:13.0-ata-1 -> ../../sda │ ├── pci-0000:00:13.0-ata-1-part1 -> ../../sda1 │ ├── pci-0000:00:13.0-ata-1-part2 -> ../../sda2 │ ├── pci-0000:00:13.0-ata-1-part3 -> ../../sda3 │ ├── pci-0000:01:00.0-ata-1 -> ../../sdb │ ├── pci-0000:01:00.0-ata-3 -> ../../sdc │ └── pci-0000:01:00.0-ata-4 -> ../../sde └── by-uuid ├── 25cf0f95-165a-4551-a6c5-38fa3da9a4a2 -> ../../sda3 ├── 470cd4dc-d89a-4b0e-a56f-d5e7b5b2abb4 -> ../../sda2 ├── 607C-4827 -> ../../sda1 └── 6f22c380-dc25-424c-bb08-5495048b58dd -> ../../sde

I'm sorry, i can really get no information out of this.

Please explain the problem and how to reproduce it. What i know out of the current information is:

  • You can not mount a BTRFS filesystem which has a label.

I need:

  • The error reported by the RPC in the UI error dialog
  • The blkid output (without any command parameter)
  • If you know the device file, then i want to have
# ls -alh /dev/disk/by-xxx/foo
# ls -alh /dev/disk/by-xxx/

@votdev
Copy link
Member

votdev commented Dec 24, 2019

Can not reproduce this behavior.

Peek 2019-12-24 01-17

root@omv5box:~# cat /etc/fstab
proc /proc proc defaults 0 0
UUID=90ee6298-385f-4841-bfdc-8b1e0e0ae5c1 / ext4 errors=remount-ro 0 1
# >>> [openmediavault]
/dev/disk/by-label/omvTestLabel00		/srv/dev-disk-by-label-omvTestLabel00	ext2	defaults,nofail,user_xattr,noexec,acl	0 2
# <<< [openmediavault]

@votdev
Copy link
Member

votdev commented Dec 24, 2019

I've checked the original code again and IMO you didn't have understood it correct. The getBackendById method can be called with a filesystem label as argument (see method docu). That's why the last else path is checking for the label. Extracting the label from a device path is dangerous and prohibited if you do not unescape it, otherwise you will not get the result you expect. That's why your approach to extract the fs label from the device file is not the way to go. If you call the method with a device file like /dev/disk/by-label/omvTestLabel00 then the line else if (realpath($id) == realpath($enumv['devicefile'])) (see line 138) should ALWAYS match if the filesystem is detected by $enums = $this->enumerate(); because when you compare the canonical paths of both device files they should always match, either the enum method returns a different device file for the filesystem, e.g. /dev/disk/by-uuid/2c0cf06e-d5de-45d7-b6fc-675aa90a05a9.

root@omv5box:~# blkid
/dev/vda1: UUID="90ee6298-385f-4841-bfdc-8b1e0e0ae5c1" TYPE="ext4" PARTUUID="c3b3b272-01"
/dev/sda: LABEL="omvTestLabel00" UUID="2c0cf06e-d5de-45d7-b6fc-675aa90a05a9" TYPE="ext2"
root@omv5box:~# ls -alh /dev/disk/by-label/omvTestLabel00
lrwxrwxrwx 1 root root 9 Dec 24 00:16 /dev/disk/by-label/omvTestLabel00 -> ../../sda
root@omv5box:~# ls -alh /dev/disk/by-uuid/2c0cf06e-d5de-45d7-b6fc-675aa90a05a9
lrwxrwxrwx 1 root root 9 Dec 24 00:16 /dev/disk/by-uuid/2c0cf06e-d5de-45d7-b6fc-675aa90a05a9 -> ../../sda

realpath(/dev/disk/by-uuid/2c0cf06e-d5de-45d7-b6fc-675aa90a05a9) === realpath(/dev/disk/by-label/omvTestLabel00)

IMO the current implementation is correct and the problem comes from elsewhere.

@votdev
Copy link
Member

votdev commented Dec 24, 2019

I will close this PR because i do not see a problem here. We should open a bug report instead, describing the problem, providing all available information and see how to fix the issue.

@votdev votdev closed this Dec 24, 2019
@votdev votdev added the reject label Dec 24, 2019
@burnbasket
Copy link
Author

Thanks, I'll do that.

@burnbasket
Copy link
Author

burnbasket commented Dec 24, 2019

Screenshots from unmodded fresh install.
00_initializeDisksAndCreateFilesystem
01_filesystemDetectedInOmv
02_mountErrorInOmv
03_mountErrorDetailsInOmv
04_lsblkOnThisMachine
05_diskTreeOnThisMachine
06_mountOutputAfterGuiMountError

@votdev
Copy link
Member

votdev commented Dec 24, 2019

Please post the output of # blkid.

Please, no screenshots, the console supports copy & paste (without loosing the format).

@burnbasket
Copy link
Author

burnbasket commented Dec 24, 2019

root@openmediavault:# blkid
/dev/sda1: UUID="710D-6276" TYPE="vfat" PARTUUID="34b64675-37c5-4609-bf7b-3a7583581c2c"
/dev/sda2: UUID="540e1951-019c-4745-88dd-f91788f2521d" TYPE="ext4" PARTUUID="0ed5021c-d409-4f2b-b83e-9root@openmediavault:
# blkid
/dev/sda1: UUID="710D-6276" TYPE="vfat" PARTUUID="34b64675-37c5-4609-bf7b-3a7583581c2c"
/dev/sda2: UUID="540e1951-019c-4745-88dd-f91788f2521d" TYPE="ext4" PARTUUID="0ed5021c-d409-4f2b-b83e-92c78c76a45a"
/dev/sda3: UUID="b0458068-f10f-4be9-ac01-2581d5db6990" TYPE="swap" PARTUUID="8e762cad-31b1-4c3f-8503-29bcd80cf353"
/dev/sdb: LABEL="openmediavaultData" UUID="b1ce5d31-00c0-4542-a9b1-295c39618b03" UUID_SUB="f401beba-fd9b-4ef5-ad68-c2b37419c40b" TYPE="btrfs"
/dev/sdc: LABEL="openmediavaultData" UUID="b1ce5d31-00c0-4542-a9b1-295c39618b03" UUID_SUB="12e44150-85d5-4c9f-9b56-0a012a8755f1" TYPE="btrfs"
/dev/sdd: LABEL="openmediavaultData" UUID="b1ce5d31-00c0-4542-a9b1-295c39618b03" UUID_SUB="71810b69-f622-4c8f-b6fa-5247d2e98161" TYPE="btrfs"
/dev/sde: LABEL="openmediavaultData" UUID="b1ce5d31-00c0-4542-a9b1-295c39618b03" UUID_SUB="92c1c012-5130-448e-8c9b-63c86410b9b9" TYPE="btrfs"
/dev/sda3: UUID="b0458068-f10f-4be9-ac01-2581d5db6990" TYPE="swap" PARTUUID="8e762cad-31b1-4c3f-8503-29bcd80cf353"
/dev/sdb: LABEL="openmediavaultData" UUID="b1ce5d31-00c0-4542-a9b1-295c39618b03" UUID_SUB="f401beba-fd9b-4ef5-ad68-c2b37419c40b" TYPE="btrfs"
/dev/sdc: LABEL="openmediavaultData" UUID="b1ce5d31-00c0-4542-a9b1-295c39618b03" UUID_SUB="12e44150-85d5-4c9f-9b56-0a012a8755f1" TYPE="btrfs"
/dev/sdd: LABEL="openmediavaultData" UUID="b1ce5d31-00c0-4542-a9b1-295c39618b03" UUID_SUB="71810b69-f622-4c8f-b6fa-5247d2e98161" TYPE="btrfs"
/dev/sde: LABEL="openmediavaultData" UUID="b1ce5d31-00c0-4542-a9b1-295c39618b03" UUID_SUB="92c1c012-5130-448e-8c9b-63c86410b9b9" TYPE="btrfs"
Screenshot_20191224_151010
2c78c76a45a"

votdev added a commit to votdev/openmediavault that referenced this pull request Dec 24, 2019
Signed-off-by: Volker Theile <votdev@gmx.de>
votdev added a commit to votdev/openmediavault that referenced this pull request Dec 24, 2019
Signed-off-by: Volker Theile <votdev@gmx.de>
votdev added a commit to votdev/openmediavault that referenced this pull request Dec 24, 2019
Signed-off-by: Volker Theile <votdev@gmx.de>
@votdev
Copy link
Member

votdev commented Dec 24, 2019

The real issue here is way more complicated than you tried to fix with your PR. BTRFS has a strange behaviour with canonical device files and what the kernel reports in the mount table.

@burnbasket
Copy link
Author

Oh boy.
Well, for the time being my hack works on the old beast I was playing around with on my vacation, which is all I originally really intended.
I just thought maybe someone else might benefit if I upstreamed it and I was unaware of the development repository.

@burnbasket
Copy link
Author

BTW: Thanks for the fast reaction!
In case this is hardware related (the machine I am running this on is made from spit, baling wire and stuff a dumpster diver could have pulled out of the trash, after all), I'm attaching the output of
lspci -vvv
lspciVvv.txt

votdev added a commit to votdev/openmediavault that referenced this pull request Dec 25, 2019
Signed-off-by: Volker Theile <votdev@gmx.de>
votdev added a commit to votdev/openmediavault that referenced this pull request Dec 25, 2019
Signed-off-by: Volker Theile <votdev@gmx.de>
votdev added a commit that referenced this pull request Dec 25, 2019
Issue #543: Fix BTRFS mounting issue.
votdev added a commit that referenced this pull request Dec 25, 2019
Signed-off-by: Volker Theile <votdev@gmx.de>
(cherry picked from commit f2a80d8)
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
Development

Successfully merging this pull request may close these issues.

None yet

3 participants