Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

OMV5 installation fails on a fresh Debian 10 Buster with btrfs root in a subvolume #564

Closed
Miwer opened this issue Jan 10, 2020 · 17 comments · Fixed by #565
Closed

OMV5 installation fails on a fresh Debian 10 Buster with btrfs root in a subvolume #564

Miwer opened this issue Jan 10, 2020 · 17 comments · Fixed by #565
Labels

Comments

@Miwer
Copy link

@Miwer Miwer commented Jan 10, 2020

Description of issue/question

Installation of OMV5 fails on a new install of Debian 10 Buster, having root filesystem in a btrfs subvolume instead of directly in the root of btrfs.

Having root in a btrfs subvolume, makes it a lot easier to handle creation/rollback of snapshots in relation to major changes. Ubuntu has been using similar btrfs layout since 11.04

This is not default Debian setup, so I'm not sure if this can be categorized as a bug, or a feature request.

Steps to reproduce issue

alternatively...

(end result is the same - root is now in a btrfs subvolume)

apt-get --yes --auto-remove --show-upgraded \
   --allow-downgrades --allow-change-held-packages \
   --no-install-recommends \
   --option Dpkg::Options::="--force-confdef" \
   --option DPkg::Options::="--force-confold" \
   install openmediavault-keyring openmediavault

fails with multiple python related errors during the configuration of openmediavault - example:

[ERROR   ] Rendering exception occurred
Traceback (most recent call last):
  File "/usr/lib/python3/dist-packages/pyudev/device/_device.py", line 220, in from_device_file
    device_type = get_device_type(filename)
  File "/usr/lib/python3/dist-packages/pyudev/_util.py", line 134, in get_device_type
    mode = os.stat(filename).st_mode
FileNotFoundError: [Errno 2] No such file or directory: '/dev/sda2[/myroot]'

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
  File "/usr/lib/python3/dist-packages/salt/utils/templates.py", line 394, in render_jinja_tmpl
    output = template.render(**decoded_context)
  File "/usr/lib/python3/dist-packages/jinja2/asyncsupport.py", line 76, in render
    return original_render(self, *args, **kwargs)
  File "/usr/lib/python3/dist-packages/jinja2/environment.py", line 1008, in render
    return self.environment.handle_exception(exc_info, True)
  File "/usr/lib/python3/dist-packages/jinja2/environment.py", line 780, in handle_exception
    reraise(exc_type, exc_value, tb)
  File "/usr/lib/python3/dist-packages/jinja2/_compat.py", line 37, in reraise
    raise value.with_traceback(tb)
  File "<template>", line 59, in top-level template code
  File "/var/cache/salt/minion/extmods/modules/omv_utils.py", line 177, in get_fs_parent_device_file
    return fs.get_parent_device_file()
  File "/usr/lib/python3/dist-packages/openmediavault/fs/__init__.py", line 163, in get_parent_device_file
    device = pyudev.Devices.from_device_file(context, self.device_file)
  File "/usr/lib/python3/dist-packages/pyudev/device/_device.py", line 223, in from_device_file
    raise DeviceNotFoundByFileError(err)
pyudev.device._errors.DeviceNotFoundByFileError: [Errno 2] No such file or directory: '/dev/sda2[/myroot]'

During handling of the above exception, another exception occurred:
--- snip ---

see more errors, and full console output in attached file.

Additional debug info

Attaching console output from both failed install (ubuntu style btrfs layout with root in a subvolume), and succesful install (default debian btrfs layout without subvolumes) for comparison.

omv5-install-fails-on-root-subvolume.txt
omv5-install-success-no-subvol.txt

@votdev votdev added 4.x 5.x labels Jan 10, 2020
@votdev

This comment has been minimized.

Copy link
Collaborator

@votdev votdev commented Jan 10, 2020

I wouldn't call it a bug, the backend simply does not support that at the moment and expects a block device hosting the root fs. Sadly it's not trivial to improve the backend Python code to support that.

@ryecoaaron

This comment has been minimized.

Copy link
Contributor

@ryecoaaron ryecoaaron commented Jan 10, 2020

makes it a lot easier to handle creation/rollback of snapshots in relation to major changes.

If you need this ability, having the root filesystem on LVM would work.

@votdev

This comment has been minimized.

Copy link
Collaborator

@votdev votdev commented Jan 10, 2020

Can you please postthe output of blkid?

@Miwer

This comment has been minimized.

Copy link
Author

@Miwer Miwer commented Jan 10, 2020

makes it a lot easier to handle creation/rollback of snapshots in relation to major changes.

If you need this ability, having the root filesystem on LVM would work.

Yes that has also been one of my considered options. However with LVM you have to pre-assign dedicated snapshot space in free space in the volume group (and make sure, snapshot size does not exceed this). So this is more or less wasted space, when not in use by snapshots. This is not a problem with btrfs snapshots. However, with ample space available, this should not be a big issue with LVM anyway, one might argue :)

Can you please postthe output of blkid?

root@debian:~# blkid
/dev/sda1: UUID="09b36bf9-e8be-46d8-a8e7-3ae76cccc0f8" TYPE="ext4" PARTUUID="402569cc-01"
/dev/sda2: UUID="2aaedfbb-79a1-48a5-98ee-288d5b4ba14a" UUID_SUB="6e5429de-118d-49cc-a4c6-e92ca3095b8f" TYPE="btrfs" PARTUUID="402569cc-02"
root@debian:~# 
@ryecoaaron

This comment has been minimized.

Copy link
Contributor

@ryecoaaron ryecoaaron commented Jan 10, 2020

So this is more or less wasted space, when not in use by snapshots.

True but that would be good for flash media :) And considering that most OMV OS installs rarely use more than 4 GB (assuming you don't store data on OS drive), 32GB or 64 GB OS media would have plenty of room.

@Miwer

This comment has been minimized.

Copy link
Author

@Miwer Miwer commented Jan 10, 2020

You make a good point ryecoaaron - It's my first OMV install, so I have no idea of how much space I'm going to need on my OS drives. I was actually planning on installing on two old 80GB sata drives in RAID1, but now I'm more hooked on the idea of two 32GB usb drives in a RAID1 setup with openmediavault-flashmemory plugin.

I really would like to do snapshotting before upgrades, or other major changes, so I will have to consider how much space to reserve for that on LVM.

OS drives are not to be used for data storage, but I cannot rule out the need to install other software, if I find it useful. But I'll have a virtual test setup anyway, so I'm not too worried.

votdev added a commit to votdev/openmediavault that referenced this issue Jan 11, 2020
…le() to handle BTRFS subvolumes.

Fixes: openmediavault#564
Signed-off-by: Volker Theile <votdev@gmx.de>
@votdev

This comment has been minimized.

Copy link
Collaborator

@votdev votdev commented Jan 11, 2020

@Miwer Could you please test the pull request above?

@Miwer

This comment has been minimized.

Copy link
Author

@Miwer Miwer commented Jan 11, 2020

@Miwer Could you please test the pull request above?

I would, but I only see 5.2.2 available - is there a testing repository I need to add?

root@debian:~# apt-cache policy openmediavault
openmediavault:
  Installed: (none)
  Candidate: 5.2.2-1
  Version table:
     5.2.2-1 500
        500 https://packages.openmediavault.org/public usul/main amd64 Packages
        100 /var/lib/dpkg/status
root@debian:~#
@Miwer

This comment has been minimized.

Copy link
Author

@Miwer Miwer commented Jan 12, 2020

Ok, I tried to test it in my failed test setup.

I'm not sure if I'm doing this right, but I downloaded the modified init.py, moved it to /usr/lib/python3/dist-packages/openmediavault/fs/ and ran 'dpkg --configure --pending' and it still fails. Console output attached.
tested-new-version.txt

@votdev

This comment has been minimized.

Copy link
Collaborator

@votdev votdev commented Jan 13, 2020

I see, your approach will always fail because the package installation will overwrite the patched file.

Seems the package must be released to be able to test it.

@Miwer

This comment has been minimized.

Copy link
Author

@Miwer Miwer commented Jan 13, 2020

Well I checked the file after running the dpgk command, and it was not overwritten. It's still the patched version I downloaded from #565 - I'm not sure if the package manager holds a copy somewhere else, that it uses during configuring. I can try to check later when I get home.

@votdev votdev closed this in #565 Jan 13, 2020
@votdev

This comment has been minimized.

Copy link
Collaborator

@votdev votdev commented Jan 13, 2020

Package 5.2.3 is in the repository now.

@Miwer

This comment has been minimized.

Copy link
Author

@Miwer Miwer commented Jan 13, 2020

@votdev - still fails with 5.2.3 - see attached console output.

Also I verified with strace, that my previous attempt to copy the file in manually and run dpkg --configure is right. It reads the patched file from the installed location, not some other copy, and it is not overwritten. So from my point of view, the patch did not work.

Anyway - I think I'll be looking at an LVM+ext4 solution instead since, as you already stated, btrfs subvolumes are not really supported, and not trivial to fix. I appreciate the attempted fix though.

5.2.3-still-fails.txt

@votdev

This comment has been minimized.

Copy link
Collaborator

@votdev votdev commented Jan 13, 2020

Arghhh, i found the problem, the regex was incorrect.

votdev added a commit that referenced this issue Jan 13, 2020
@votdev

This comment has been minimized.

Copy link
Collaborator

@votdev votdev commented Jan 13, 2020

Package 5.2.3-2 is in the repository now.

@Miwer

This comment has been minimized.

Copy link
Author

@Miwer Miwer commented Jan 14, 2020

@votdev nope - still fails. Attaching console output.

Another thing: Does your patch take into account btrfs installations on /dev/mapper devices, like mdraid, lvm or encrypted volumes? To me it seems only block devices like /dev/sd* is accounted for, please correct me if I'm wrong.

5.2.3-2_still_fails.txt

Edit: If you need me to test further changes, just post them like the first time. No need to release a new package into public for every change. ;)

@votdev

This comment has been minimized.

Copy link
Collaborator

@votdev votdev commented Jan 15, 2020

The OMV Python library is not that powerful at the moment as the PHP one. For the moment it will be not possible to fix that.
Need to implement a storage layer same to the one in PHP first.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
3 participants
You can’t perform that action at this time.