Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Btrfs subvolumes support #9

Closed
Chrysostomus opened this issue Feb 6, 2017 · 16 comments
Closed

Btrfs subvolumes support #9

Chrysostomus opened this issue Feb 6, 2017 · 16 comments

Comments

@Chrysostomus
Copy link
Owner

@Chrysostomus Chrysostomus commented Feb 6, 2017

I wish to port btrfs subvolume functions from previous manjaro cli-installer, so that installation to btrfs subvolumes can be done without manual mounting.

@fhdk

This comment has been minimized.

Copy link
Collaborator

@fhdk fhdk commented Feb 18, 2017

I you no one else volunteered I could give it a try.

@Chrysostomus

This comment has been minimized.

Copy link
Owner Author

@Chrysostomus Chrysostomus commented Feb 18, 2017

Thanks! Greatly appreciated!

I poached the btrfs subvolume functions from the old cli-installer https://github.com/Chrysostomus/manjaro-architect/blob/master/bin/btrfs-subvol-functions.in
They might rely on some other stuff that has not yet been ported though.

@fhdk

This comment has been minimized.

Copy link
Collaborator

@fhdk fhdk commented Feb 21, 2017

I must conclude I am a lousy shell scripter :) But you manage all right :)

This might take me some time cause I can't get the my brain about the logic.

C#, javascript, Python was a lot easier to grasp than shell :)

@Chrysostomus

This comment has been minimized.

Copy link
Owner Author

@Chrysostomus Chrysostomus commented Feb 21, 2017

It's okay. The schedule for this feature is "before summer"

The logic I would think that we need is

  1. after choosing mount options for partition, if the chosen partition is btrfs, start subvolume creation function
  2. subvolume creation function: ask user if they want to create a subvolume on this partition. Keep asking until they choose "done".
  • if user chooses yes, ask for name of the subvolume. In next window, ask mountpoint for the subvolume.
  • mount chosen mountpoint to chosen directory with mount options from previous menu
  • repeat loop
  1. when loop exits, continue to next

No idea how to handle raid options, but I find it a lesser priority

@fhdk

This comment has been minimized.

Copy link
Collaborator

@fhdk fhdk commented Feb 21, 2017

@Chrysostomus

This comment has been minimized.

Copy link
Owner Author

@Chrysostomus Chrysostomus commented Feb 21, 2017

Yes, I install half of the test systems on btrfs too. But if user wants to install on btrfs subvolumes to be able to take snapshots, he needs to create and mount them manually separate of the installer.

Idea would be to to integrate subvolume creation to existing btrfs functionality.

@fhdk

This comment has been minimized.

Copy link
Collaborator

@fhdk fhdk commented Feb 21, 2017

@Chrysostomus

This comment has been minimized.

Copy link
Owner Author

@Chrysostomus Chrysostomus commented Feb 21, 2017

Subvolumes, snapshots and compression are stable features. Raid is too, except for raid5/6. The unstable ones are fsck, dedublication, quotas and raid5/6. Nothing in installer guides user to try unstable features.

And swap files cannot be used to.

Most gotchas of btrfs are already automatically handled handled by the installer. Incompatible settings are disabled from grub and fsck is removed from mkinitcpio.conf.

@fhdk

This comment has been minimized.

Copy link
Collaborator

@fhdk fhdk commented Feb 21, 2017

I continued your thoughts from

The logic I would think that we need is

select_filesystem

  • btrfs on partition
    creating btrfs takes a long time
    display message instead of buttons

https://wiki.archlinux.org/index.php/Btrfs#Mounting_subvolumes
Simplify subvolume layouts by not using the toplevel(ID=5) as /.
Instead create a subvolume on the subvolume and use that as /

the flow
format partition with btrfs
use selected mount options

  • what sane default mount options for annie doe?
    noatime
    recovery
    space_cache (default in kernel?)
    most recommendations is not to use compression with /home
    most recommendations is to use compression with /
    but to achieve both it must be separate disks as mount options for the first mount takes precedence

mkdir -p /mnt/btrfs
mount /dev/sdX2 /btrfs

It would be sane - in case of average user - to create a default structure for system if user choose btrfs?
Just to complicate it - also the experienced user should be able to mess with the basics?

  • create subvolume
    btrfs create subvolume /mnt/btrfs/system
    btrfs create subvolume /mnt/btrfs/system/root
    btrfs create subvolume /mnt/btrfs/system/home

  • ask for more subvolumes - if yes create
    btrfs create subvolume /mnt/btrfs/user
    create and subsequent volumes
    ask for snapshot or subvolume
    btrfs create subvolume /mnt/btrfs/user/new_vol1
    btrfs create snapshot /mnt/btrfs/user/new_vol2
    etc

  • mount system subvolumes
    create install_root
    mkdir -p /mnt/install_root
    mount -t btrfs -o subvol=system/root /dev/sdxY /mnt/install_root
    mkdir /mnt/install_root/boot[/efi]
    mkdir /mnt/install_root/home
    mount -t btrfs -o subvol=/system/home /dev/sdx/ /mnt/install_root/home
    mount /dev/sdX1 /mnt/install_root/boot[/efi]

  • adding/removing subvolumes requires mount of the basic partition

  • using the subvolumes can be achieved without mounting the basic partition
    a user is required to have knowledge about btrfs to benefit from it
    are snapshots done automagically?

@fhdk

This comment has been minimized.

Copy link
Collaborator

@fhdk fhdk commented Feb 23, 2017

@Chrysostomus
I must give in - I cannot get the bigger picture - it is much too confusing for me with variables where I do not know where the value comes from and why.

This shell scripting takes a special brain which I do not have.

At least I tried - keep up the good work 😺 t

@Chrysostomus

This comment has been minimized.

Copy link
Owner Author

@Chrysostomus Chrysostomus commented Feb 23, 2017

Thanks, I appreciate the effort!

@Chrysostomus

This comment has been minimized.

Copy link
Owner Author

@Chrysostomus Chrysostomus commented Mar 24, 2017

I noticed now that the Acidburn0zzz branch has btrfs implemented. I'll take a look if we can import that...

@Chrysostomus

This comment has been minimized.

Copy link
Owner Author

@Chrysostomus Chrysostomus commented Apr 15, 2017

I have been thinking about this. It feels too complicated to properly implement btrfs subvolumes in the current mount partitions phase, because the phase is split across several functions and the times I can work with this are few and far between.

I think it would be, for the time being, make a separate "mount btrfs subvolumes" menu item. It would be ran after basic partition mounting. It would

  1. check if /mnt is on btrfs volume. If not, refuse to do anything and give explanation.
  2. Offer automatic and manual mode. Automatic creates and mounts volumes @, @home, @var, @cache and @snapshots.
  3. Manual mode ask for names for subvolumes, separated by spaces. Pressing Enter creates those volumes.
  4. After subvolume creation unmount btrfs root and loop to ask a mountpoint for each of these subvolumes. Use mount options of the original btrfs root.
  5. In the end, maybe offer to add devices to the btrfs volume?

What do you guys think? @oberon2007 @papajoker ?

@Chrysostomus

This comment has been minimized.

Copy link
Owner Author

@Chrysostomus Chrysostomus commented Apr 16, 2017

something like this:

btrfs_subvolumes()

{
    if [[ $(lsblk -lno FSTYPE,MOUNTPOINT | awk '/ \/mnt$/ {print $1}') == btrfs ]]; then 
        DIALOG " Your root volume is not formatted in btrfs " --infobox "\nMount partitions first!\n " 0 0
        return 0
    fi
    #1) save mount options and name of the root partition 
    mount | grep "on /mnt " | grep -Po '(?<=\().*(?=\))' > /tmp/.root_mount_options
    lsblk -lno MOUNTPOINT,NAME | awk '/^\/mnt / {print $2}' > /tmp/.root_partition
    #2) choose automatic or manual mode
    DIALOG " Choose mode " --menu "\n$_Note\nAutomatic mode is deigned to\nallow integration with snapper,\nnon-recursive snapshots,\nseparating system and user data\nand restoring snapshots without losing data. " 0 0 2 \
      "1" "automatic" \
      "2" "manual" 2>/tmp/.subvol_mode

    if [[ $(cat /tmp/.subvol_mode) != "" ]]; then
        if [[ $(cat /tmp/.subvol_mode) -eq 2 ]]; then
            # Create subvolumes manually
            DIALOG " Create subvolumes " --inputbox "\nInput names of the subvolumes separated by spaces." 0 0 "" 2>/tmp/.subvols || return 0
            cd /mnt
            for subvol in $(cat /tmp/subvols); do
                btrfs subvolume create $subvol
            done
            cd
            # Mount subvolumes
            umount /mnt
            for sub in $(btrfs subvolume list -p /mnt); do
                DIALOG " Mount this subvolume " --inputbox "\nInput mountpoint of the subvolume\nas it would appear in installed system\n(without prepending /mnt)\n." 0 0 "" 2>/tmp/.mountp || return 0
                mount -o "$(cat /tmp/.root_mount_options)",subvol="$sub" /dev/"$(cat /tmp/.root_partition)" /mnt"$(cat /tmp/.mountp)"
            done
        else
            DIALOG " This creates subvolumes @ for /,\n@home for /home,\n@cache for /var/cache and subvolume snapshots. " --infobox "\nMount partitions first!\n " 0 0
            # Create subvolumes automatically
            cd /mnt
            btrfs subvolume create @
            btrfs subvolume create @home
            btrfs subvolume create @cache
            btrfs subvolume create snapshots
            cd
            # Mount subvolumes
            umount /mnt
            mount -o "$(cat /tmp/.root_mount_options)",subvol=@ /dev/"$(cat /tmp/.root_partition)" /mnt
            mkdir -p /mnt/home
            mkdir -p /mnt/var/cache
            mount -o "$(cat /tmp/.root_mount_options)",subvol=@home /dev/"$(cat /tmp/.root_partition)" /mnt/home
            mount -o "$(cat /tmp/.root_mount_options)",subvol=@cache /dev/"$(cat /tmp/.root_partition)" /mnt/var/cache
        fi
    else
        return 0
    fi
}
@Chrysostomus

This comment has been minimized.

Copy link
Owner Author

@Chrysostomus Chrysostomus commented Apr 16, 2017

Making progress with this after moving it within the root volume mounting action. At least automatic mode seems to work now.

@Chrysostomus

This comment has been minimized.

Copy link
Owner Author

@Chrysostomus Chrysostomus commented Apr 17, 2017

Pushed now to the online repo. Automatic and manual modes are still a bit rudimentary, but they seem to work somewhat...

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
Linked pull requests

Successfully merging a pull request may close this issue.

None yet
2 participants
You can’t perform that action at this time.